[resolved] tLdapInput : decoding the objectSID Attribute

[Anonymous]

Hello,
I am using a tLdapInput to read Active Directory attributes. It gets perfectly String attributes such as cn, distinguishedName, ...
However, I cannot handle Microsoft Active Directory objectSid which is a binary type.

When guessing metadata schema, Talend Open Studio detects a "String" type. I have forced the metadata to be byte[] but I have errors "cannot convert byte[] to string".
Following the article http://www.jroller.com/eyallupu/entry/java_jndi_how_to_convert I have developped two routines to handle the String and convert objectSid to S-1-IdentifierAuthority-SubAuthority1-SubAuthority2-...-SubAuthorityn

public static byte[] convertStringToByteArray(String objectSidAsString) {
byte[] theByteArray = objectSidAsString.getBytes();
return theByteArray;
}
but it looks like when I used this method getSIDAsString from http://www.jroller.com/eyallupu/entry/java_jndi_how_to_convert I have an error.
Are there any tricks to implement such as base64 encoding ? If so how ?
Should I use an Object java type for objectSid and use some ByteArrayOutputStream to deal with the object ? I have tried the following routine, but the method getSIDAsString in the article http://www.jroller.com/eyallupu/entry/java_jndi_how_to_convert does not work either.
public static byte[] getBytes(Object obj) throws java.io.IOException{
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(bos);

oos.writeObject(obj);
oos.flush();
oos.close();
bos.close();

byte [] data = bos.toByteArray();
return data;
}
I bet it is an encoding issue but I do not understand what I should do.
Any idea ? Should I have to develop a tLdapInput component handling this case ?
Thanks. Olivier

[Anonymous]

After investigation a bit, I have written the following class:

public class objectSIDConv {
public static String getSIDAsString(String objectSid) {
// Add the 'S' prefix
String strSID = "";
int version;
long authority;
int count;
String rid = "";
strSID = "S";
byte[] SID = new byte;;
byte[] defaultBytes;
String roundTrip ="";

SID = objectSid.getBytes();
// get version

version = SID;
strSID = strSID + "-" + Integer.toString(version);
for (int i=2; i<=7; i++) {
rid += byte2hex(SID );
}


// get authority
authority = Long.parseLong(rid);
strSID = strSID + "-" + Long.toString(authority);


//next byte is the count of sub-authorities
count = SID&0xFF;
System.out.println("version + Authority + Sub-auth-nb = " +rid + strSID + count);

//iterate all the sub-auths
for (int i=0;i<count;i++) {
rid = "";
for (int j=11; j>7; j--) {
rid += byte2hex(SID);

System.out.println("sub-auth = " + rid);
}
strSID = strSID + "-" + Long.parseLong(rid,16);
}
System.out.println("Final = " + strSID);

// That's it - we have the SID
return strSID;
}
}

BUT I still have errors:
Exception in component tMap_1
java.lang.ArrayIndexOutOfBoundsException: 27
at routines.objectSIDConv.getSIDAsString(objectSIDConv.java:283)
at certfi.test_ad_pmad_0_1.test_AD_PMAD.tLDAPInput_1Process(test_AD_PMAD.java:1314)
at certfi.test_ad_pmad_0_1.test_AD_PMAD.runJobInTOS(test_AD_PMAD.java:1869)
at certfi.test_ad_pmad_0_1.test_AD_PMAD.main(test_AD_PMAD.java:1743)
It looks like that SID = objectSid.getBytes() returns corrupt chain byte[] type. Is there any encoding required such as base64 ???
Many thanks. Olivier

[Anonymous]

Actually I confirm even when the method getSIDAsString ends up without any errors, I have buggy results, objectSID computed with the method are incorrect (different from objectSid that I browse with other tool such as apache directory studio) .
There is an encoding issue ...
Please help !

[Anonymous · Accepted solution]

Okay, finally I have fixed the problem.
objectSid is a byte[] asn.1 encoded. It is not possible to "get" it directly from LDAP in string and then use method such as public static String getSIDAsString(String objectSid). It cannot work.
You must use env.put("java.naming.ldap.attributes.binary","objectSid");
Next problem is : if you open an ldap request per objectSid you will quickly open too much connections. Ldap pool (establishing one session, and sharing this session to perform your ldap requests) is highly recommended.

In order to do so, I have designed one routine and also the following design :
tLDAPInput ----> tJavaRow ---> tLogrow --> tFileOutpuExcel
tJavaRow's code is the following:
output_row.cn = input_row.cn;
output_row.SID = routines.UsePool.LdapPoolSid(input_row.cn);
Add int the output schema an "SID" row with String type.

Here is the code of the routine UsePool (quick & dirty):

import javax.naming.*;
import javax.naming.directory.*;
import java.util.Hashtable;
public class UsePool {
// @SuppressWarnings("unchecked")
public static String LdapPoolSid(String objectName) {
// Set up environment for creating initial context
Hashtable env = new Hashtable(11);
env.put(Context.INITIAL_CONTEXT_FACTORY,"com.sun.jndi.ldap.LdapCtxFactory");

String dirRoot = "/ou= People,DC=mycompany,DC=com";
String adminName = "cn=myadmin, ou=srvaccount, ou=it, DC=mycompany,DC=com";
String adminPassword ="itisasecret";
String ldapURL ="ldap://myserver:389";
//set security credentials, note using simple cleartext authentication
env.put(Context.SECURITY_AUTHENTICATION,"simple");
env.put(Context.SECURITY_PRINCIPAL,adminName);
env.put(Context.SECURITY_CREDENTIALS,adminPassword);

//connect to my domain controller
//env.put(Context.PROVIDER_URL, ldapURL);

// Enable connection pooling
env.put("com.sun.jndi.ldap.connect.pool", "true");
//specify attributes to be returned in binary format
env.put("java.naming.ldap.attributes.binary","objectSID");

try {
//connect to my domain controller
env.put(Context.PROVIDER_URL, ldapURL);

env.put("com.sun.jndi.ldap.netscape.schemaBugs", "true");
// debug only
//env.put("com.sun.jndi.ldap.trace.ber", System.out);

// Create one initial context (Get connection from pool)
DirContext ctx = new InitialDirContext(env);


//Setup the search object. With this, we will pass in a base DN, and search all the child nodes
//In the ldap
SearchControls searchControls = new SearchControls();
searchControls.setSearchScope(SearchControls.SUBTREE_SCOPE);

//specify the LDAP search filter
String searchFilter = "(&(objectClass=user)(cn=" + objectName +"))";


//Specify the Base for the search
String searchBase ="DC=fr,DC=ema,DC=ad,DC=pwcinternal,DC=com";
//initialize counter to total the group members for debug ... not useful here
int totalResults = 0;

//Specify the attributes to return
String returnedAtts[]={"cn","distinguishedName","objectSID"};
searchControls.setReturningAttributes(returnedAtts);
//Search for objects using the filter
NamingEnumeration answer = ctx.search(searchBase, searchFilter, searchControls);
//Loop through the search results
while (answer.hasMoreElements()) {
SearchResult sr = (SearchResult)answer.next();
Attributes attrs = sr.getAttributes();
if (attrs != null) {
try {
for (NamingEnumeration ae = attrs.getAll();ae.hasMore() {
Attribute attr = (Attribute)ae.next();
// System.out.println("Attribute: " + attr.getID());


System.out.println("XXXXXXXXXXX " +attrs.get("cn").get());

byte[] SID = (byte[])attrs.get("objectSID").get();
String strSID = getSIDasStringOfBytes(SID);
System.out.println("YYYYYYYYYYYYYY " + strSID );
return strSID;
}

}
catch (NamingException e) {
System.err.println("Problem listing membership: " + e);
return "objectSID_ERROR";
}
ctx.close();
}
}

} catch (NamingException ne) {
ne.printStackTrace();
System.out.println("Error: " + ne);
return "objectSID_ERROR";
}
return "objectSID_ERROR";
}
public static String getSIDasStringOfBytes(byte[] sid) {
String strSID = "";
int version;
long authority;
int count;
String rid = "";
strSID = "S";

// get version
version = sid;
strSID = strSID + "-" + Integer.toString(version);
for (int i=6; i>0; i--) {
rid += byte2hex(sid );
}

// get authority
authority = Long.parseLong(rid);
strSID = strSID + "-" + Long.toString(authority);

//next byte is the count of sub-authorities
count = sid&0xFF;

//iterate all the sub-auths
for (int i=0;i<count;i++) {
rid = "";
for (int j=11; j>7; j--) {
rid += byte2hex(sid);
}
strSID = strSID + "-" + Long.parseLong(rid,16);
}
return strSID;
}

public static String byte2hex(byte b) {
String ret = Integer.toHexString((int)b&0xFF);
if (ret.length()<2) ret = "0"+ret;
return ret;
}

Bye, Olivier
PS: I bet there is some cleanup to do in the code, any feedback is welcome !

[gorotman]

It works...thanks you saved me...I should be mad investigating about this....

[Anonymous]

Hi gorotman,
Maybe you need to tick "Binay" which you want to get in tLDAPInput advancedSetting.
jjzhou

[gorotman]

Hi jjzhou,
with your suggest now it works!
Thanks.

[_AnonymousUser]

KGfAUH <a href="">gerbjncaogxc</a>, cssuledokkaj, cejypflwsbci,

[_AnonymousUser]

If you are using tFileInputLDIF or tFileOutputLDIF, do the following if not done and check.
1. In Basic settings, for each of the attributes you require check the "Binary" and "Base64"
2. In Advanced Settings, make sure the Encoding format is "UTF-8"
?????. ??????? ??????? ???????? ???????.

[Anonymous]

I tried many things:

1) tLdapInput objectSid as Object -> serialize as byte[] -> convert to string

2) tLdapInput objectSid as byte[] -> convert to string

It did not work until i set the Advanced "binary" setting! Many thanks @jjzhou!

I was then able to get tLdapInput objectSid as byte[] and run routine method to convert to SID string.

The method convertSidToStringSid from https://miromannino.com/blog/convert-a-sid-to-string-with-java/ works. Similarly as do

LdapUtils method convertBinarySidToString(byte[] sid) (ref: https://docs.spring.io/spring-ldap/docs/current/apidocs/org/springframework/ldap/support/LdapUtils.h...)

These are the Routine Libraries i used, and some import statements.and routine method definition.

spring-ldap-core-2.3.2.RELEASE.jar
spring-core-4.3.10.RELEASE.jar
sl4j-log4j12-1.7.6.jar
sl4j-api-1.7.6.jar
sl4j-jdk14-1.6.1.jar

import org.springframework.ldap.support.LdapUtils;
import org.springframework.core.*;
import org.slf4j.*;

public static String convertBinarySidToString(byte[] sid){
  String sidStr = LdapUtils.convertBinarySidToString(sid);
  return sidStr;
}