Salesforce Managed Package - Development vs. Production

[Anonymous]

Wondering if anyone has experience developing jobs that work with Salesforce managed packages in both development and production environments. The challenge is as follows:
With Salesforce, one can create a Managed Package and list it through Salesforce's App Exchange. This package can then be installed in any Salesforce org. With such packages, the custom object names get prefixed with a namespace that is unique to that package. E.g. someone selling a financial app could have a namespace called FinGold. So, a custom object called Commodity_Price__c would actually be FinGold__Commodity_Price__c when the managed package is installed in an org. However, the developer of the app, when working in a development environment, would see the custom object name as Commodity_Price__c i.e. without the namespace prefix. Every field of the custom object, likewise, gets the namespace prefix. There are multiple problems this creates when developing a transform. One is the query you would run. In production, it would need the namespace prefix, but in development, it wouldn't. You might think of using context variables and custom SOQL queries to address this. That may work for the object name, but with tens of fields in the object (or hundreds across multiple custom objects), it quickly becomes impossible to create and maintain queries. Even if you were to somehow manage this complexity, how do you account for the namespace prefix in other components? E.g. how do you handle no prefix in development, and a prefix in production, in tMap?
Does TalenD offer a simple way to deal with this complexity? I could export the job or jobs and do a search and replace using regex in the various TalenD files. I have tried it, and it works, but obviously, it isn't a scalable, reliable, or maintainable approach for deploying complex jobs.
Thanks,
Sumit

[willm1]

Sumit - 

I've used packaged apps on SF (Service Max) and have not run into the issue you're describing. The SF Sandbox - which is your development environment - is a copy of Production. And the objects you created in Production should come across to your sandbox as they were in Production. 

Developer SandboxDeveloper sandboxes are intended for coding and testing. These environments include a copy of your production organization’s metadata, or Setup data.

I don't want to sound presumptuous here, but the first question is - is the developer who installed and configured the app, do this separately in Production, then in your Sandbox or Dev? If so, there could be differences in your schemas. If the sandbox is a straight copy, I'd expect the schemas to be identical.

In the example I quoted above, yes, all custom objects in the app were prefixed with ServiceMax (or a close-enough abbreviation). But fields didn't have the prefix - because they didn't need to. Once you referenced the table (API Object) to read from or write to, you'd specify the fields to extract or update for that object.

Check with your Admin if your sandbox is an image of production... And let me know... I'm always learning
Will

[Anonymous]

Will,
Thanks for your response. We are in a slightly different situation. Let me provide more details.
We, like ServiceMax, are a Salesforce ISV partner with a Managed app of our own that we sell. However, our app is not an add-on, or extension of CRM, or even related to CMR functionality in any way. We sell to a very different customer base. Our customers are on Force.com only because they purchase this app from us. Even if they are using Salesforce for CRM, our app sits in an org that is dedicated to our app only. Because sandboxes add significantly to cost, our customers don't typically purchase them.
When the app goes into our customers' orgs, the Salesforce object names in our app get prefixed with our assigned namespace prefix. The fields in our custom objects get the prefix as well. Not sure why you don't see that with Service Max,but that is how it is supposed to be per Salesforce (can't post a URL since I have less than 10 posts, but a Google search for "salesforce namespace prefix" will bring it up - second search result for me). As a result, all the queries in our TalenD jobs have extensive namespace prefix usage.
Some of our customers have asked us to deliver custom reports that require complex table joins of the kind that the Salesforce reporting functionality doesn't support. These are not joins on standard objects, but rather custom objects that are packaged in our app. We want to use TalenD to generate these reports. We would write jobs to pull the appropriate tables, join them, and then write to CSV or MS Excel.
While we are developing the TalenD jobs, we prefer to not work with customers' production orgs. It is safer for our developers to work with Salesforce Developer Edition orgs, load our software and objects into them, and develop the TalenD jobs. Once the jobs are developed and quality checked, then we would release them to our customers. And that brings up the challenge of the namespace prefix - Developer Edition orgs don't have the namespace prefix, but our Managed package deployed in customer orgs do. The question then is of how to add our namespace prefix to the jobs in a scalable way.
Thanks,
Sumit

[willm1]

Sumit - you're right, the field names also have the prefix. I got so used to looking at the fields that I didn't notice them anymore... So yes, both the 'table' object and fields have the prefix.
Hmm... Sounds like if you know that the namespace prefix will always be unique for your product, one easy solution would be to update your development environment to mimic your client's production environments. That way your development will not suffer many changes when being applied.
Another not-so-good option would be to have a set of schemas for Dev & Prod that are different. But again, your jobs would have to be updated prior to deploying to the client's production environment.
Using contexts for table name prefixes are very easy, but what is tricky is the field name prefixes. If you're using the Talend Enterprise or Platform version, you might be able to leverage the Dynamic Schema feature where Talend will bring in all fields for whatever object you're pulling from. What I'm not entirely sure about is whether that feature works for SalesForce API or not. I can try later and let you know.

[willm1]

Sumit - just confirmed that unfortunately, dynamic schema are not supported in Talend Input & Output Salesforce components...