function readOnly(count){ }
Starting November 20, the site will be set to read-only. On December 4, 2023,
forum discussions will move to the Trailblazer Community.
+ Start a Discussion
Laxmaya ChnLaxmaya Chn 

Can I send data from salesforce to hadoop(integration)?

In my current project we need to send data from salesforce to hadoop through integration. I have basic knowledge on integration but never done hadoop integration. Anyone have done this before, kindly share your comments and appreciate your help here. Thank you in advance.
nagendra 6989nagendra 6989
HI Laxmaya Chn,

Essentially, getting salesforce data into Hadoop is a bit challenging. It opens a new world of database integration awaiting exploration. It is a relentless opportunity of combining salesforce data and other essentials like log data and domain specific data necessary for ideal business operations.
                                                                  
                                                                    Depending on the salesforce data you are handling, transferring essential information from Salesforce to Hadoop Clusters does not have to be a daunting task.

By using innovative tools such as Salesforce2Hadoop, it becomes easier facilitating the transfers of such entries. This tool normally comes in the form of command line. With the tool, it is possible to carry out a complete import. Alternatively, one can use the tool to increase the importation of data from Salesforce platform to local file systems. What makes the tool an incredible option in transferring data from salesforce to Hadoop is the fact that it supports other common salesforce data types like Opportunity and Account. Additionally, it offers support for custom type data types. This makes it an incredible tool for transferring data from Salesforce to Hadoop Cluster.

Some of the key features of data transferring tools are :

Scalar Programming Language :  It makes the interaction with Hadoop relatively easier. This easily accessible language also makes the user interface friendly and more accessible to average users.

Based on KiteSDK Library : It is a packed library used in the setup of salesforce2hadoop; an incredible data transfer tool. With this advanced knowledge, it becomes relatively easier creating data sets with particular shema.

The Data Transfer Process :
The process of transferring information using the salesforce2hadoop tool is even more challenging but not without its own share of interest. Every single import involves updating of Avro Schema. In the process, the contents of the Enterprise WSDL of your organisation will be duly reflected.
Most notably, the data extraction process uses WSC. This Java library component creates an interaction with Salesforce using SOAP. Notably so, the WSC involved is an advanced level abstraction in addition to the regular SOAP interface.
Laxmaya ChnLaxmaya Chn
Thank you very much Nagendra yesterday to import data from salesforce to hadoop, I suggested to my client sf2hadoop connector tool option and have given all credentials from salesforce. But they are asking about SOAP/REST options to connect with salesforce, can we consider REST/SOAP integration with hadoop as a good option?. Can we send reports directly from salesforce to hadoop via integration?.
nagendra 6989nagendra 6989

Hi Laxmaya Chn,

As per my knowledge whenyou try to integrate with hadoop better go for SOAP  API, the reason is when you are trying to integrate from one technology yo another technology you may need to carry large volumes of data.Where SOAP API comes handy in picture.

In the transfer process from one technology to another technology at any particular given time you need to stop the process or hold it, This time regression testing(Regression testing is a type of software testing that verifies that software previously developed and tested still performs correctly after it was changed or interfaced with other software. Changes may include software enhancements, patches, configuration changes, etc.) comes into picture where it can be done in  better in SOAP API.

I am not pretty much sure of sending reports from salesforce to hadoop, but as per my knowledge it can be done but it has to be converted into neutral language(xml) initially and then to the native code language what ever the platform is using.

One more contraint where SOAP API is handy is when you integrate large volumes of data SOAP API probably would be the best one.

Thanks,

Please mark it as solved if it was useful.


 
Nosika FisherNosika Fisher
The salesforce2hadoop connector chockes up when the Salesforce wsdl schema changes. I can no longer update my Salesforce data to Hadoop once the schema changes.
Any suggestions?