You can modify the memory allocated to Talend Studio by modifying the relevant Studio .ini configuration file for your system, such as TOS_DI-win-x86_64.ini for 64-bit Windows systems. For Linux/Solaris/Windows systems, the relevant .ini configuration file is located in your Studio installation folder.
For Mac systems, the studio .ini configuration file named TOS_DI-macosx-cocoa.ini is located in the Talend Studio install dir\TOS_DI-macosx-cocoa.app\Contents\MacOS directory.