topic Re: Floating Point Excel Output automatically rounding but need precision in Talend Studio

Floating Point Excel Output automatically rounding but need precision

mobmsc2 — Sat, 16 Nov 2024 06:59:57 GMT

I have a tMap with runs a calculation and stores it in a float variable before its outputted to excel. My problem is that the output appears as if its rounded but I can't figure out why

My calculation starts by adding 3 integers and stores that result in a variable "IntValue"

Var.IntValue is then divided it by both 11 and 12 and stores those in float variables Float11 & Float12 respectively.

When I output Var.Float11 and Var.Float12 to a generic schema based output table to Columns of type float I get whole numbers (no decimal values) in both excel and csv file.

If I calculate the formula in Excel myself I get more expected values which include decimal places.

Where in Talend am I going wrong that I don't get the same result?

Re: Floating Point Excel Output automatically rounding but need precision

Anonymous — Fri, 21 Dec 2018 08:05:14 GMT

Hello,

Would you mind posting your tMap setting screenshots into forum?

Best regards

Sabrina

Re: Floating Point Excel Output automatically rounding but need precision

mobmsc2 — Fri, 21 Dec 2018 09:35:07 GMT

Hi Sabrina,

Attached is the Variables Table and the output schema settings of the relevant tMap. Is that what you mean?

tMapSettings-ExcelFloating.png

Re: Floating Point Excel Output automatically rounding but need precision

mobmsc2 — Wed, 02 Jan 2019 11:36:19 GMT

Also if add a tRowLog to output the flow the column in question is also a whole number

My solution was to change the tMap Variable Counted from int to float (in the image attached earlier). That means the calculations where I divide by other numbers is based on a float as input instead of an integer value. I assumed only the end result needed to be floating and not the dividend.

Re: Floating Point Excel Output automatically rounding but need precision

akumar2301 — Wed, 02 Jan 2019 12:09:58 GMT

use value like 11.0 instead of 11.

In Java , if you divide Int to Int , it will be rounded before Type Conversion.

check result of below commands

System.out.println(11/10);
System.out.println(11/10.0);