Skip to main content
Code Review

Questions tagged [floating-point]

Ask Question

For questions with concerns specifically related to the usage of floating point, such as accuracy and precision of calculations, handling of 0, infinity, and over/underflow, input/output, and binary representation. Not for code that casually happens to use floating point.

194 questions
Newest Active Bountied Unanswered
Filter by
Sorted by
Tagged with
6 votes
1 answer
342 views

This is a library I've been writing, for now named ALFPN, for converting between native floats and layouts that are not native - half, various 8-bit formats etc. ...
herhor67's user avatar
  • 81
3 votes
2 answers
149 views

This is a simple function that accepts a null-terminated string that represents a non-negative number of seconds. It can optionally end in the suffix "s" (seconds, default), "m" (...
xiazhu's user avatar
  • 39
6 votes
2 answers
479 views

When we add lots of floating-point values, we can lose precision if we blindly use std::accumulate() or similar, when the running total becomes much larger than the ...
Toby Speight's user avatar
  • 88.4k
5 votes
2 answers
354 views

I am trying to make an is_integer template function to determine a number is an integer or not. The experimental implementation ...
JimmyHu's user avatar
  • 7,575
3 votes
2 answers
128 views

Whatever language you choose, you may encounter a rounding problem. In fact, this is due to the limit of the required number of bits needed to get the right number after a calculation. Simple example: ...
Sébastien Josserand's user avatar
  • 33
9 votes
3 answers
3k views

I implemented the following code for determining if two floating point numbers are equal is_float_equal(...). It handles the tests for my use cases well, but I was ...
dvnlo's user avatar
  • 153
2 votes
1 answer
1k views

I have designed a class to perform float compare. I used Knuth's strong compare to do it. Float compare is really really tricky, I'm sure there's something wrong :) Any comments? ...
greywolf82's user avatar
  • 123
1 vote
1 answer
232 views

This is a follow up to https://codereview.stackexchange.com/questions/272333/conversion-from-long-double-to-string where I do the reverse conversion. This time around it is string to long double. It ...
user253344's user avatar
  • 11
1 vote
1 answer
94 views

I wanted to make an algorithm that is a counter-example of Cantor's diagonalization argument. Given an integer, this Python code will produce a unique rational number. Fed the sequence of positive and ...
Brent's user avatar
  • 461
1 vote
3 answers
147 views

I have a function which takes one argument, a fraction, and returns two numbers - an integer and a power of two. So, when you divide the the integer by 2 raised to power power of two, you get the ...
Krokodil's user avatar
  • 123
3 votes
1 answer
684 views

latest updated version in cross-post from SO I'm testing performance regression of some code I wrote (this is not that code) by timing its execution in Unit tests. I would like to see if execution ...
Greedo's user avatar
  • 2,635
2 votes
1 answer
592 views

The rationale behind this code is to implement a runtime-safe number conversions in situations when precision loss is possible, but is not expected. Example: passing a ...
Dmitry's user avatar
  • 121
4 votes
1 answer
289 views

The objective was to write a locale-independent function that prints floating-point numbers (i.e. one that always uses . as a decimal point regardless of the locale)...
Kirby Kevinson's user avatar
  • 43
5 votes
1 answer
254 views

Faced with converting floating-point values from a sensor obtained to string on an embedded system to transmit over UART, I came up with the following dbl2str() to ...
David C. Rankin's user avatar
  • 927
8 votes
2 answers
505 views

Given a floating-point number, which is representing a dollar value, I'm aiming to return a rounded, human-readable string. Here are the rules for numbers equal to or greater than 1 million, equal to ...
Martin Burch's user avatar
  • 475
3 votes
1 answer
1k views

I have done a clustering algorithm and represented the results in a pie chart as shown below. ...
P H's user avatar
  • 133
2 votes
1 answer
119 views

Relatively new to Python. Have some experience in C++. I have written a small program that reads a CSV file (SAMPLE) and initialises two arrays and few values. ...
Aditya Soni's user avatar
  • 85
25 votes
4 answers
4k views

I have an implementation of the sqrt function that uses a combination of IEEE 754, packed bitfields, and the Newton-Raphson algorithm: ...
S.S. Anne's user avatar
  • 1,785
0 votes
2 answers
231 views

I would like to merge two double arrays and remove from the result array approximately equal items. My two input arrays have ...
Wollmich's user avatar
  • 125
3 votes
1 answer
813 views

Working with currencies I am aware it is not good to work with floating point values due to rounding and maintaining precision. Better is to work with integers. Would the following Currency class make ...
Bruno Vermeulen's user avatar
  • 947
6 votes
3 answers
2k views

I've got a project where I get product data and need to get the prices which come in, in different formats. Some examples would be: US$17, USD17.00, 17,00€, 17€, GBP17, Only 17,-€, 17.000,00€, 17,000....
Chris's user avatar
  • 161
-1 votes
1 answer
1k views

Given input array of UINT8 (unsigned char) with numElements how could one efficiently ...
Royi's user avatar
  • 582
3 votes
2 answers
1k views

Am making a pure .net library with helper functions (GitHub). However I wanted to have a thousand separator for all number types and here is what I am currently doing ...
Billy Watsy's user avatar
  • 33
2 votes
2 answers
549 views

How could I make this code more effective? ...
ken's user avatar
  • 107
3 votes
0 answers
118 views

I am implementing a new compression algorithm for the weights of a neural network for the Leela Chess project. the weights are roughly 100Mb of ...
Oscar Smith's user avatar
  • 3,717

1
2 3 4