Pelles C forum
C language => Beginner questions => Topic started by: PaulM. on March 28, 2012, 02:20:50 PM
-
Hi,
I have a function called Verlet_integration(), that requires a counter as an input variable, along with some other variables. This counter is an integer called 'count_nodes' and the value of it is 96.
Within that function, I call for five other functions in this order: Positions(), Int_velocities(), Forces(), Accelerations() and Velocities(). All of these functions, except for Forces(), require the counter as an input argument.
However, in between the functions Forces() and Accelerations(), the variable 'count_nodes' for some reason becomes 1178657792 - always the same number.
If I disable Forces() everything works fine, so it has to be somewhere within that function. But the weird thing is if I print the value at the end of Forces(), just before its return, the counter is still 96 - see code below.
I cannot find the mistake I have made. I tried comparing the current version with an older but working version using UltraCompare, but I could not find the mistake and I don't know how to find the problem.
Hopefully someone can give me some directions to track down the problem.
/*******************************************************************
*
* Function : Verlet_integration()
* Purpose : Output node-coördinates to XML-file
* External file : lib_time.c
*
*******************************************************************/
int Verlet_integration (
Node* Nodes ,
Link* Links ,
Head* Paul ,
int count_nodes ,
int count_links )
{
float iv_x[count_nodes];
float iv_y[count_nodes];
// Step 1 : Calculate new positions
Positions( Nodes, Paul, count_nodes );
// Step 2 : Intermediate velocities
Int_velocities( Nodes, Paul, count_nodes, iv_x, iv_y );
// Step 3 : Forces (pressure, springs, drag)
if ( Forces( Nodes, Links, Paul, count_nodes, count_links ) != 0 )
{
return 1;
}
printf("%i\n", count_nodes); // Returns 1178657792
// Step 4 : Accelerations
Accelerations( Nodes, Paul, count_nodes );
// Step 5 : Update velocities
Velocities( Nodes, Paul, count_nodes, iv_x, iv_y );
return 0;
}
/*******************************************************************
*
* Function : Forces
* Purpose : Forces on nodes
*
*******************************************************************/
int Forces (
Node* Nodes ,
Link* Links ,
Head* Paul ,
int count_nodes ,
int count_links )
{
float rho = RHO;
float vsound = V_SOUND;
int node1;
int node2;
float k; // Stiffness
float perm; // Permeability
float length; // Length of link
float strain; // Strain
float rx; // Length-vector
float ry;
float tx; // Tangential vector
float ty;
float nx; // Normal vector
float ny;
float fp_x; // Force-vector due to pressure
float fp_y;
float fs_x; // Springforce vector
float fs_y;
float avx; // Average velocity of link
float avy;
float fd_x; // Drag force due to pressure drag
float fd_y;
float sum_x1; // Sumvector for Node 1
float sum_y1;
float sum_x2; // Sumvector for Node 2
float sum_y2;
Force Fc[count_nodes+1]; // Forces due to impact
int j;
Fc[count_nodes+1].x = 12345; // Set check-number
// Turn this off for testing
CheckImpact(Nodes, Paul, Fc, count_nodes);
// Safety check
if ( Fc[count_nodes+1].x != 12345 )
{
printf("Bounds exceeded for Fc.\n");
return 1;
}
for ( j = 0; j < count_links; j++ )
{
node1 = Links[j].node1;
node2 = Links[j].node2;
k = Links[j].k;
perm = Links[j].perm;
rx = Nodes[node2].r_x - Nodes[node1].r_x; // Vector in link-direction
ry = Nodes[node2].r_y - Nodes[node1].r_y;
length = sqrt( pow(rx,2) + pow(ry,2) ); // Length of link
tx = rx / length; // Tangential vector
ty = ry / length;
nx = ty; // Normal vector n = R * t with R = [0 1; -1 0]
ny = -tx;
// Return 1 if vectors > 1 (impossible)
if ( sqrt( pow(tx,2) + pow(ty, 2) ) > 1.001 || sqrt( pow(nx,2) + pow(ny, 2) ) > 1.001)
{
printf("[%f %f] [%f %f]\n", tx, ty, nx, ny);
printf("%f %f\n", sqrt( pow(tx,2) + pow(ty, 2) ), sqrt( pow(nx,2) + pow(ny, 2) ) );
return 1;
}
// Pressure
fp_x = 0.5 * 4000 * length * nx;
fp_y = 0.5 * 4000 * length * ny;
//fp_x = 0;
//fp_y = 0;
// Springs
// F = k * strain * t
strain = ( length / Links[j].l0 ) - 1;
fs_x = k * strain * tx;
fs_y = k * strain * ty;
//fs_x = 0;
//fs_y = 0;
// Drag forces
// Fd = rho * vsound * v * L * n
// Turn off for testing
avx = 0.5 * (Nodes[node1].v_x + Nodes[node2].v_x) * nx;
avy = 0.5 * (Nodes[node1].v_y + Nodes[node2].v_y) * ny;
fd_x = - 0.5 * rho * vsound * length * avx * nx;
fd_y = - 0.5 * rho * vsound * length * avy * ny;
//fd_x = 0;
//fd_y = 0;
// Sum of forces
sum_x1 = (1 - perm)*(fp_x + fd_x) + fs_x + Fc[node1].x;
sum_y1 = (1 - perm)*(fp_y + fd_y) + fs_y + Fc[node1].y;
sum_x2 = (1 - perm)*(fp_x + fd_x) - fs_x + Fc[node2].x;
sum_y2 = (1 - perm)*(fp_y + fd_y) - fs_y + Fc[node2].y;
// Store sum of forces to linked nodes
Nodes[node1].f_x = Nodes[node1].f_x + sum_x1;
Nodes[node1].f_y = Nodes[node1].f_y + sum_y1;
Nodes[node2].f_x = Nodes[node2].f_x + sum_x2;
Nodes[node2].f_y = Nodes[node2].f_y + sum_y2;
}
printf("%i\n", count_nodes); // Returns 96, as it is supposed to
return 0;
}
-
Try a little experiment... Go into your project options and try recompiling with Optimizations set to "None".
Also, set your Warning level to 2 and be sure to clear out any non-referenced variables you may have... the compiler's report at the bottom should warn you about it.
This may or may not fix the problem, but there is a known bug in the new register-allocator (added at ver 6.0) that does cause problems similar to this. (Look in the bug reports forum.)
-
Also for an other little experiment, change the variable name for the argument from count_nodes to c_n for example...
Ralf
-
Paul make a check on compilation switches. The only possible cause that I can think of, looking at the code, is that Forces is compiloed as __stdcall and clean the calling stack from himself, but it is called as __cdecl implying another stack cleaning. In this way you remove your original arguments and stack relative addressing reads somewhere else in memory. Anyway if this is the issue also the other parameters should be wrong unless the previous calling function have the same stack parametres again.
-
Try a little experiment... Go into your project options and try recompiling with Optimizations set to "None".
Also, set your Warning level to 2 and be sure to clear out any non-referenced variables you may have... the compiler's report at the bottom should warn you about it.
This may or may not fix the problem, but there is a known bug in the new register-allocator (added at ver 6.0) that does cause problems similar to this. (Look in the bug reports forum.)
I did this and it worked again! To check, I re-set the options to default and the program crashed again. Could this be the only cause of this? If it is I can tell my tutors that the bug is causing this, otherwise I will have to find the real cause. (I use version 6.50.8 by the way).
Also for an other little experiment, change the variable name for the argument from count_nodes to c_n for example...
I changed the input argument to "c_n" for all the functions within the Verlet_integration(), and in Verlet_integration() itself, but no succes.
Paul make a check on compilation switches. The only possible cause that I can think of, looking at the code, is that Forces is compiloed as __stdcall and clean the calling stack from himself, but it is called as __cdecl implying another stack cleaning. In this way you remove your original arguments and stack relative addressing reads somewhere else in memory. Anyway if this is the issue also the other parameters should be wrong unless the previous calling function have the same stack parametres again.
If I get it right, I have to check how my functions are called because Forces() might be called for wrongly? How do I check this?
Thanks for the help so far!
-
I did this and it worked again! To check, I re-set the options to default and the program crashed again. Could this be the only cause of this? If it is I can tell my tutors that the bug is causing this, otherwise I will have to find the real cause. (I use version 6.50.8 by the way).
I suspected as much... What Pelle is doing (as far as I understand it) is using extra CPU registers to hold some variables but in some cases (certainly not all) the registers are being corrupted and we end up with "magic numbers" because of it. Setting optimizations to "none" disables the register allocator and stops the problem. There is a slight performance hit... but when the alternative is that it doesn't work at all...
(I'm hoping Pelle will have it fixed in the next version...)
If I get it right, I have to check how my functions are called because Forces() might be called for wrongly? How do I check this?
It's on the project settings -> compiler page, along with the optimizations setting. The thing is to be consistent ... You can read up on the various calling conventions in the apendix to the help file. (Just push F1, inside the IDE...) Generally... if you are working console mode with no Windows libraries _cdecl makes the most sense. If you are using Windows API calls or external libraries, _stdcall will probably make more sense. If you are working 64 bit code, the only available convention is _fastcall.
Main always has to be _cdecl ... so you can write this:
int _cdecl main (void)
{
// stuff
return 0; }
... if you are setting your calls to _stdcall
Frankie is much more knowledgeable about calling conventions than I... so I'll defer to him for the finer points.
-
The default calling convention for a project is set in project->project_options than compiler tab. It applies to all functions that do not have an explicit calling convention declaration. If your modules (.c files) are compiled together using an IDE project it is not probable that they are compiled with different conventions, but not impossible (wrong declaration in header files that are not included in the respective source files so the compiler cannot make any compliance check). More easy to happen if you compiled each module from command line with different switches.
What I suggest to restrict the problem investigation is:
Try to remove (convert to comment) as much as possible of the problematic function so to live only the minimum code that trigger the problem and post a simplified project that permit us to reproduce it. Unless you are able to check the generated assembler by yourself and find the wrong.
About the calling conventions the standard __cdecl push all function parameters on the stack, the caller function have to clean the stack after the called function returned. This mechanism is required for variable parameters functions available with C language (the called function don't know at the compile time how many parameters has been passed while the caller knows). The disadvantage is that the compiler cannot use more efficient machine code instruction (like retn where you can specify how much stack remove on return) that clean stack faster. M$ invented than the __stdcall specifically for Win programs, using it the first two 32 bits arguments (integers, pointer, etc) are passed via registers (eax and edx) the others by stack, on return the stack is cleaned by retn instruction.
__stdcall convention doesn't allow variable number of aguments (if you try the compiler will give you a warning and automatically convert it to __cdecl).
-
Thanks for the explanation, Frankie. The calling convention in the project is by default set to "__cdecl". However going into the aspect of the calling conventions goes way beyond the scope of the assignment.
Setting the Optimizations to "None" has solved the problem, so we're going to stick with that.
Thanks for the help!
-
I'm happy to know that this solved your assignement, but doesn't helped us to find and correct compiler bug.
We will be gratefuls if you'll contribute with what asked for the sake of compiler improvement.
-
Thanks for the explanation, Frankie. The calling convention in the project is by default set to "__cdecl". However going into the aspect of the calling conventions goes way beyond the scope of the assignment.
Setting the Optimizations to "None" has solved the problem, so we're going to stick with that.
Like Friankie, I'm very happy to see your problem is solved. As you mentioned earlier you should advise your tutors this is a "known issue" with the POCC compiler and that setting optimizations to "None" will mask the problem. You can refer them to this thread and the bug reports section of this forum... a fair bit is known about it.
While this is a known problem with the compiler it is not certain that all aspects of it are known... For example: Is it CPU specific? I use an AMD64 x2 processor... does it also affect Intel?
I would encourage you to continue on (on your own, if need be) and concoct the smallest bit from your code that demonstrates the problem and post it as a bug report so that more is known about it. An interesting oddity in our world is that most program bugs are found by inexperienced operators... who tend to do stuff we'd never consider ???
For now, continue to compile your projects with Optimizations off and you should do fine.
Good luck on your course.... The world needs more people who C.
-
I will post the project later tonight.
I would encourage you to continue on (on your own, if need be) and concoct the smallest bit from your code that demonstrates the problem and post it as a bug report so that more is known about it. An interesting oddity in our world is that most program bugs are found by inexperienced operators... who tend to do stuff we'd never consider ???
Isn't that Murphy's Law? ;)
-
Isn't that Murphy's Law? ;)
Yep, sure is :D .... "Anything that can go wrong, will go wrong."
And don't forget Erzats law: "When given the opportunity to do something stupid, most people will."
Ours is probably the only profession that counts on this kind of stuff to produce a better product.
-
To write foolproof code we need some to to certify the product....... ;)
-
:D Good one Frankie! :D
-
And I can add:
more seriously they take their job ..... most apt they are at it ;)
-
I tried to strip down the function Forces(), and I noticed some weird things.
- If I turn off the first part of the function, the check-up part before the loop, the problem moves to the next variable that is within the for-loop (rx).
- If I turn off the check-up AND I turn off the last 4 lines or the 4 lines before those, everything works well.
- If only the check-up or the lines at the end of the loop are turned off, but the other block isn't, it does not work.
I hope you understand what I mean. I've added some comments with the linenumbers in the source. The stripped down version is attached.
-
Maybe it is not a good new for you, but maybe your assignement is not fully complete...
In function Forces if I change the line
Force Fc[c_n+1]; // Forces due to impact
to
Force Fc[c_n+2]; // Forces due to impact
It doesn't crash.
Seems that the problem is simply a wrong array access, you are writing beyond the bounds, overwriting stack variables.
Please check and report so we close the issue. 8)
-
That indeed solved the problem. An honest but stupid mistake ;D
-
I hope that you are studying physics and not computer science...... :D
It's a joke
Anyway is always a bad idea to create large local variables (because they eat processor stack that generally is a limited resource), creating arrays is even worst because indexing problems can create what happened.
-
So... forgive my confusion... but how does setting optimizations to none fix an array bounds error?
In all respect to my friends here, I'm not entirely sure the solution is as simple as making a bigger array. If it was merely an array bounds overrun, changing the compiler settings should not have had any effect and I should have been wrong in my first suggestion.
I'm guessing the +2 suggested simply pushed the next variable back out of harms way, and the overwritten memory location remained unused by the OP's code, hiding the problem rather than fixing it.
I do agree that he probably should have used malloc to create the arrays...
-
I didn't said that making the array bigger solved the problem, but that the test I made, enlarging the array, demonstrates that there was an array bounds problem, and asked him to check the program correctness (I cannot waste time studying his whole program). But from the answer seems that he really made an error on array dimensioning.
With optimization none all variables are layout in memory while with optimizations on many are allocated in registers, memory layout could be different more dead space can exist in data structures.
In the first case maybe the overwriting involved different variables that weren't used after, while with optimization the compiler kept almost only the big arrays on the stack so they were overwrited (the bloated counter is on the calling function stack, the local value seems correct). The casualties are so frequent that some programs can work for years and the bug remains hidden (there are many cases with M$).
To exactly understand what happened you should trace the assembler and check memory writes, unfortunately with large arrays and so many data manipulation to do it would really be a nightmare :-\
-
Hi Frankie
I agree there may be to concomittant issues here... but all that means is there's more than one problem.
-
It was an array issue. Forces had c_n+1 elements, which is 96+1 = 97 in this case. That means indexes 0-96 can be accessed. I tried to acces index c_n+1 = 97, and that index is out of bound.
I hope that you are studying physics and not computer science...... :D
It's a joke
Anyway is always a bad idea to create large local variables (because they eat processor stack that generally is a limited resource), creating arrays is even worst because indexing problems can create what happened.
I study Mechanical Engineering. Normally we do numerical modelling with Matlab, but for this project we had to learn C to understand programming a little bit. We've ony been doing this for 6 weeks (that's why I posted in the Beginners section).
-
We've ony been doing this for 6 weeks (that's why I posted in the Beginners section).
My compliments! For someone a very few weeks into this, you've done really well. From your source code I would have pegged you at the 1 year level, or beyond...
Even though it was an array access issue that still doesn't explain how turning off the optimizer appeared to solve the problem... Oh well, we may never have the answer to that one...
-
Yes Paul good job!
Matlab models are in C and the free compiler that come with it is LCC-Win32, so this work on Verlet integration will be very usefull to bet understand modeling in Matlab.
Tater the issue with the nooptimization is clear. Of course if you would like to understand the details you have to follow in assembler the whole process to see where the code overwrites the variables, how overwrite them and which of them are overwrited.
-
Frankie...
As you know, my knowledge of ASM is just a tad lacking. I know how to do a few little things; like the garbage collector experiment in user contributions, but to sit down and analyse ASM listings I've not written is a bit beyond my skillset.
Sooooo... I'll trust you on this one :D
Still... until that bug is fixed, I think I'll continue building my projects with optimizations off ...
-
Paul...
Taking a last look at your code...
There is one little change I'd make ... in read_data() ... if the data file doesn't open you need to exit the program completely since that is a fatal error for the rest of the code. There are two ways to engineer that... either change the return 0 in the "did it open" test to exit(GetLastError()) -- or -- change the final return 0 in the function to a return 1 and test the returned value in main.c, exiting from there if you get 0 back. What you can't do is let your program go crashing along if that file doesn't open.
This might not seem like a big deal in what amounts to a class assignment, but it's never too early to form good habits regarding safe code. I would suggest that when you write anything that relies upon external forces --data files, dlls, user input, ports, etc.-- you should always spend a couple of minutes asking yourself "What happens if the file isn't there?", "...the port isn't connected?", etc. And try to deal with as much of it as you can before causing an actual disaster. :D
You will see that I have a thread running in the Expert forum where I'm seeking advice on the best way to deal with unknown values from GetLastError() ... So you should expect an ongoing struggle.
-
Thanks for the compliments. I hope the tutors will agree with you; they also judge heavily on the organisation and readibility of the code (because that is the main learning objective of this course). I have to admit that I've been working with Matlab for three years now, and before I went to study at the university I had scripted some PHP in my sparetime for one and a half years; that gave me a nice headstart on my fellow students.
CommonTater, you're absolutely right about the safety check. When the code stopped working I gradually started adding extra failure notifications - some of the are already coded in the project I uploaded earlier. But the external datafile is indeed crucial for the code to work. I've changed the code so the program stops immediately if there's an error in reading that file.
Again, thanks for all the help! I appreciate it.
-
Hey ... no worries!
Good luck on the course!
-
Tater I meant that it's a huge job to trace puntually each memory read and write. And it's not worth of because it is well known, it relates also to buffer overflow techniques used by virus. If you get a local variable address (tipically a char buffer) and start to fill stack with the address of the virus code there is avery high probability that you will overwrite return address (that lays on the same stack) with the maliciust code address. The result is that when your function finish will pick-up from the stack the virus address instead of the return address. What happen after is sadly famous :'(
In my years of system code programming, more than 20, this is engraved in my mind.
So at the end there is no bug at all in generated code. Anyway not using optimization with this compiler is like to drive a Ferrari using only the first gear ;)
Paul maybe readability is not the best, I suggest to add more comments, and some constructs shows PHP habits, but in general I confirm my opinion that your code is good enough.
-
Tater I meant that it's a huge job to trace puntually each memory read and write. And it's not worth of because it is well known, it relates also to buffer overflow techniques used by virus.
A huge job that right now I wouldn't know where to begin... I used to do quite a bit of ASM for Z-80 in cashregisters but that was so specialized that almost none of it carries over to general purpose programming and I've let the skill atropy over the years. Thing is the kind of work I do just never seems to take me back to it. I do know some about buffer overflow tricks and such, but probably not as much as I should.
In my years of system code programming, more than 20, this is engraved in my mind.
So at the end there is no bug at all in generated code. Anyway not using optimization with this compiler is like to drive a Ferrari using only the first gear
Now maybe it's the kind of code I write or maybe it's the way I write it... but I've never been able to spot any significant difference between optimizations off or on ... I use mainly Windows API calls, even in console mode, and tend to use pre-written API functions before writing my own, so maybe I've just never gotten into any situation where it would make a noteworthy difference.
The optimizer bugs are very real, I've posted code in the "Bug Reports" section that demonstrates them. This is a big deal to me because what I do very often messes very deeply with people's data and I tend to favour caution over performance.
I suppose I should grab a spare morning and do some testing to see how much difference it actually makes...
Thanks for your input on this, my friend... very interesting stuff. 8)
-
I worked too on the glorious Z80 CPU. Many cash counters used Z80 or 8085. I wrote a multitasking OS on industrial boards that used that uP (taking ideas from DIGITAL RSX executive). At that time that CPU could do many things (minicomputer had a clock at 15-20MHz, today have 2GHz) ::) Then I moved to 68k and x86 :'(
The worst is that as soon as new faster CPU's come out the programmers have already produced bloated softwares that eat all the computational power.... :-\ with unusefull gadgets >:(
You are right that there are many bugs on optimization, but our job is to find and signal them so Pelle can fix the compiler. It's our payment for use of the compiler itself.
Anyway with no optimization you'll spare some problems and with actual CPU clock's it's hard to see any difference. Unless when strong compute power is used like in math routines (matricial calculations, discrete integration, real-time FFT and DCS for video, etc), but also intensive string manipulation, substitution & replacement can show the difference.
-
The worst is that as soon as new faster CPU's come out the programmers have already produced bloated softwares that eat all the computational power.... :-\ with unusefull gadgets >:(
Don't even get me started on this one! :D I've long held that Microsoft would make more money selling a minimalist, rock solid, OS... and nothing but the OS... Add your own options --browser, word pro, etc.-- by download, don't ship it with the OS. But then I see people go all gaga over an onscreen CPU monitor (why?) or a fancy clock that manages to consume 10% CPU... It's really amazing the crap people do to perfectly good computers...
And FWIW... yes, I agree, there is a LOT of really crappy software out there.
You are right that there are many bugs on optimization, but our job is to find and signal them so Pelle can fix the compiler. It's our payment for use of the compiler itself.
Well, that and an annual donation...
Anyway with no optimization you'll spare some problems and with actual CPU clock's it's hard to see any difference. Unless when strong compute power is used like in math routines (matricial calculations, discrete integration, real-time FFT and DCS for video, etc), but also intensive string manipulation, substitution & replacement can show the difference.
Surprisingly, I do very little math in the things I do. Most of it is file access --Inventory records, file copies, etc. The busiest parts of it are generally broken up by the relatively slow disk access. The slowest parts of my applications are the keyboard, most spend 80% of their time waiting for someone to type something. As such it's likely the optimizations don't make much overall difference. On AMD CPUs with "cool and quiet" enabled, it's unusual for them to ever get up to full speed, mostly they just idle along at 1/2 speed. It's not like I'm sending it off to do 6 hour long calculation runs or something...