Pelles C forum

Pelles C => Bug reports => Topic started by: borgfan on August 17, 2005, 06:30:54 PM

Title: fatal error: Internal error: put_tokens.
Post by: borgfan on August 17, 2005, 06:30:54 PM
Just started using Pelles C and am frequently using it instead of MSVC - easy and quick to use.

Am a bit stuck compiling a particular file and get a number of errors which I suspect indicate a bug or compiler limitation - the file does perform loads of preprocessor manipulation.  (the source compiles ok with MSVC or Watcom).

E:\aes\aestab.c(204): error #1025: Eval botch (unknown operator).
E:\aes\aestab.c(207): error #1025: Eval botch (unknown operator).
E:\aes\aestab.c(210): error #1025: Eval botch (unknown operator).
E:\aes\aestab.c(213): error #1025: Eval botch (unknown operator).
E:\aes\aestab.c(230): fatal error: Internal error: put_tokens.

Any suggestions?
Title: fatal error: Internal error: put_tokens.
Post by: Pelle on August 17, 2005, 07:01:19 PM
From the error messages, without looking at the actual code, I can only say that there is a problem with a preprocessor expression - it seems to contain an "unknown" operator. AFAIK, all standard C preprocessor operators should be supported...

What exactly *is* on line 204, 207, ...?

Pelle
Title: fatal error: Internal error: put_tokens.
Post by: borgfan on August 17, 2005, 07:23:11 PM
I've attached the relevant source files, hope this helps.  The error lines are #if expressions , but I would suspect that the problem sits elsewhere.  I have compiled the file ok with VC++ in MS Visual Studio .Net 2003

Gerard
Title: fatal error: Internal error: put_tokens.
Post by: Pelle on August 17, 2005, 09:03:04 PM
The lines all looked like this:

#if RC_LENGTH > 10


and RC_LENGTH had this definition:

#if !defined(BLOCK_SIZE)
#define RC_LENGTH   29
#else
#define RC_LENGTH   (5 * BLOCK_SIZE / 4 - (BLOCK_SIZE == 16 ? 10 : 11))
#endif


So with BLOCK_SIZE defined, the conditional operator was used. A bugfix in the preprocessor evaluator, exactly a year ago, was great - except that it "killed" the conditional operator. My regression tests didn't cover this case - and it's obviously an uncommon operator since it survived a whole year...

Attached is a bugfix version of the compiler. If it works for you too, it will be included in the next beta version of 4.00.
Title: fatal error: Internal error: put_tokens.
Post by: borgfan on August 17, 2005, 11:36:29 PM
Thanks for the speedy response - impressive!

I tried a re-compile with the bugfix you supplied but I'm getting other errors which look like they are related to the previous problem:


Building aestab.obj.
E:\aes\filenec\aesopt.h(258): warning #1031: Multibyte character constant undefined.
E:\aes\filenec\aesopt.h(260): warning #1031: Multibyte character constant undefined.
E:\aes\filenec\aestab.c(230): fatal error: Internal error: put_tokens.

where
258   #elif (('1234' >> 24) == '1')
259   #  define PLATFORM_BYTE_ORDER AES_LITTLE_ENDIAN
260   #elif (('4321' >> 24) == '1')
261   #  define PLATFORM_BYTE_ORDER AES_BIG_ENDIAN
262   #endif
Title: fatal error: Internal error: put_tokens.
Post by: Pelle on August 18, 2005, 12:24:36 AM
Right. I extracted code to test the "unknown operator" problem, I never tested with the entire file. You actually have two problems (at least).

The "put-tokens" problem is because a *huge* buffer seems to be needed to generate the preprocessor output in this case. This will take a little longer to fix - I will look at it tomorrow (it's sleepy time now...)

The following "character constant" is non-standard, and I don't support it even in MS mode. You can probably work around this - see comments in aesopt.h around line 220.

'1234'


Pelle
Title: fatal error: Internal error: put_tokens.
Post by: Pelle on August 18, 2005, 06:09:14 PM
Let's try again. The attached version seems to work. I still get errors because I'm missing aes.h, and possibly other #include files, but I think the errors are from missing declarations rather than compiler problems.

Pelle
Title: fatal error: Internal error: put_tokens.
Post by: borgfan on August 18, 2005, 07:11:03 PM
Success!  Thanks very much for this correction - hope it wasn't so obscure that it wasted your time. :D


Gerard
Title: fatal error: Internal error: put_tokens.
Post by: Pelle on August 18, 2005, 07:28:12 PM
Great news! Thanks!

A bit obscure - but if it made the compiler a little bit better, probably a good investment in time... ;-)

Pelle