Code completion doesnt follow #include in struct

[zacaj]

I posted this in the bug tracker, and was told to post here

object.h:

Code Select Expand

typedef struct
{
#include "object_struct.h"
} Object;
object_struct.h:

Code Select Expand

   vec3f pos; ///< The 3D position of the object
   vec3f lpos; ///< The 3D position of the object last update
   float r; ///< The radius of the Object
   uchar type; ///< Used to identify if the Object has been inherited
Quaternion q; ///< The rotation of the Object, as a Quaternion
void *data; ///< Pointer to the data used by this object for rendering

main.c:

Code Select Expand

#include "object.h"

int main()
{
 Object *object=malloc(sizeof(Object));
 object->ty//HERE
}
after typing ty at //HERE type will not be suggested, nor will any other member of Object

This is even more annoying if you have another object 'inherit' Object:
enemy.h:

Code Select Expand

typedef struct
{
 #include "object_struct.h"
 float hp;
} Enemy
main.c:

Code Select Expand

#include "enemy.h"

int main()
{
 Enemy *enemy=malloc(sizeof(Enemy));
 object->//HERE
}
Code completion will automatically complete with hp (I know this can be turned off, but its useful in lots of other cases)

[oBFusCATed]

C::B version, OS version?
If C::B is <=10.05 then try some of the latest nightlies...

[ollydbg]

@zacaj
Ok, I know why cc does not show it's member.
Because Currently, CC's parser dose not do a full preprocessor. I mean, both files were parsed separately, so, the members in "object_struct.h" will be added to "global namespace" instead of the struct. 

It is too hard to implement a full parser. so the bug can't finished soon. The most reasonal way in the feature was: gcc/clang, those two were all compilers, so they do a full parse on your code.

[zacaj]

C::B rev 6992, Windows 7 32 bit

Is the CC parser custom?  A full preprocessor does seem like it would be complex, but it doesnt seem (at least in my mind, I havent looked at the code) like it should be that hard to jump into another file if its included and just keep reading from there

[oBFusCATed]

zacaj: Why don't you use proper inheritance? Yes, C has it, too.

It is something like:

Code Select Expand


typedef struct A
{
     members of A;
};

typedef struct B {
    A base;
    members of B;
};


[zacaj]

Because that looks really weird.  I use it in less used classes, but for something as common as Object, I dont want to have to type an extra ->base-> every time I change the position

[ollydbg]

Because that looks really weird.  I use it in less used classes, but for something as common as Object, I dont want to have to type an extra ->base-> every time I change the position

I think we can not solve your problem unless we use a full preprocessor/parser framework. 

[JGM]

some weeks ago I started working on a simple to use cpp parser mainly a prepreocessor just for fun. I worked on it about 2 days and was fixing some special cases where the parser wouldn't parse some things as it should do, so some code is commented out since I was debugging it, I was developing it on ubuntu and only using c/c++ standard libraries. I was programming it on a way that the preprocessor would return the final code as it should look for normal parsing and each elements on the code tokenized and identified by enums. Also I did some function place holders for expressions parsing and other things left to do. So if someone is interested on checking it out I uploaded the code to mediafire on a zip file:

http://www.mediafire.com/?yqvsstq23jot650

Comments and suggestions are welcome, I stopped after reading about clang with all it's advanced features, but if someone thinks this could be useful a spark of motivation may come to light xD (i gave up but then thought that the code may be of some use, and I think I was implementing it on a way not so hard to maintain for the future, but after all I'm a noob xD)

Edit: whoa I reached post 500 

[ollydbg]

nice work, I am downloading your code and do some checking.

I know clang was a full preprocessor/parser framework. but it was too complex, and it was released under BSD style license. And finally I switch my mind to gcc. I have read the gcc's cpp internal manual (development manual), and found that developing a preprocessor was really complex.

the advantage of a full preprocessor/parser framework is that it has one AST tree for every translation unit, so, it is precise, but slow.
and currently cc's implementation is: parse every file only once, and collect every token to a single tree, just like ctags did, even there are some files missing, cc's parser still do some guess and continue parsing.

for a preprocessor framework, I remembered that One c::b developer  has implemented a preprocessor(but it was hard to read for me)

[ollydbg]

Comments and suggestions are welcome, I stopped after reading about clang with all it's advanced features, but if someone thinks this could be useful a spark of motivation may come to light xD (i gave up but then thought that the code may be of some use, and I think I was implementing it on a way not so hard to maintain for the future, but after all I'm a noob xD)

here are my two point
1, I think using a generated lexer will make things much easier. a generated lexer can handle somethings like: line counting, column counting, and it use a state machine which will catch the "keyword" much faster than "clang or gcc". (both clang and gcc does not distinguish between a keyword or an identifier, they just do a hashtable search when an identifier returned), from this point, I'd suggest my work on Quex based lexer. ( I do benchmarks showing that it was 200% as the speed of flex generated lexer under windows). I put the test code here( also it include a clang test project to test codecompletion feature of clang )
http://code.google.com/p/quexparser/
would you like to have a look?

2, I found you use std::vector in the code, does std::list is much better? when doing a macro replacement, a vector will always re-size itself.

Currently, I feel a little confused about my quexparser, I do not have a clean direction, I found that even doing a macro replacement need many tricky.
you can look at
http://gcc.gnu.org/onlinedocs/cppinternals/

[oBFusCATed]

2, I found you use std::vector in the code, does std::list is much better? when doing a macro replacement, a vector will always re-size itself.

Or probably a std::deque -> http://www.gotw.ca/publications/mill10.htm

[MortenMacFly]

some weeks ago I started working on a simple to use cpp parser mainly a prepreocessor just for fun.

[...]

http://www.mediafire.com/?yqvsstq23jot650

I cannot test it atm, but do you mean "a preprocessor", or "for preprocessors?

My recent idea concerning preprocessor was using tools like:
http://dotat.at/prog/unifdef/
(...and there are other going in the same direction like "sunifdef") to "clean up" source files in a pre-process and parse what's left. If we do this in memory it should also be pretty fast.

[JGM]

...
1, I think using a generated lexer will make things much easier. a generated lexer can handle somethings like: line counting, column counting, and it use a state machine which will catch the "keyword" much faster than "clang or gcc". (both clang and gcc does not distinguish between a keyword or an identifier, they just do a hashtable search when an identifier returned), from this point, I'd suggest my work on Quex based lexer. ( I do benchmarks showing that it was 200% as the speed of flex generated lexer under windows). I put the test code here( also it include a clang test project to test codecompletion feature of clang )
http://code.google.com/p/quexparser/
would you like to have a look?
...

I started writing a custom tokenizer for the preprocessor since I thought the output would be much simple to analyze on the future and also I wanted it to be smart and produce a tree more easy to analyze. Also I wanted to produce the cleaned code after preproccessing with correct column and line numbers for the code parser as optimizable if possible. Still I need to correctly manage multiple line preprocessors (#define blah blah(123) \).

Whoa! that quexparser looks a little kind of complex for my brain to digest I will try to analyze it deeply.

...
2, I found you use std::vector in the code, does std::list is much better? when doing a macro replacement, a vector will always re-size itself.
...

2. I have read several c++ books and read about the performance on available containers as inner structure but I always forget the differences on each of them (lack of practice) but it may be easy to substitute since containers almost always share same interface (I think)

Currently, I feel a little confused about my quexparser, I do not have a clean direction, I found that even doing a macro replacement need many tricky.
you can look at
http://gcc.gnu.org/onlinedocs/cppinternals/

Yep, this whole c++ parsing thing is hard since the language itself has so many features to look up, but it is fun, I just wanted to create a simple to use preprocessor after several months of c++ inactivity on my blood.

[JGM]

some weeks ago I started working on a simple to use cpp parser mainly a prepreocessor just for fun.

[...]

http://www.mediafire.com/?yqvsstq23jot650

I cannot test it atm, but do you mean "a preprocessor", or "for preprocessors?

My recent idea concerning preprocessor was using tools like:
http://dotat.at/prog/unifdef/
(...and there are other going in the same direction like "sunifdef") to "clean up" source files in a pre-process and parse what's left. If we do this in memory it should also be pretty fast.

yep a preprocessor, with the future goal of complete parser. (my english vocabulary and game of words suck  )

Current logic is to identify preprocessor type when tokenizing (function ex: #define test(x) (x*2) or just a declaration ex #define test_delcared) add it to a vector to then make correct replacements on code to parse it correctly. Actually nested preprocessors as I tested worked correctly, I was fixing some issues with multiple line preprocessors (handle incorrectly to produce correct line and column positions) and then write an expression parser to evaluate macro expressions. Also include files are parsed only once as normally, it handles global and local includes and you can indicate to the class the paths to search. Also the use of string class should be replaced by the wstring one, but since I was playing around at first there are things to be improved.

I wanted to have a much complete and documented code before posting it but well, after reading some threads here I decided to let it go as it is and see if it is understandable by other developers, wishing for the best. The main.cpp file should serve as an example of how I intended to make use of it. If people think the code is not that hard to understand then it may merit it's completeness.

[ollydbg]

some weeks ago I started working on a simple to use cpp parser mainly a prepreocessor just for fun.

[...]

http://www.mediafire.com/?yqvsstq23jot650

I cannot test it atm, but do you mean "a preprocessor", or "for preprocessors?

My recent idea concerning preprocessor was using tools like:
http://dotat.at/prog/unifdef/
(...and there are other going in the same direction like "sunifdef") to "clean up" source files in a pre-process and parse what's left. If we do this in memory it should also be pretty fast.

I briefly read the site: unifdef - selectively remove C preprocessor conditionals
it said:

It is useful for avoiding distractions when studying code that uses #ifdef heavily for portability (the original motivation was xterm's pty handling code), or as a lightweight preprocessor to strip out internal routines from a public header (the Linux kernel uses unifdef to strip out #ifdef __KERNEL__ sections from the headers it exports to userland)

Great, I think we need a lightweight preprocessor, as my point of view, gcc's preprocessor code base was too big and too complex.
The main two job is:
1, handle conditional preprocessor directive, like #if  and do a expression evaluation.
2, do macro expansion

In fact this two method was done in the current implementation of cc, but I think they need to be refactored. Morten, can you give a direction?

@JGM
quex's lexer generator is quite easy to lean, and it's grammar is very easy to learn. once you use this, you can give(retern) a token once a time. the token contains several information include at least four field.
1, token id (identifier, keyword, open-bracket......)
2, string value if it is an identifier, otherwise, it is empty
3, column count value
4, line count value

then you don't care about anything else, you just use the token, and do everything you like. So, quex's lexer stands on a low level, and you can implement the high level preprocessor on that.

I have implement a const value expression solver by "shunting yard algorithm" on the code. There is a quite similar one in the CC's source code. We can have further discussion to collaborate.