Is the VC++ code DOM accessible from VS addons?

Question

Visual Studio IntelliSense for VC++ includes the "complete" EDG C++ parser (also used by Intel and others). Since the C# Code DOM is accessible to addons (correct me if I'm wrong), is the C++ Code DOM also accessible? Can this be used to analyse an open VC++ project within the VS environment?

Puparium

My bet is it was someone for whom DOM boils down to HTML. It is a good question, I ended up writing my own compiler to do it. – Antelope

Geo - do you know about VisualAssistX for VS? I am pretty sure that's what they are doing, it really makes VS "livable" in a C++ environment, which historically has been quite sparse of intelligent autocomplete for quite a while. I don't know about recent versions, since I've walked away from MS tools altogether and forever. But if VAX can do it, that means it can be done, if that is your question and not "How". On a side note, don't you think it would be easier and faster to work directly with the DOM and use it to generate the actual code instead ;) – Antelope

Just came from your other thread; this is a good question. – Keeshakeeshond

Again, "unclear what you're asking?" - really folks? "I don't know anything about the premise of this question" does not mean "Unclear what you're asking". – Malefic

@ddriver - Consider adding an answer. Too many comments! I use VAX, but I have no idea how they access the Code DOM, but I'm guessing they enhance it too (ie. parse the code manually and add data that wasn't parsed by VS) .. (offtopic) Which environment do you use for C++ and why is it better than VS? – Puparium

@Geotarget - ATM I am using Qt - it comes with Qt Creator, which has decent auto-complete and macro support similar to VAX. The framework itself is not only vast but also portable - the same codebase targets win+mac+linux+android+ios plus a bunch of other, more exotic platforms. The APIs are very intuitive to use. And then... there is QtQuick and declarative, which is a huge productivity boost. Ah yes, and under LGPL you can use it for free in commercial software, as long as you link dynamically. – Antelope

Also - I didn't post an answer because I don't think it is "answer-worthy" - just some on the side info. If it answers your question - all the better. – Antelope

"Since the C# Code DOM is accessible to addons (correct me if I'm wrong)" is not correct, and that's why Microsoft recently open sources Roslyn to fill the gap. – Sheba

@Geotarget no not really - but you havent been able to answer my question. Why should anyone, without insider MS knowledge be able to answer this question? Questions should be based upon real problems that you actually face - what is your actual problem? – Osborn

@Dave - I've added an answer to prove that non MS-employees can solve this issue as well. There are people who've done things close to what I want. – Puparium

@Dave - "Whats my problem?" As I said in my question, I need to analyse C++ code and if I could use the inbuilt VS parser to give me a CodeDOM I could write addons easier. Otherwise I'd have to go build my own parser. Isn't that obvious? – Puparium

@DaveHillier If it's on-topic, it's perfectly welcome here. There is no discussion beyond that. If we start getting into "Can anyone here even answer this?" then we rapidly approach this kind of sticky angst where anything hard is potentially unanswerable. Unless we actually open the question for answers, we frankly have no concrete idea on who could answer it. That's the whole point of asking in the first place. – Malefic

@bmargulies What harm is this question doing by simply existing? It's on-topic, it's interesting, and someone could conceivably answer it. Heck, someone from MS might see this and answer it, and then it's a great addition to the site. Please stop looking for anything potentially negative about a question before attempting to see anything positive. – Malefic

Another unclear question sent me here: #23302352 – Wicks

Is your focus on using MS visual studio machinery only for such analyses, or are you willing to consider alternative engines that can be used to analyze MS VS code? – Yoon

@IraBaxter - Anything will do. Add any suggestions as an answer below, please. Too many comments alreadY! – Puparium

@bmargulies, "only MS employees can answer this." Not sure I understand your point. I'm sure lots of MS employees post here, if at least when they're "off duty" – Hurwitz

Answer 1

The Visual C++ Refactoring extension is able to rename a member project-wide. Its built by MS but obviously they used the internal Code DOM to achieve this. So it is possible, I just don't know how, yet.

The CppLister extension is able to read the intellisense databases created by VS to list the various members within a class.

You can always use the open source Clang C++ parser (actually compiler) and read the AST into a C# Object Model. See CppSharp and ClangSharp for C# bindings to Clang.

Puparium

Answer 2

I'm not sure what the "C++ Code DOM" is, if it even exists. What matters is that MSVS is using the EDG front end to parse and determine meaning of symbols, to support MSVS IDE actions. EDG IIRC builds its own internal data structures representing the program; I have no reason to believe that those data structures are the "C++ Code DOM", or that they are visible to you or you'd be able to find out about them at MSDN.

Your real stated problem is you want to analyze C++ source code. I agree, having the EDG front end information would be a significant aid to do that; you really really don't want to attempt to write your own C++ parser (and you need lots of stuff past parsing, google my essay on "life after parsing").

So you kind of have the following choices:

• Find a door into the EDG machinery in MSVS. Since you haven't had a lot of luck and there appears to be nothing documented from MS saying this is available, you probably won't have a lot of luck this way. If I were in MS's shoes, I wouldn't make it public; it would just be another support headache, and on a piece of software that isn't even theirs.
• Use the commercial EDG front end, directly from EDG. My understanding is they offer individual use licenses at no charge. (My understanding may be wrong). This way you skip any restrictions that MS may have on access... at the price of having to configure the EDG front end yourself. A downside: EDG wants to be the front end of a compiler, not the front end of an analyzer. That distinction may seem subtle but it will probably bite you. For instance, I suspect EDG throws away comments; compiler front ends don't need them. If you want to inspect comments in your analyzer, this might be a real problem.
• Use Clang. This is an open source C++ parser, designed to use for a wide variety of program analysis purposes as well as for front ending a C++ compiler. I have no experience with this, but it seems pretty well thought out, and appears to offer lots of facilities. I don't know if it has specific support for the MS dialect of C++.
• Use another commercial front end, our (DMS) C++ Front End. Being the architect of this, I'm pretty sure it is well thought out (including support for MS Visual C++); there is specific experience with using this to carry out complex C++ analysis and transformation tasks. Unlike EDG, it is designed to support analysis, transformation and generation (e.g., it captures comments and even the radix of literals so they can be regenerated correctly). The foundation, DMS, has lots of machinery built in to support custom analysis: AST and symbol table construction, attribute grammars, data flow frameworks, intraprocedural control and data flow analysis at the AST level, BDD management, source pattern matches, source-to-source transformations. Clang and EDG offers AST and symbol table construction; Clang (but I don't think EDG) has it has flow analysis (at the LLVM level), but not flow analysis at the AST level (AFAIK). Neither Clang nor EDG offer the source pattern/transformation capability, so which is better depends on your long term tasks. Compared to the other options, our C++ front end isn't open source or free; one can get research licenses.

Yoon

"DOM" is HTML-speak for an AST, basically. It does assume there's a canonical syntax, though, whereas C++ compilers usually use slightly different syntaxes. (E.g. to create better error messages.). – Dextrosinistral

DOM in C# means "AST with bad resolution" used for code generation. You can't realistically analyze C++ programs with the kind of resolution the C# dom offers. – Yoon

See inevitablesoftware.com/Products.aspx for what a good C# codedom offers – Puparium

@Geotarget: and what has a C# Dom got to do with the question? – Yoon

@IraBaxter - What I'm looking for when I say "Code DOM" - many people have commented that they don't understand what a Code DOM is and/or they confuse it with the JS HTML DOM. Of course I'm not a C++ expert (as I have stated before) so I don't know what might be needed in a C++ code DOM, although I understand that Inevitablesoftware provides a very simple and easy to use code DOM, and I am on the lookout for a C++ code DOM with a similar API. – Puparium

@Geotarget: "Similar API" means "has a tree data structure, can parse source and produce the tree, provides access/update functions, can prettyprint tree back to source code"? I believe DMS does this spectacularly well (including capturing preprocessor directives), the EDG does this for preprocessed source code (I don't much see the point), and that Clang does this badly (you can't really modify the tree, you have store your changes off to the side and then regenerate with the changes, but this means you don't have a real up to date tree). Are you looking for an alternative option? – Yoon