Extending homework testing platform to include code analysis (C/C++)

Question

I'm maintaining/developing a platform for homework testing. It's mostly automatic. What I need to add now is code analysis. I need to check the code for specific constructs.

For example:

Does the file main.cpp contain a class named user with a const method get_name()?

Is there some tool out there that would allow me to do such stuff (ideal would be something that can be scripted). Linux only.

Deville

Answer 1

I have discovered dehydra tool from Mozilla. It seems to be mostly written for internal purposes, but it may just be exactly what I was looking for.

https://developer.mozilla.org/En/Dehydra/Using_Dehydra

Edit: Dehydra is great. It is missing some minor features like determining const methods, but otherwise great.

Deville

Answer 2

One possibility might be to run the code through GCC and use the GCC-XML extension to produce XML descriptions of the program's internal structure. You could then use your favourite XML library to parse the document, or apply an XSLT to it if all you need to do is display it as HTML or whatever.

Germinative

Answer 3

You could probably hack together something that used the GCC-XML framework without too much difficulty.

Sherrillsherrington

Answer 4

How does this apply to C? :)

Does the file main.cpp contain a class named user with a const method get_name()?

Create another file (test.cpp) with

void test(void) {
  const user x;
  x.get_name();
}

compile test.cpp and main.cpp together. If there's an error (exit code != 0) then NO!, the file main.cpp does not define a (public) class named user with the specific method.

Caveat: I know no C++, so excuse any major (or minor) errors in the above.

---

Edit script added

#! /bin/sh

cat main.c test.c > 3710532.c
if gcc 3710532.c > /dev/null 2>&1
then echo OK
else echo BZZZT
fi
rm 3710532.c

I don't have a ready-to-use C++ compiler installed on this machine I'm on, so my test was with a C compiler and C files. My compiler didn't "work with" gcc -combine main.c test.c so I tweaked that part.

Running this script with a working combination of main.c and test.c outputs 'OK', otherwise it outputs 'BZZZT'.

---

For the tests I used this main.c

typedef int user;

int get_name(void) {
  return 0;
}

int main(void) {
  return 0;
}

and this test.c

void test(void) {
  const user x;
  get_name();
}

Sample run

$ ./3710532.sh
OK

Bambi

Hmmm ... if there is no error, then YES!: main.cpp defines the class and method; otherwise the error could be anything – Bambi

This would compile just fine with a non-const method get_name . You need to make x a const user notwithstanding pmg's point regarding other possible errors. – Rennold

Thanks Mark. Code changed: x is now const. I didn't even register the const for the method. Just making x const guarantees the method is const too? – Bambi

yes, if x is const the compiler will not allow you to call a non-const method on it. This is actually an intriguing way of solving the asked question, but you would probably need to use the same compiler and options to prevent mangling from changing the (underlying) function name. – Rennold

Yes this is exactly what we do now. The problem is that it is extremely limited. How do you test if certain function is using/not using certain other function, how do you test if a class is derived from other class, how do you test if there is a method (doesn't matter if it is static). Sure it can be done, but it just requires way to much work (plus we get a lot of false-positives). – Vagrancy

Answer 5

If you want to do arbitrary code analysis, you need arbitrary parsing/matching/etc. GCC-XML will give you declaration information, but not the content of the methods.

Our DMS Software Reengineering Toolkit will provide the same abstract information as GCC-XML, but additionally include complete detail for the content of definitions (e.g., method body information), supported by its C++ Front End. THis will let you access delcarations and content to check your student programs.

DMS provides general purpose parsing to ASTs, symbol tables and source-pattern matching. The C++ front end provides full C++ parsing, building C++ ASTs and corresponding symbol information. What you do after that for recognition is up to you, but your example seems to be about looking for a specific pattern.

Half of your example would be handled by few DMS source patterns for C++:

pattern is_correct_student_class(m:members):class =
  " class user { \m } ".

pattern is_correct_student_method_present(p:parameters,s:statements):method =
  " const  get_name(\p) { \s } "

(forgive my C++ syntax, I don't write a lot of it) which will match any AST, respectively, corresponding the named user class and the desired const method. The quotes are meta-quotes, with the stuff inside being C++ syntax with escapes \p, \m and \s representing the metavariables p, m, and s, which must syntactically be a parameter list, a method list and statement lists respectively in order to match the pattern. The definitions of the parameter list, etc. are automatically derived from the C++ grammar portion of the C++ Front End.

The other half is implemented by a bit of DMS PARLANSE code executed after invoking the C++ parser and name/type resolver:

   (define has_student_code (lamdba (function boolean [tree AST]))
      (AST:IsInTree tree
          (lambda (function boolean [tree1 AST]
              (&& (Registry:MatchPattern tree1 "is_correct_student_class")
                  (AST:IsInList (AST:GetNthGrammarChild tree1 4) ; the member list
                       (lambda (function boolean [tree2 AST]) 
                           (Registry:MatchPattern tree2 ; a member
                                 "is_correct_student_method_present")
                       )lambda
          )lambda
      )
   )define

with some liberties taken to simplify the presentation.

This is a pretty simple check; you can access the symbol table from the PARLANSE code to do more sophisticated checking if that makes sense.

While DMS doesn't run directly under Linux, it does seem to run under Wine.

Nonessential