Assembly programming - WinAsm vs Visual Studio 2017
Asked Answered
M

1

10

I'm here to ask you some stuff about VS2017.
In the past I had used WinAsm for MASM and I never got problems with it.

However, when I'm trying to do something with MASM in VS2017, I always gonna get problems and stuff...
I've checked the whole internet about "how to set up VS for MASM", but nothing has helped me as I'm always getting troubles...

Is there any way to use Visual Studio 2017 for MASM32/64bit without any kind of headache?

Can someone give me the ultimate guide to set up VS2017 for assembly programming?

Thanks you very much and sorry for my weak english.

Metaphysic answered 13/10, 2018 at 19:2 Comment(0)
H
14

How to build a x64/x86-project with a standalone x64/x86 assembly file

1) Start Visual Studio (Community) 2017 and choose FILE - New - Project.

Screenshot

2) In the next window choose Empty Project.

Screenshot

3) Make sure, that the project is highlighted in the Solution Explorer and and choose PROJECT - Build Customizations....

Screenshot

4) In the next window tick masm(.targets,.props) and click on OK.

Screenshot

5) Choose PROJECT - Add New Item from the menu.

Screenshot

6) In the next window choose C++File(.cpp) and - IMPORTANT! - give it a name with an .asm extension. Click on Add.

Screenshot

7) Now you can fill the file with content.

Source.asm:

EXTERN GetStdHandle : PROC
EXTERN WriteFile    : PROC
EXTERN ExitProcess  : PROC

.DATA?
    hFile           QWORD ?
    BytesWritten    DWORD ?

.DATA
    hello   BYTE 'Hello world!', 13, 10

.CODE
main PROC
    ; https://blogs.msdn.microsoft.com/oldnewthing/20160623-00/?p=93735
    sub rsp, 40                 ; Shadow space (4 * 8) & 1 parameter (8 bytes)
    ; https://learn.microsoft.com/en-us/cpp/build/stack-allocation
    and spl, -16                ; Align to 16

    ; https://msdn.microsoft.com/library/windows/desktop/ms683231.aspx
    mov ecx, -11                ; DWORD         nStdHandle = STD_OUTPUT_HANDLE
    call GetStdHandle           ; Call WinApi
    mov hFile, rax              ; Save returned handle

    ; https://msdn.microsoft.com/library/windows/desktop/aa365747.aspx
    mov rcx, hFile              ; HANDLE        hFile (here: Stdout)
    lea rdx, hello              ; LPCVOID       lpBuffer
    lea r9, BytesWritten        ; LPDWORD       lpNumberOfBytesWritten
    mov r8d, LENGTHOF hello     ; DWORD         nNumberOfBytesToWrite
    mov qword ptr [rsp+32], 0   ; LPOVERLAPPED  lpOverlapped = NULL
    call WriteFile              ; Call WinAPI

exit:
    ; https://msdn.microsoft.com/library/windows/desktop/ms682658.aspx
    xor ecx, ecx                ; Set RCX to null for return value
    call ExitProcess            ; Call WinAPI to exit
main ENDP

end

This is a 64-bit Console application that starts at the procedure main.

8) Change the Solution Platforms to x64

Screenshot

9) Choose PROJECT - Properties.

Screenshot

10) In the Properties window you have to complete two linker options:

  • Entry Point: main
  • SubSystem: Console (/SUBSYSTEM:CONSOLE)

Choose at the left side Configuration Properties - Linker - All Options , change both options at once and click OK.

Screenshot

Screenshot

11) Build and run the .exe with CTRL-F5. The application will be opened in a new window.


Now overwrite Source.asm with a 32-bit Console application:

.MODEL flat, stdcall

; https://learn.microsoft.com/en-us/cpp/assembler/masm/proto
GetStdHandle PROTO STDCALL,     ; https://learn.microsoft.com/en-us/windows/console/getstdhandle
    nStdHandle: SDWORD
WriteFile PROTO STDCALL,        ; https://learn.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-writefile
    hFile: DWORD,                       ; output handle
    lpBuffer: PTR BYTE,                 ; pointer to buffer
    nNumberOfBytesToWrite: DWORD,       ; size of buffer
    lpNumberOfBytesWritten: PTR DWORD,  ; num bytes written
    lpOverlapped: PTR DWORD             ; ptr to asynchronous info
ExitProcess PROTO STDCALL,      ; https://learn.microsoft.com/en-us/windows/desktop/api/processthreadsapi/nf-processthreadsapi-exitprocess
    dwExitCode: DWORD                   ; return code

.DATA                   ; https://learn.microsoft.com/en-us/cpp/assembler/masm/dot-data
    Hallo db "Hello world!",13,10

.DATA?                  ; https://learn.microsoft.com/en-us/cpp/assembler/masm/dot-data-q
    lpNrOfChars dd ?

.CODE                   ; https://learn.microsoft.com/en-us/cpp/assembler/masm/dot-code
main PROC               ; learn.microsoft.com/en-us/cpp/assembler/masm/proc
    invoke GetStdHandle, -11            ; -> StdOut-Handle into EAX
    invoke WriteFile, eax, OFFSET Hallo, LENGTHOF Hallo, OFFSET lpNrOfChars, 0
    invoke ExitProcess, 0
main ENDP

END main                ; https://learn.microsoft.com/en-us/cpp/assembler/masm/end-masm

Change the Solution Platforms to x86 (No. 8 above) and complete the project properties with SubSystem: Console (/SUBSYSTEM:CONSOLE) (No. 10 above). You must not set the Entry point, because ml32 expects the entry point after the END directive (END main). Build and run the .exe with CTRL-F5.

Hopi answered 14/10, 2018 at 14:8 Comment(10)
Related: external assembly file in visual studio for adding an asm file to a C++ projectTipton
@Hopi When I build and run the .exe with CTRL+F5 in x64, I'm getting a linker error: error LNK2001: unresolved external symbol mainCRTStartup.Aitch
@Belloc: Strange. I just tested it with VS Community 2017 Version 15.9.16. It works. Did you forget to specify the Entry Point (number 10 of my instruction)?Hopi
Mine is VS2017 15.9.17. I recheck step 10 and it's OK.And the message remains. See the printscreen in my OneDrive.Aitch
VS2017 says it cannot open file *.asm (A1000 error). On VS2015 it works fine (both x86 and x64). However, you have to delete all comments, otherwise there is a compilation error (at least on my machine). And it doesn't recognize .data? directive, so, put all the related variables under .data section.Lareine
A long time ago, in a galaxy far, far away, there were famous tutorials composed by Iczelion. As I remember, in order to make linker recognize WinAPI functions, we have to manually include kernel.dll and some other files into additional dependencies section. However, here it works pretty well without any additional effort. Does VS automatically include all necessary libraries? What is the magic behind?Lareine
@tenghiz: Does VS automatically include all necessary libraries? Yes. Take a look to the properties of the project, especially the Linker section and there Input-Additional Dependiencies and Command Line. For your other concerns, I'm gonna need a little more time - I cannot reproduce the errors yet.Hopi
As for x86 code, why do you use WriteFile instead of WriteConsole function? Any advantage? Or it is a common practice?Lareine
@tenghiz: Primarily because you can redirect it. I use a program that catches the console output - there it is quite practical. It's similar to Linux' SYS_WRITE (int80h/eax=4 resp. syscall/rax=1). Even DOS has a similar function: INT 21h/AH=40h.Hopi
I have no problem compiling x64. On x86 I get a LNK1104: Impossible to open the file source\repos\Project1\Debug\Project1.exeTephra

© 2022 - 2024 — McMap. All rights reserved.