file 5.36 says it clearly
file 5.36 actually prints it clearly if the executable is PIE or not as shown at: https://unix.stackexchange.com/questions/89211/how-to-test-whether-a-linux-binary-was-compiled-as-position-independent-code/435038#435038
For example, a PIE executable shows as:
main.out: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, not stripped
and a non-PIE one as:
main.out: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped
The feature was introduced in 5.33 but it did just a simple chmod +x check. Before that it just printed shared object for PIE.
In 5.34, it was meant to start checking the more specialized DF_1_PIE ELF metadata, but due to a bug in the implementation at commit 9109a696f3289ba00eaa222fd432755ec4287e28 it actually broke things and showed GCC PIE executables as shared objects.
The bug was fixed in 5.36 at commit 03084b161cf888b5286dbbcd964c31ccad4f64d9.
The bug is present in particular in Ubuntu 18.10 which has file 5.34.
It does not manifest itself when linking assembly code with ld -pie because of a coincidence.
Source code breakdown is shown in the "file 5.36 source code analysis" section of this answer.
The Linux kernel 5.0 determines if ASLR can be used based on ET_DYN
The root cause of the file "confusion", is that both PIE executables and shared libraries are position independent and can be placed in randomized memory locations.
At fs/binfmt_elf.c the kernel only accepts those two types of ELF files:
/* First of all, some simple consistency checks */
if (interp_elf_ex->e_type != ET_EXEC &&
interp_elf_ex->e_type != ET_DYN)
goto out;
Then, only for ET_DYN does it set the load_bias to something that is not zero. The load_bias is then what determines the ELF offset: How is the address of the text section of a PIE executable determined in Linux?
/*
* If we are loading ET_EXEC or we have already performed
* the ET_DYN load_addr calculations, proceed normally.
*/
if (loc->elf_ex.e_type == ET_EXEC || load_addr_set) {
elf_flags |= elf_fixed;
} else if (loc->elf_ex.e_type == ET_DYN) {
/*
* This logic is run once for the first LOAD Program
* Header for ET_DYN binaries to calculate the
* randomization (load_bias) for all the LOAD
* Program Headers, and to calculate the entire
* size of the ELF mapping (total_size). (Note that
* load_addr_set is set to true later once the
* initial mapping is performed.)
*
* There are effectively two types of ET_DYN
* binaries: programs (i.e. PIE: ET_DYN with INTERP)
* and loaders (ET_DYN without INTERP, since they
* _are_ the ELF interpreter). The loaders must
* be loaded away from programs since the program
* may otherwise collide with the loader (especially
* for ET_EXEC which does not have a randomized
* position). For example to handle invocations of
* "./ld.so someprog" to test out a new version of
* the loader, the subsequent program that the
* loader loads must avoid the loader itself, so
* they cannot share the same load range. Sufficient
* room for the brk must be allocated with the
* loader as well, since brk must be available with
* the loader.
*
* Therefore, programs are loaded offset from
* ELF_ET_DYN_BASE and loaders are loaded into the
* independently randomized mmap region (0 load_bias
* without MAP_FIXED).
*/
if (elf_interpreter) {
load_bias = ELF_ET_DYN_BASE;
if (current->flags & PF_RANDOMIZE)
load_bias += arch_mmap_rnd();
elf_flags |= elf_fixed;
} else
load_bias = 0;
I confirm this experimentally at: What is the -fPIE option for position-independent executables in gcc and ld?
file 5.36 behaviour breakdown
After studying how file works from its source. We will conclude that:
- if
Elf32_Ehdr.e_type == ET_EXEC
- else if
Elf32_Ehdr.e_type == ET_DYN
- if
DT_FLAGS_1 dynamic section entry is present
- if
DF_1_PIE is set in DT_FLAGS_1:
- else
- else
- if file is executable by user, group or others
- else
And here are some experiments that confirm that:
Executable generation ELF type DT_FLAGS_1 DF_1_PIE chdmod +x file 5.36
--------------------------- -------- ---------- -------- -------------- --------------
gcc -fpie -pie ET_DYN y y y pie executable
gcc -fno-pie -no-pie ET_EXEC n n y executable
gcc -shared ET_DYN n n y pie executable
gcc -shared ET_DYN n n n shared object
ld ET_EXEC n n y executable
ld -pie --dynamic-linker ET_DYN y y y pie executable
ld -pie --no-dynamic-linker ET_DYN y y y pie executable
Tested in Ubuntu 18.10, GCC 8.2.0, Binutils 2.31.1.
The full test example for for each type of experiment is described at:
ELF type and DF_1_PIE are determined respectively with:
readelf --file-header main.out | grep Type
readelf --dynamic main.out | grep FLAGS_1
file 5.36 source code analysis
The key file to analyse is magic/Magdir/elf.
This magic format determines file types depending only on the values of bytes at fixed positions.
The format itself is documented at:
man 5 magic
So at this point you will want to have the following documents handy:
Towards the end of the file, we see:
0 string \177ELF ELF
!:strength *2
>4 byte 0 invalid class
>4 byte 1 32-bit
>4 byte 2 64-bit
>5 byte 0 invalid byte order
>5 byte 1 LSB
>>0 use elf-le
>5 byte 2 MSB
>>0 use \^elf-le
\177ELF are the 4 magic bytes at the start of every ELF file. \177 is the octal for 0x7F.
Then by comparing with the Elf32_Ehdr struct from the standard, we see that byte 4 (the 5th byte, the first one after the magic identifier), determines the ELF class:
e_ident[EI_CLASSELFCLASS]
and some of its possible values are:
ELFCLASS32 1
ELFCLASS64 2
In file source then, we have:
1 32-bit
2 64-bit
and 32-bit and 64-bit are the strings that file outputs to stdout!
So now we search for shared object in that file, and we are led to:
0 name elf-le
>16 leshort 0 no file type,
!:mime application/octet-stream
>16 leshort 1 relocatable,
!:mime application/x-object
>16 leshort 2 executable,
!:mime application/x-executable
>16 leshort 3 ${x?pie executable:shared object},
So this elf-le is some kind of identifier that gets included on the previous part of the code.
Byte 16 is exactly the ELF type:
Elf32_Ehdr.e_type
and some of its values are:
ET_EXEC 2
ET_DYN 3
Therefore, ET_EXEC always gets printed as executable.
ET_DYN however has two possibilities depending on ${x:
pie executableshared object
${x asks: are the file executable or not by either user, group or other? If yes, show pie executable, else shared object.
This expansion is done in the varexpand function in src/softmagic.c:
static int
varexpand(struct magic_set *ms, char *buf, size_t len, const char *str)
{
[...]
case 'x':
if (ms->mode & 0111) {
ptr = t;
l = et - t;
} else {
ptr = e;
l = ee - e;
}
break;
There is, however, one more hack! In src/readelf.c function dodynamic, if the DT_FLAGS_1 flags entry of the dynamic section (PT_DYNAMIC) is present, then the permissions in st->mode are overridden by the presence or absence of the DF_1_PIE flag:
case DT_FLAGS_1:
if (xdh_val & DF_1_PIE)
ms->mode |= 0111;
else
ms->mode &= ~0111;
break;
The bug in 5.34 is that the initial code was written as:
if (xdh_val == DF_1_PIE)
which means that if another flag was set, which GCC does by default due to DF_1_NOW, the executable showed as shared object.
The DT_FLAGS_1 flags entry is not described in the ELF standard so it must be a Binutils extension.
That flag has no uses in the Linux kernel 5.0 or glibc 2.27, so I seems to be purely informative to indicate that a file is PIE or not.
test.ccontainingmain? – Redheadedgcc -vto see exactly what commands and options gcc really ends up using. – Cinchonism