Race Condition

Mooo Meltdown, DirtyCOW and more! What is a Race Condition?

Race conditions are the result of code that need to happen in a certain order but have unintended consequences when they are misordered.

Exploiting a race condition vulnerability:

- Make instructions execute in a unintended order - Take advantage of undefined behavior History of Race Condition

The term race condition has been used since 1954 in Huffman’s notes on his thesis of sequential switching circuits

Did not describe the vulnerability in software, but was used to describe an event that would occur in hardware

For example, suppose there is an AND gate that takes 2 inputs: A and NOT A If A were to change from false to true, the term race condition was used to describe a glitch in which there was a time lag for NOT A to change to false

For a short moment, both A and NOT A will be true → results in (A AND NOT A) = True Why are Race Conditions prevalent?

● Large code bases ○ “The kernel has around 27.8 million lines of code in its Git repository...”

● Writing good, error free multithreaded code is HARD ● Testing for race conditions is HARD ● New features might have unintended effects Unlimited Starbucks Credits

Vulnerability discovered by Egor Homakov in 2015

When multiple requests are made to transfer points from card A to card B in a short time frame, a race condition occurs

3 non-atomic processes

- check that card A has enough credits - transfer credits to card B - deduct credits from card A Thread Sequence

Let’s say Card A has a balance of $5 and we want to transfer $5 to Card B two times

Correct Sequence:

Thread 1 checks the balance of Card A

Card A = $5 Card B = $0 Thread Sequence

Let’s say Card A has a balance of $5 and we want to transfer $5 to Card B two times

Correct Sequence:

Thread 1 adds $5 to Card B

Card A = $5 Card B = $5 Thread Sequence

Let’s say Card A has a balance of $5 and we want to transfer $5 to Card B two times

Correct Sequence:

Thread 1 deducts $5 from Card A

Card A = $0 Card B = $5 Thread Sequence

Let’s say Card A has a balance of $5 and we want to transfer $5 to Card B two times

Correct Sequence:

Thread 2 checks the balance of Card A

Card A = $0 Card B = $5 Thread Sequence

Let’s say Card A has a balance of $5 and we want to transfer $5 to Card B two times

Race Condition Sequence:

Thread 1 checks the balance of Card A

Card A = $5 Card B = $0 Thread Sequence

Let’s say Card A has a balance of $5 and we want to transfer $5 to Card B two times

Race Condition Sequence:

Thread 2 checks the balance of Card A

Card A = $5 Card B = $0 Thread Sequence

Let’s say Card A has a balance of $5 and we want to transfer $5 to Card B two times

Race Condition Sequence:

Thread 1 & 2 each add $5 to Card B

Card A = $5 Card B = $10 Thread Sequence

Let’s say Card A has a balance of $5 and we want to transfer $5 to Card B two times

Race Condition Sequence:

Thread 1 & 2 each deduct $5 from Card A

Card A = -$5 Card B = $10 Meltdown (CVE-2017-5754)

Attacker takes advantage of the use of parallel threads for caching

2 non-atomic processes:

- moving the data the cache - checking for user authorization to access the data

Attacker sets base_address and tries read(base_address + (read(x)=y)) where ‘y’ is data that is accessible only by root Reads from addresses until they get a fast receive Attacker now knows the value of ‘y’

See: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5754 History of Meltdown (CVE-2017-5754) Meltdown officially discovered and assigned a Common Vulnerabilities and Exposures ID in January 2018

Discovered by Jann Horn (Google Project Zero), Werner Hass & Thomas Prescher (Cyberus Technology) Daniel Gruss et al. (Graz University of Technology)

Initially thought to be a false vulnerability by security analysts because of catastrophic it is

Attempts to patch the vulnerability resulted in reduced performance (around 5-30% performance speed decrease) DirtyCOW (CVE-2016-5195)

Exploits the kernel functions that handle Copy-on-Write (COW) Process requests a copy of some data → Kernel does not create an actual copy until it is written to

2 non-atomic processes:

- locating the physical address to write to - writing to the physical address

In between the 2 processes, the attacker tricks the kernel into writing to the original file instead of the copy

See: https://cve.mitre.org/cgi-bin/cvename.cgi?name=cve-2016-5195 DirtyCOW (cont’d)

Attacker logs in as a normal user and uses mmap to request a private mapping of a root file that they do not have write privileges to Instead of writing to the address returned by mmap, the attacker writes to the file /proc/self/mem which allows process-to-process virtual memory access

Attacker writes to the private mapping → kernel now looks for a physical address to write to Before kernel can write to the address, attacker calls Madvise to tell the kernel to toss the private mapping

Kernel writes to the root file instead History of DirtyCOW (CVE-2016-5195)

First discovered by Phil Oester in September 2007

Reported the exploit when he found that it was used in an attack against one of his servers

DirtyCOW is thought to have been existed even before because Phil discovered it as a victim and not as an attacker

The first patch for it was released 9 years later and it wasn’t until November 2017 that the vulnerability was fully patched ZNUI In 2017, over 1200 Android Apps were found to be infected with ZNUI

Disguises itself as pornography apps and utilizes DirtyCOW to gain root access to the device, allowing the malware to plant a back door - once app launches, malware communicates with its C&C servers to search for updates - executes DirtyCOW → attacker now has access to the user’s information - purpose of ZNUI is to pose as the user’s network carrier to make micro-transactions then delete any logs that may alert the user - luckily, carrier transactions are not available internationality so this money scam only works within China but attacker can still steal information from foreign users Drupalgeddon2

In 2018, DirtyCOW was used together with Drupalgeddon2 to exploit websites running on an older version of Drupal

Attackers perform massive scan for sites running on outdated versions of Drupal

Using Drupalgeddon2, attackers were able to access local configuration files and search for root accounts that they could then to attempt and log into

In the case where they cannot log in, they use DirtyCOW to elevate to root privileges and install SSH daemons Why Exploit it? Exploiting race condition vulnerabilities often lead to gaining root access or executing commands with root privileges

Often used in combination with other exploits and malware

As we’ve seen in the previous examples, attackers often use these exploits to gain profit.

Common Uses: Financial transactions Android Jailbreaking IOS How to Defend Against it: DEVELOPER As a developer: - make sure that code that needs to happen in a certain order happens in that order - mutexes - semaphores - completion flags How to Defend Against it: USER As a user: - be smart about where you download applications - keep software up-to-date with the latest security patches Run the command uname -a 4.8.0-26.28 → Ubuntu 16.10 4.4.0-45.66 → Ubuntu 16.04 LTS 3.13.0-100.147 → Ubuntu 14.04 LTS 3.2.0 -113.155 → Ubuntu 12.04 LTS 3.16.36-1+deb8u2 → Debian 8 3.2.82-1 → Debian 7 4.7.8-1 → Debian unstable Technical Explanation: Dirty Cow

● Linux kernel versions >= 2.6.22 and < {4.8.3, 4.7.9, 4.4.26} ● Any version or distro between September 2017 and November 2017 ○ Ubuntu < 12.04, 14.04, 16.04 ○ Debian < 8

see: https://github.com/dirtycow/dirtycow.github.io/wiki/PoCs Technical Explanation: Dirty Cow

● Targets files that are: ○ read only e.g. configuration files ○ setuid files/binaries e.g. passwd, ping, etc

see: https://github.com/dirtycow/dirtycow.github.io/wiki/PoCs

TECHNICAL EXPLANATION DirtyCow Step by Step: Start

START: READ ONLY file owned by root on our physical disk GOAL: WRITE to the file mmap: maps the file on the physical disk to virtual memory The prot argument describes the desired memory protection of the mapping (and must not conflict with the open mode of the file). -Linux manual mmap MAP_PRIVATE : Create a private copy-on-write mapping. Updates to the mapping are not visible to other processes mapping the same file, and are not carried through to the underlying file. -Linux manual mmap ● Shares the same memory ● Runs in the same process ● Randomly schedule ● Shares the same memory ● Runs in the same process ● Randomly schedule madvise: give advice to the kernel about use of memory MADV_DONTNEED: hints the kernel to not except access to this region in memory DirtyCow Step by Step

map=mmap(NULL,st.st_size,PROT_READ,MAP_PRIVATE,f,0); void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset); mmap: maps the file on the physical disk to virtual memory

Result: Kernel loads a mapping of the file into memory Aside: Why mmap?

● Reading/Writing to physical memory is slow ● Virtual Memory (ram) is fast ● Load the files into memory so we can access them faster DirtyCow Step by Step

map=mmap(NULL,st.st_size,PROT_READ,MAP_PRIVATE,f,0); void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset);

> The prot argument describes the desired memory protection of the mapping (and must not conflict with the open mode of the file). PROT_READ: pages in memory can be read DirtyCow Step by Step

map=mmap(NULL,st.st_size,PROT_READ,MAP_PRIVATE,f,0); void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset);

> Create a private copy-on-write mapping. Updates to the mapping are not visible to other processes mapping the same file, and are not carried through to the underlying file. It is unspecified whether changes made to the file after the mmap() call are visible in the mapped region. MAP_PRIVATE: allows us to write! Aside: Copy on Write (COW)

● We have limited memory ● Resource loaded in memory ● Multiple “users” of that resource e.g. a file ● Share that resource between users ● If any user wants to write to that resource copy it ○ Allow them to write to the new copy ○ Other users are unaffected DirtyCow Step by Step: mmap DISK/PHYSICAL MEMORY VIRTUAL MEMORY

VIRTUAL MAPPING OF “foo” DirtyCow Step by Step: pthread

pthread_create(&pth1,NULL,madviseThread,argv[1]); pthread_create(&pth2,NULL,procselfmemThread,argv[2]);

● pthreads share the same memory ● Pros: ○ Lighter ○ Communicate easier ● Cons: ○ Harder to reason about ○ Shared memory/resources ○ Scheduling is RANDOM DirtyCow Step by Step: function 1

int f=open("/proc/self/mem",O_RDWR);

/proc/self/mem: pseudo-file that represents dirty_cow’s virtual memory

● Doesn’t actually exists on disk ● Generated by the kernel DirtyCow Step by Step: function 1 for(i=0;i<100000000;i++) { lseek(f,(uintptr_t) map,SEEK_SET); c+=write(f,str,strlen(str)); } lseek: repositions the file offset to the beginning of the file write: causes a copy on write and should write to that new copied region DirtyCow Step by Step: function 2 for(i=0;i<100000000;i++) { madvise(map,100,MADV_DONTNEED); } int madvise(void *addr, size_t length, int advice); madvise: give advice to the kernel about use of memory MADV_DONTNEED: hints the kernel to not except access to this region in memory

Result: If that region of memory is dirty then write that region of memory back to the disk otherwise do nothing. DirtyCow Step by Step: mmap DISK/PHYSICAL MEMORY VIRTUAL MEMORY

VIRTUAL MAPPING OF “foo” DirtyCow Step by Step: mmap DISK/PHYSICAL MEMORY VIRTUAL MEMORY

VIRTUAL MAPPING OF “foo” DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY lseek

VIRTUAL MAPPING OF “foo” DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY write X VIRTUAL MAPPING OF READ ONLY! “foo” DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY write X VIRTUAL MAPPING OF READ ONLY! “foo” COPY ON WRITE

COPY OF FOO DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY write

VIRTUAL MAPPING OF “foo” COPY ON WRITE write COPY OF FOO DISK/PHYSICAL MEMORY DirtyCow Step by Step: madviseThread

VIRTUAL MEMORY

madvise() VIRTUAL MAPPING OF NO CHANGES“foo” DONTNEED

COPY OF FOO DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY lseek

VIRTUAL MAPPING OF “foo” DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY write X VIRTUAL MAPPING OF READ ONLY! “foo” DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY write X VIRTUAL MAPPING OF READ ONLY! “foo” COPY ON WRITE

COPY OF FOO DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY write

VIRTUAL MAPPING OF “foo” COPY ON WRITE write COPY OF FOO DISK/PHYSICAL MEMORY DirtyCow Step by Step: madviseThread

VIRTUAL MEMORY

madvise() VIRTUAL MAPPING OF NO CHANGES“foo” DONTNEED

COPY OF FOO DirtyCow Step by Step: mmap DISK/PHYSICAL MEMORY VIRTUAL MEMORY

VIRTUAL MAPPING OF “foo” DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY lseek

VIRTUAL MAPPING OF “foo” DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY write

VIRTUAL MAPPING OF “foo” DISK/PHYSICAL MEMORY DirtyCow Step by Step: madviseThread

VIRTUAL MEMORY

madvise() VIRTUAL MAPPING OF “foo” DONTNEED ??? DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY write

MOOOOOO VIRTUAL MAPPING OF DIRTY“foo” DirtyCow Step by Step: KERNEL DISK/PHYSICAL MEMORY

VIRTUAL Kernel writes MEMORY dirty bits back to disk VIRTUAL MAPPING OF “foo”DIRTY MOOOOOO COPY OF FOO What goes wrong?

● write is not atomic ○ mem_write ■ access_remote_vm <- madvise(...) ● __get_user_pages ○ faultin_page ■ handle_mm_fault ● __handle_mm_fault ● handle_pte_fault <- madvise(...) ● do_wp_page ● PageAnon() ● reuse_swap_page <- madvise(...) ● Wp_page_reuse ○ Maybe_mkwrite ● ... What goes wrong?

● write is not atomic ○ mem_write ■ access_remote_vm Sets foll_flags:why and what kind ● __get_user_pages of access we want to the request memory ○ faultin_page FOLL_WRITE ■ handle_mm_fault ( ) ● __handle_mm_fault ● handle_pte_fault ● do_wp_page ● PageAnon() ● reuse_swap_page ● Wp_page_reuse ○ Maybe_mkwrite ● ... What goes wrong?

● write is not atomic Pages: smallest unit of data for ○ mem_write memory management in a virtual ■ access_remote_vm ● __get_user_pages memory ○ faultin_page ■ handle_mm_fault ● __handle_mm_fault ● handle_pte_fault ● do_wp_page ● PageAnon() ● reuse_swap_page ● Wp_page_reuse ○ Maybe_mkwrite ● ... What goes wrong?

● write is not atomic ○ mem_write ■ access_remote_vm ● __get_user_pages ○ faultin_page ■ handle_mm_fault ● __handle_mm_fault ● handle_pte_fault ● do_wp_page ● PageAnon() copy on write ● reuse_swap_page ● Wp_page_reuse ○ Maybe_mkwrite ● ... What goes wrong?

● write is not atomic ○ mem_write ■ access_remote_vm ● __get_user_pages <- Tries to get our requested page ○ faultin_page ■ handle_mm_fault (simplification) ● __handle_mm_fault ● handle_pte_fault ● do_wp_page ● PageAnon() ● reuse_swap_page ● Wp_page_reuse ○ Maybe_mkwrite ● ... What goes wrong?

● write is not atomic ○ mem_write ■ access_remote_vm ● __get_user_pages ○ faultin_page <- if there is a problem ■ handle_mm_fault ● __handle_mm_fault (e.g.permission error) ● handle_pte_fault faultin_page calls ● do_wp_page ● PageAnon() handle_mm_fault to handle ● reuse_swap_page that for us ● Wp_page_reuse ○ Maybe_mkwrite ● ... What goes wrong?

● write is not atomic ○ mem_write ■ access_remote_vm ● __get_user_pages ○ faultin_page ■ handle_mm_fault <- Gives us a valid page ● __handle_mm_fault ● handle_pte_fault ● do_wp_page ● PageAnon() ● reuse_swap_page ● Wp_page_reuse ○ Maybe_mkwrite ● ... What goes wrong?

● write is not atomic ○ __get_user_pages <- finds and returns our requested page ■ faultin_page (simplification) ● handle_mm_fault ○ __handle_mm_fault ○ handle_pte_fault ○ do_wp_page ○ PageAnon() ○ reuse_swap_page ○ Wp_page_reuse ■ Maybe_mkwrite ○ ... What goes wrong?

● write is not atomic ○ __get_user_pages ■ faultin_page <- if there is a problem (e.g. permission ● handle_mm_fault error)faultin_page handles that ○ __handle_mm_fault ○ handle_pte_fault For us. It contains the logic ○ do_wp_page to return a copy and return ○ PageAnon() a read only page (COW). ○ reuse_swap_page ○ Wp_page_reuse ■ Maybe_mkwrite ○ ... foll_flags = FOLL_WRITE PAGE IS READ ONLY Returns COWed Page

foll_flags = FOLL_WRITE foll_flags = FOLL_WRITE madvise: drops the COW page foll_flags = FOLL_WRITE returns ORIGINAL Page SUCCEEDS RETURNING ORIGINAL PAGE What goes wrong?

● write is not atomic ○ mem_write ■ access_remote_vm ● __get_user_pages ○ faultin_page ■ handle_mm_fault ● __handle_mm_fault ● handle_pte_fault ● do_wp_page ● PageAnon() ● reuse_swap_page ● Wp_page_reuse ○ Maybe_mkwrite ● ... Why /proc/self/mem

● /proc/self/mem is pseudo-file ● Generated by the kernel -> kernel has to access our processes’ memory ● Special access_remote_vm function (Kernel code) ○ Allows the kernel to access memory of another process ○ Normally used for debugging ● Being run by the kernel -> access to everything even read only files ● Writes the dirty page back to disk DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY write

VIRTUAL MAPPING OF “foo” DISK/PHYSICAL MEMORY DirtyCow Step by Step: madviseThread

VIRTUAL MEMORY

Causes the kernel to accidently madvise() return the VIRTUALoriginal MAPPING page OF “foo” DONTNEED DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY write

MOOOOOO VIRTUAL MAPPING OF DIRTY“foo” DirtyCow Step by Step: KERNEL DISK/PHYSICAL MEMORY

VIRTUAL Kernel writes MEMORY dirty bits back to disk VIRTUAL MAPPING OF “foo”DIRTY MOOOOOO COPY OF FOO

DISK/PHYSICAL MEMORY DirtyCow Step by Step: procselfmemThread

VIRTUAL MEMORY

VIRTUAL MAPPING OF COPY ON WRITE “foo”

COPY OF FOO DISK/PHYSICAL MEMORY DirtyCow Step by Step: function 1

VIRTUAL MEMORY

VIRTUAL MAPPING OF “foo”

write COPY OF FOO MOOOOOO DIRTY DISK/PHYSICAL MEMORY DirtyCow Step by Step: function 1

VIRTUAL MEMORY lseek

VIRTUAL MAPPING OF “foo”

COPY OF FOO DIRTY DISK/PHYSICAL MEMORY DirtyCow Step by Step: function 2

VIRTUAL MEMORY

madvise() VIRTUAL MAPPING OF “foo” DONTNEED

COPY OF FOO DIRTY DISK/PHYSICAL MEMORY DirtyCow Step by Step: function 1

VIRTUAL MEMORY write

MOOOOOO VIRTUAL MAPPING OF “foo” DIRTY

COPY OF FOO DIRTY DISK/PHYSICAL MEMORY DirtyCow Step by Step: function 1

VIRTUAL MEMORY

VIRTUAL MAPPING OF COPY ON WRITE “foo” DIRTY

COPY OF FOO DISK/PHYSICAL MEMORY DirtyCow Step by Step: function 2

VIRTUAL MEMORY

madvise() VIRTUAL MAPPING OF “foo” DONTNEED DIRTY

COPY OF FOO DirtyCow Step by Step: KERNEL DISK/PHYSICAL MEMORY

VIRTUAL Kernel writes MEMORY dirty bits back to disk VIRTUAL MAPPING OF “foo”DIRTY MOOOOOO COPY OF FOO Takeaways

● Mutexes and joins must be utilized to achieve a predictable execution order and outcome in threads ● Multithreaded code is hard PATCH see: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=19be0eaffa 3ac7d8eb6784ad9bdbc7d67ed8e619