Rechercher
Fermer ce champ de recherche.

Vous êtes victime d’un incident de sécurité ? Contactez notre CERT

13/02/2017

Blog technique

Virtualization Based Security – Part 2: kernel communications

Adrien Chevalier

This blog post is a second article covering Virtualization Based Security and Device Guard features. In the [first part]({filename}/content/article11.md), we covered the system boot process, from the Windows bootloader to the VTL0 startup. In this part, we explain how kernel communications between VTL0 and VTL1 actually work. As they use hypercalls to communicate, we will first describe the Hyper-V hypercalls implementation, then how the kernels use them to communicate. To finish with, we list all the different hypercalls and secure service calls we have identified during this work.

Hyper-V hypercalls

Kernel communications between VTL0 and VTL1 use Hyper-V hypercalls. These hypercalls are performed using the VMCALL instruction, with the hypercall number in the RCX register, and RDX pointing to a Guest Physical Page (GPA) which contains the parameters. If the 0x10000 RCX bit is set, the hypercall is a “FAST” hypercall, and the parameters (and the return values) are stored in R8, RDX and XMM registers (size lays on the R9 register). In order to perform the call, Windows uses a “hypercall trampoline”, which is a small *fastcall *routine which basically executes a VMCALL and RET.

This routine is stored in the “hypercall page”. This page contains at this day five trampolines, and is given by Hyper-V at its startup to winload.efi, which will copy it in the VTL0 and VTL1 address spaces. The main difference between these five trampolines is that the first one just does a VMCALL/RET, but the four next ones (they are defined contiguously) store RCX into RAX, and enforce a fixed value into RCX. The second and the third enforce RCX to 0x11, and the next ones to 0x12.

These four trampolines are actually used by the different VTLs. Each kernel may “ask” Hyper-V for the 0xD0002 Virtual Processor Register value (internal Hyper-V values which can be queried or set using their identifier) using a dedicated hypercall, which will return two offsets. These offsets are hypercall page related, and will be used by the kernel to call the correct trampolines. Actually, VTL1 and VTL0 use the 0x11 trampoline to communicate with each other, and the VTL1 uses the 0x12trampoline to finalize its initialization.

As x86 and x64 hypercalls does not behave the same way (x86 hypercalls numbers are set in the EAX register), the page also embeds x86 trampolines and HyperV caches two 0xD0002 virtual registers and serves them regarding the caller’s processor mode.

The hypercall page’s content can be represented as:

				
					VMCALL               <-- first trampoline
RET
MOV ECX, EAX         <-- second one (enforces 0x11, x86)
MOV EAX, 0x11 
VMCALL
RET 
MOV RAX, RCX         <-- third (0x11, x64)
MOV RCX, 0x11 
VMCALL 
RET 
MOV ECX, EAX         <-- fourth (0x12, x86)
MOV EAX, 0x12 
VMCALL 
RET 
MOV RAX, RCX         <-- fifth (0x12, x64)
MOV RCX, 0x12 
VMCALL 
RET 
db 0x00 
db 0x00 
db 0x00 
db 0x00 
db 0x00 
db 0x00 
... 
... 
db 0x00 
				
			

We therefore have five trampolines, at offsets 0x00, 0x04, 0x0F, 0x1D and 0x28. Note that their contents can be easily retrieved in Windows crashsdumps using WinDbg or within the Hyper-V binary (hvix64.exe/hvax64.exe for intel/amd) inner code.

Remark: several hypercalls may specify a data size in the 12 least significant bits of the RCX most significant DWORD. These sizes are not the data length in bytes, but are related to the current call, and may indicate entries count, etc.

For a hypercall example, the VTL1’s ShvlProtectContiguousPages hypercall (12) parameter is a structure which follows this scheme:

				
					typedef struct _param {
    ULONGLONG infinite; // always 0xFFFFFFFFFFFFFFFF
    ULONG protection_asked; // 0xD EXECUTE, 0xF WRITE
    ULONG zero_value; // always 0 
    ULONGLONG pfns[]; // entries count is set in the hypercall number
} param;

				
			

In order to tell Hyper-V the pfns parameter size, RCX high DWORD must contain its elements count. For only one entry and a *FAST *hypercall, RCX value will therefore have to be 0x10010000C.

Secure kernel hypercalls

The two VTL are able to perform multiple hypercalls in order to communicate with Hyper-V. They may perform the same hypercalls, but Hyper-V will refuse several hypercalls if they are called from VTL0. The two VTL also use one dedicated hypercall to communicate with each other. This is summarized in Figure 1 below.

Figure 1: Hypercalls categories

Let’s first describe the “VTL1 to Hyper-V” hypercalls (in green). We will then cover the 0x12 hypercall later.

The VTL1 can use three hypercall trampolines:

  • ShvlpHypercallCodePage, which is equivalent of the NTOS HvlpHypercallCodePage one (offset 0), and points over the first trampoline;
  • ShvlpVtlReturn, which enforces RCX to 0x12 and allows for VTL0 and VTL1 communications;
  • ShvlpVtlCall, which enforces RCX to 0x11 and is only used during the VTL1 initialization.

The last two are retrieved using the 0xD0002 virtual register (through the 24 least significant bits of the ShvlpGetVpRegister return value, each offset is 12 bits length). These two offsets point towards the 0x11 and the 0x12 trampolines.

By the way, the VTL0 NTOS kernel gets its HvlpVsmVtlCallCodeVa value (the hypercall trampoline used for VTL0 to VTL1 communications) using the same process, as any VM gets the same hypercall page from Hyper-V.

The following tables give the possible VTL1 hypercalls, for each trampoline:

ShvlpVtlCall

Hypercall number
Caller - Usage
0x11
ShvlInitSystem – end of VTL1 initialization?

ShvlpVtlReturn (VTL0 returns/calls)

Hypercall number
Caller - Usage
0x12
SkCallNormalMode/SkpPrepareForReturnToNormalMode – returns to NTOS / calls NTOS

ShvlpHypercallCodePage (HyperV)

Remark: We wrote an (H) when realizing an hypothesis.

Hypercall number
Caller - Usage
0x2
ShvlFlushEntireTb, etc. - Translation buffer flushs
0x3
Translation buffer flushs
0xB
SkpgPatchGuardCallbackRoutine – (H) HyperGuard delayed routines registering
0x15
0xC
ShvlProtectContiguousPages – Memory protection modification
0xD
ShvlInitSystem – Called just before ShvlEnableVpVtl, seems to send several settings to HV
0xF
ShvlEnableVpVtl – Sends settings to HV, and notably the ShvlpVtl1Entry function pointer
0x50
ShvlGetVpRegister – Gets a virtual processor (VP) register
0x51
ShvlSetVpRegister – Sets a virtual processor register
0x52
SkpgTranslateVa – (H) VTL0 memory access by HyperGuard
0x86
ShvlPrepareForHibernate
0x87
ShvlNotifyRootCrashDump
0x94
BugCheck
0x8E
LiveDumpCollect
0x97
SkpGetPageList
0xAE
ShvlSetGpaPageAttributes – 1607 build: changes a GPA attributes, seems to only been used on VTL1 memory

VTL0 to VTL1 transition

Almost all of NTOS “Vsl” prefixed functions end up in VslpEnterIumSecureMode, with a Secure Service Call Number (SSCN). This functions calls HvlSwitchToVsmVtl1, which uses the HvlpVsmVtlCallVa hypercall trampoline (regular hyper-V hypercalls use HvcallCodeVa trampoline). The SSCN is then copied into RAX, and RCX value is set to 0x11.

Hyper-V dispatches the 0x11 hypercall into the securekernel.exe function SkpReturnFromNormalMode, which then calls IumInvokeSecureService (actually we are not sure IumInvokeSecureService is called directly or not, we think that SkpReturnFromNormalMode must be called first in order to make IumInvokeSecureService returning to VTL0 after secure service call completion). IumInvokeSecureService is mostly a big switch/case block, which handles all possible SSCNs.

Finally, the SkCallNormalMode is called, which ends up on SkpPrepareForReturnToNormalMode. Actually, the secure kernel’s NTOS calls can be considered as “fake returns” to VTL0, since they consist in 0x11 hypercalls too.

We have identified all the possible SSCNs from VTL0 in the array below. For each one, we only specified the called functions as their name is generally self-explanatory. The corresponding parameters must be however retrieved manually by reverse engineering the VTL0 callers or the VTL1 callees.

SSCN
Called function
1
SkmmInitializeUserSharedData / SkInitSystem
2
SkeStartProcessor
3
SkpsRegisterSystemDll
4
InterlockedCompareExchange(IumSystemProcessRegistered) / PsIumSystemProcess manipulation
5
SkmmCreateProcessAddressSpace/SkobCreateHandle
6
SkeInitializeProcess
7
IumCreateThread
8
SkiTerminateAllThreads
9
IumTerminateThread
10
SkeRundownProcess
11
SkpsIsProcessDebuggingEnabled / SkmmDisableProcessMemoryProtection
12
Unknown
13 & 14
SkmmMapMdl / IumpGetSetContext
15
SkeReferenceProcessByHandle / SkmmMapDataTransfer / SkpEncryptWithTrustletKey
16
SkeReferenceProcessByHandle / Unknown
17
SkRetrieveMailbox
18
SkIstTrustletRunning
19
SkmmCreateSecureAllocation
20
SkmmMapDataTransfer / SkmiFillSecureAllocation
21
SkmmConvertSecureAllocationToCatalog
22
SkmmCreateSecureImageSection
23
SkmmFinializeSecureImageHash
24
SkmmFinishSecureImageValidation
25
SkmmPrepareImageRelocations
26
SkmmRelocateImage
27
SkobCloseHandleEx
28
SkmmValidateDynamicCodePages
29
SkmmTransferImageVersionResource
30
EntropyProvideData / BCryptGenRandom
31
SkpEncryptHiberData / SkpSetHiberCrashState
32
SkpSetHiberCrashState / SkpgHibernateActive = 0 / SkFinalizePageEncryption
33
SkmmConfigureDynamicMemory
34
IumConnectSwInterrupt
35
Unknown = 0x3000
36
SkLiveDumpStart
37
SkpLiveDumpContext / Unknown
38
SkLiveDumpSetupBuffer
39
SkLiveDumpFinalize
40
SkpLiveDumpFreeContext / SkpReleaseLiveDumpLock
41
SkNotifyPowerState
42
IumDispatchQueryProfileInformation
192
SkeReferenceProcessByHandle
193
SkmmValidateSecureImagePages
208
SkmmInitSystem / IumpInitializeSystem
209
SkpWorkItemList / Unknown
210
SkmiReleaseUnprivilegedPagesInRange / SmiReserveNtAddressRange
211
SkmmApplyDynamicRelocations
212
SkEtwEnableCallback
224
SkiAttachProcess / SkmiFlushAddressRange
225
SkmmFastFlushRangeList
226
SkmmSlowFlushRangeList
227
SkmmRemoveProtectedPage
228
SkmmCopyProtectedPage
229
SkmmMakeProtectedPageWritable
230
SkmmMakeProtectedPageExecutable
231
(H) Gets *Skmi *flags
232
SkhalEfiInvokeRuntimeService
233
SkLiveDumpCOllect
234
SkmmRegisterFailureLog
235
SkPrepareForHibernate
236
SkPrepareForCrashDump
237
SkhalpEfiRuntimeInitialize / SkhalpReportBugCheckInProgress
238 & 240
Returns an error code
241
SkKsrCall
Otherwise
SkeBugCheckEx(0x121, 0xFFFFFFFFC000001C, , 0, 0)
As you can see, several called functions are unknown. This is because they do not perform obvious calls, and we did not spent a lot of time on analyzing what they manipulate.

Conclusion

Almost all of NTOS “Vsl” prefixed functions end up in VslpEnterIumSecureMode, with a Secure Service Call Number (SSCN). This functions calls HvlSwitchToVsmVtl1, which uses the HvlpVsmVtlCallVa hypercall trampoline (regular hyper-V hypercalls use HvcallCodeVa trampoline). The SSCN is then copied into RAX, and RCX value is set to 0x11.

Hyper-V dispatches the 0x11 hypercall into the securekernel.exe function SkpReturnFromNormalMode, which then calls IumInvokeSecureService (actually we are not sure IumInvokeSecureService is called directly or not, we think that SkpReturnFromNormalMode must be called first in order to make IumInvokeSecureService returning to VTL0 after secure service call completion). IumInvokeSecureService is mostly a big switch/case block, which handles all possible SSCNs.

Finally, the SkCallNormalMode is called, which ends up on SkpPrepareForReturnToNormalMode. Actually, the secure kernel’s NTOS calls can be considered as “fake returns” to VTL0, since they consist in 0x11 hypercalls too.

We have identified all the possible SSCNs from VTL0 in the array below. For each one, we only specified the called functions as their name is generally self-explanatory. The corresponding parameters must be however retrieved manually by reverse engineering the VTL0 callers or the VTL1 callees.

Update

Thanks to a reader, we identified that we misread the HyperCallPage contents, and made a few modifications. We thought that there were all x64 ones, and that HyperV served one or the other depending on the VTL level, and inverted the values.

Disclaimer

These findings have almost all been retrieved by static analysis using IDA Pro. We apologize if they contain mistakes (actually they probably do), and ask the readers to take this “as it is”. Any helpful remark or criticism is welcome, just email us at etudes{at}amossys.fr!

Voir les derniers articles du Blog technique

11 avril 2024
M&NTIS Platform est une solution SaaS destinée au test d'efficacité de produits de défense (AV, EDR, sondes réseau/NDR, SIEM, XDR, ...) et d'architectures de supervision. Une nouvelle version majeure de la plateforme vient de sortir.
25 mars 2024
Through this article, we propose a way to find LPE in Windows applications, by using SysInternals tools. What and how to look at? How to exploit in an easy and quick way?
31 janvier 2024
During the inter-CESTI challenge organized by ANSSI, many vulnerabilities were included to test our abilities to find them and, if possible, to exploit them. In this article, we explore one vulnerability that we found during the original challenge, and explainhow we exploited it: can we find a secret key from a singlesignature?
4 décembre 2023
Find here the crypto and reverse challenges that our teams created for the European Cyber Week pre-qualification and qualification tests of CTF, a recognized cybersecurity event that took place in Rennes from November 21 to 23, 2023.
29 mars 2023
On the 24th of January, AMOSSYS and Malizen put together a Blue Team CTF, for the SupSec seminar organized by Inria. In this blog post, we explain how we, at AMOSSYS, generated the dataset used in this challenge.
20 décembre 2022
Find here the crypto and web challenges that our teams created for the European Cyber Week pre-qualification tests of CTF, a recognized cybersecurity event that took place in Rennes from November 15 to 17, 2022.
22 novembre 2022
Find here the write-ups of the crypto and web challenges that our teams created for the European Cyber Week pre-qualification tests of CTF
14 septembre 2022
This article starts with a quick overview on NIDS (Network Intrusion Detection System) evasions to remind what it is and why it could happen.
31 mars 2021
Essor du numérique, diversification des surfaces d’exposition, multiplication des cyberattaques… Depuis plusieurs années, la sécurité informatique est devenue une composante essentielle de l'administration d'un Système d'Information (SI).
18 septembre 2020
Depuis plusieurs années, l'écosystème informatique a dû faire face à une recrudescence de compromissions de systèmes d'informations par des rançongiciels, ou cryptolockers, qui s'introduisent principalement par des méthodes automatiques (_spear phishing_, etc.).