Introduction
As part of my job, I was asked to analyze the recent disclosed CVE-2021-22986. By the time I wrote this, I saw some other analysis online which differs from mine, so I thought it’d be interesting to share my point of view. Also note that I am a noob, so there is a chance I understand the vulnerability wrong. Without further ado, let’s get into it.
From the information told by F5 official, we know this CVE is a unauthenticated RCE. So while our team was poking at the application, my team leader decided to fuzz the entire REST API URL and all he got was 401 Unauthorized. Then he made a conclusion that it must be a authentication bypass vulnerability otherwise how are you supposed to gain access to those APIs? And with that in our mind, later we found there are two authentication bypass vulnerabilities in the entire application.
But first, let’s understand how will a HTTP request reach the backend server. Once a client has sent a HTTP request to the server, it will first go through Apache, after Apache does some credential and header checking, it will forward the request to a Jetty server, which is written in Java. Then, you can guess it, the Jetty does some other authentication stuff and respond to the client.
Now, let me talk about how we did our analysis. In F5 Big-IP application, the https:<host>/mgmt/ URL is for management purpose, and certainly it would ask for authentication. But when we were fuzzing directories, we found out https://<host>/mgmt/toc would return a 200 OK. The page asks us for username and password, but the status code definitely caught our eyes. Then we searched the related files on server side with the keyword of /mgmt/toc, and we found a binary file matches the search, mod_auth_pam.so. This is a shared library for Apache, and we thought maybe the first authentication bypass exists in this file.
Finally, my team leader found that he could bypass authentication with an empty HTTP X-F5-Auth-Token and a basic authentication header with only username: Authorization: Basic $(base64_encode("admin:")). Later we found out the basic authentication only checks for the admin username, but not the password, and I will go through that in a bit. Then with a little researching, I found the API to execute system command is at https://<host>/mgmt/tm/util/bash, and that’s how we achieved unauthenticated RCE on the F5 Big-IP server.
Now let’s analyze what went wrong.
The Actual Analysis
I hope you still remember the scientific method from your science class, because we will be using that today. As I mentioned before, there are two layers of authentication happening in this application. And we have two headers to bypass authentication, by toggling those headers we can know how the server is doing its filtering.
Let’s first try when one of the headers appear in the request:
When only an empty X-F5-Auth-Token is present, the respond is sent by Jetty.
When only basic authentication is present, the respond is sent by Apache, and note that the only the username in basic authentication is correct.
And with both headers present, haza!
And I forgot to take screenshot, but you can also get a 200 OK with only correct basic auth credentials.
So with the above information we can conclude that Apache does check credentials of basic auth, only when there is no X-F5-Auth-Token; and Jetty doesn’t even bother to check basic auth values.
Let’s look at why. We shall start from mod_auth_pam.so:
Apache Auth Bypass
I tried to load the binary in IDA Pro, but for some reason IDA was never able to decompile one function into pseudo C code, and guess what? That particular function is the most important one. However, with some keyword searching and logical graph, I still managed to get a rough idea.
First Apache checks the X-F5-Auth-Token header, but I believe it does not check if the value is valid or not.
If the token exists, it will proceed to some miscellaneous URL checks, and the keyword(/mgmt/toc/) we were searching for before is also here. But if the header does not exist, it will jump to some other header checking functions, and in there, it checks for the basic auth header and its values.
From the function call below we can assume it’s checking if the credential is legit.
So now that explains it, the problem is Apache only checks if the X-F5-Auth-Token headers exists in request, but not validating its value, so with an empty header we can avoid getting checked by Apache for basic auth.
But what about the Jetty side? How come it doesn’t check both headers? Next, let’s take a look.
Jetty Auth Bypass
I searched for root directory of Jetty server, and I was able to find multiple .jar files, I put all of them in jd which is a Java decompiling software and searched for the two headers we were looking for. With the result, I found the f5.rest.jar being the most suspicious one.
Here in the f5.rest.workers.authz.AuthzHelper.class,
public static BasicAuthComponents decodeBasicAuth(String encodedValue) {
BasicAuthComponents components = new BasicAuthComponents();
if (encodedValue == null) {
return components;
}
String decodedBasicAuth = new String(DatatypeConverter.parseBase64Binary(encodedValue));
int idx = decodedBasicAuth.indexOf(':');
if (idx > 0) {
components.userName = decodedBasicAuth.substring(0, idx);
if (idx + 1 < decodedBasicAuth.length())
{
components.password = decodedBasicAuth.substring(idx + 1);
}
}
return components;
}
we can see how Jetty deals with basic auth header. And in f5.rest.RestOperationIdentifier.class, we have:
private static boolean setIdentityFromBasicAuth(RestOperation request) {
String authHeader = request.getBasicAuthorization();
if (authHeader == null) {
return false;
}
AuthzHelper.BasicAuthComponents components = AuthzHelper.decodeBasicAuth(authHeader);
request.setIdentityData(components.userName, null, null);
return true;
}
}
public RestOperation setIdentityData(String userName, RestReference userReference, RestReference[] groupReferences) {
if (userName == null && !RestReference.isNullOrEmpty(userReference)) {
String segment = UrlHelper.getLastPathSegment(userReference.link);
if (userReference.link.equals(UrlHelper.buildPublicUri(UrlHelper.buildUriPath(new String[] { WellKnownPorts.AUTHZ_USERS_WORKER_URI_PATH, segment }))))
{
userName = segment;
}
}
if (userName != null && RestReference.isNullOrEmpty(userReference)) {
userReference = new RestReference(UrlHelper.buildPublicUri(UrlHelper.buildUriPath(new String[] { WellKnownPorts.AUTHZ_USERS_WORKER_URI_PATH, userName })));
}
this.identityData = new IdentityData();
this.identityData.userName = userName;
this.identityData.userReference = userReference;
this.identityData.groupReferences = groupReferences;
return this;
}
And after the setIdentity call, the request variable becomes:
identityData.userName = 'admin';
identityData.userReference = 'http://localhost/mgmt/shared/authz/users/admin'
identityData.groupReference = null;
which will be used later.
Lastly, in f5.rest.workers.EvaluatePermissions.class, the problem showed up:
private static void completeEvaluatePermission(final RestOperation request, AuthTokenItemState token, final CompletionHandler<Void> finalCompletion) {
final String path;
//since we didn't provide a value, so the value is null.
if (token != null) {
if (token.expirationMicros.longValue() < RestHelper.getNowMicrosUtc()) {
String error = "X-F5-Auth-Token has expired.";
setStatusUnauthorized(request);
finalCompletion.failed(new SecurityException(error), null);
return;
}
request.setXF5AuthTokenState(token);
}
//the request object is discussed above. and notice something? right, the function call does not check if the password is valid.
request.setBasicAuthFromIdentity();
//since the URL doesn't match, so it will skip the following two conditional block.
if (request.getUri().getPath().equals(EXTERNAL_LOGIN_WORKER) && request.getMethod().equals(RestOperation.RestMethod.POST)) {
finalCompletion.completed(null);
return;
}
if (request.getUri().getPath().equals(UrlHelper.buildUriPath(new String[] { EXTERNAL_LOGIN_WORKER, "available" })) && request.getMethod().equals(RestOperation.RestMethod.GET)) {
finalCompletion.completed(null);
return;
}
//the userRef will be admin as we provided admin in the basic auth header.
final RestReference userRef = request.getAuthUserReference();
if (RestReference.isNullOrEmpty(userRef)) {
String error = "Authorization failed: no user authentication header or token detected. Uri:" + request.getUri() + " Referrer:" + request.getReferer() + " Sender:" + request.getRemoteSender();
setStatusUnauthorized(request);
finalCompletion.failed(new SecurityException(error), null);
return;
}
//by default, the default admin is just "admin", so in this case, we are default admin. "root" also works, because the local admin on machine is also set to be default admin. and that's it, since we satisfy the condition, it will return complete, and thus we are authenticated.
if (AuthzHelper.isDefaultAdminRef(userRef)) {
finalCompletion.completed(null);
return;
}
//the rest is useless for us as we are already authenticated and those code won't get to executed.
if (UrlHelper.hasODataInPath(request.getUri().getPath())) {
path = UrlHelper.removeOdataSuffixFromPath(UrlHelper.normalizeUriPath(request.getUri().getPath()));
} else {
path = UrlHelper.normalizeUriPath(request.getUri().getPath());
}
final RestOperation.RestMethod verb = request.getMethod();
if (path.startsWith(EXTERNAL_GROUP_RESOLVER_PATH) && request.getParameter("$expand") != null) {
String filterField = request.getParameter("$filter");
if (USERS_GROUP_FILTER_STRING.equals(filterField) || USERGROUPS_GROUP_FILTER_STRING.equals(filterField)) {
finalCompletion.completed(null);
return;
}
}
if (token != null && path.equals(UrlHelper.buildUriPath(new String[] { EXTERNAL_AUTH_TOKEN_WORKER_PATH, token.token }))) {
finalCompletion.completed(null);
return;
}
roleEval.evaluatePermission(request, path, verb, new CompletionHandler<Boolean>()
{
public void completed(Boolean result)
{
if (result.booleanValue()) {
finalCompletion.completed(null);
return;
}
String error = "Authorization failed: user=" + userRef.link + " resource=" + path + " verb=" + verb + " uri:" + request.getUri() + " referrer:" + request.getReferer() + " sender:" + request.getRemoteSender();
EvaluatePermissions.setStatusUnauthorized(request);
finalCompletion.failed(new SecurityException(error), null);
}
public void failed(Exception ex, Boolean result) {
request.setBody(null);
request.setStatusCode(500);
String error = "Internal server error while authorizing request";
finalCompletion.failed(new Exception(error), null);
}
});
}
Then from the F5 website we can find which endpoint can let us execute code, follow the instruction and boom!
Conclusion
And we took a look at the patched version, for both .jar and the .so file, unfortunately, I don’t think there is a way to bypass that again, but as the first CVE I analyzed, I’m quite happy with what I have come to accomplished.
From what I’ve noticed, there is also a SSRF lead to RCE vulnerability being talked on Twitter and other social medias. I was not able to find that and that’s probably the reason of why I suck, but I mean, the one I found also works… There is one thing I’m wondering though, is this one considered to be “unauthenticated”? We did bypass it, but is it still authenticated…? Anyway, hope you can take a thing or two from this post. You can find the official analysis written by me here (in Chinese tho).
Reference
https://support.f5.com/csp/article/K02566623
https://support.f5.com/csp/article/K43371345
https://attackerkb.com/topics/J6pWeg5saG/k03009991-icontrol-rest-unauthenticated-remote-command-execution-vulnerability-cve-2021-22986
! 