The impact can be mitigated by having low TTLs and using refresh tokens. This will give you a rolling window. If the TTL is 10 minutes and the client doesn't make any requests in 10 minutes, they will be timed out. But if the client continues to have at least one request every 10 minutes the session persist. Session persistence can also be ensured by having your web-client for example make a request to the server every couple of minutes.
Server-side key invalidation is entirely possible but it would require having a blacklist of disabled keys and comparing every requests against the black list. This would obviously concede the benefits of scale from JWT tokens since you are doing the same thing as server side sessions. However, the black list should be considered only as an escape hatch and need not be enabled at all times. In fact, once all the tokens in the black list expire, the black list itself can be disabled and things go back to the way they were.
Server-side key invalidation is entirely possible but it would require having a blacklist of disabled keys and comparing every requests against the black list. This would obviously concede the benefits of scale from JWT tokens since you are doing the same thing as server side sessions. However, the black list should be considered only as an escape hatch and need not be enabled at all times. In fact, once all the tokens in the black list expire, the black list itself can be disabled and things go back to the way they were.