Just-in-Time SSH Access: Why Permanent PEM Files Are a Liability

SSH key management is one of those problems that doesn't feel like a problem until it is. When your team is small and your infrastructure is simple, a handful of SSH keys — one per engineer, added to authorized_keys on the servers they need access to — works fine. It's familiar. It's standard. It's what everyone does.

Then the team grows. Engineers come and go. Contractors get temporary access that becomes permanent. A shared key gets created for "emergencies" and lives on six laptops. A deploy script gets its own key added to production servers. The keys accumulate. Nobody has a complete inventory of what key is on what server.

This is the SSH key problem. And just-in-time (JIT) SSH is the solution.

The problem with permanent SSH keys

Permanent SSH keys have a fundamental property that makes them increasingly dangerous over time: they accumulate.

Every time someone needs server access, a key is added. When that person no longer needs access — they leave the company, the project ends, the contract expires — the key often stays. Removing it requires knowing which servers it's on (you probably don't have a complete list) and having the access to remove it from each one.

The result is that authorized_keys files grow over time, containing keys for people and systems that should no longer have access. In a post-mortem after an incident, you might find a key from an engineer who left two years ago, or a key from a contractor who finished their engagement 18 months ago.

This matters for two reasons:

Security surface. Every authorized key is a potential entry point. An engineer who left the company still has their key. Their laptop might be sold. Their GitHub private key might be exposed in a breach. If their key is still in authorized_keys on your production servers, they (or someone who gets their key) still has access.

Auditability. If something happens on a server, who had access? With permanent keys, you can answer "who has a key on this server right now" (barely — only if you audit every authorized_keys file), but you can't easily answer "who accessed this server last Thursday at 2pm."

What just-in-time SSH is

Just-in-time SSH is an access model where credentials are issued at the time of access and expire after a defined period. There are no permanent authorized_keys entries. Access isn't standing — it's requested, verified, granted for a session, and then gone.

The mechanics vary by implementation, but the general pattern is:

Access request — A user or system requests access to a specific target. The request includes a target host, a requested TTL (e.g., 4 hours), and authentication credentials.
Verification — The JIT system verifies that the requester is authorized to access the requested target.
Credential issuance — A time-limited SSH certificate or token is issued. This might be a short-lived SSH certificate signed by a CA that the target trusts, or a token that the JIT bastion validates on connection.
Scoped connection — The SSH connection is routed through the JIT system (as a bastion), which validates the credential and logs the session.
Expiry — When the TTL expires, the credential is invalid. No cleanup required. The session ends, and there's nothing to revoke on the target host.

JIT SSH vs traditional bastion hosts

Traditional bastion hosts (jump servers) solve the problem of network access — you connect to the bastion, and from there to the target. But they don't solve the credential problem. You still need a permanent SSH key on the bastion, and often permanent keys on the targets too. The bastion is a single point for network routing, not for access control.

JIT SSH combines the bastion concept with credential lifecycle management. The bastion isn't just a jump point — it's the authority that validates credentials, enforces TTLs, and logs sessions. Access that hasn't been explicitly granted for the current session isn't possible, regardless of what keys might exist on the target host.

The security properties of JIT SSH

JIT SSH gives you several security properties that permanent keys can't provide:

No standing access. An attacker who compromises a user's machine gets their current session token, which has a TTL measured in hours, not years. After it expires, access is gone.

Instant revocation. If a laptop is reported stolen, you revoke the session token immediately — one action that affects all targets. With permanent keys, you'd have to find and remove the key from every server it's authorized on.

Scoped access. Tokens can be scoped to specific targets. An engineer working on the database doesn't need (and doesn't get) a token that works on the payment processing servers. Scope is enforced at issuance time, not as an honor system.

Complete audit trail. Every connection request, every token issuance, and every session is logged. You can answer "who accessed prod-db-01 last Thursday between 2pm and 4pm" with a query, not an investigation.

Works with zero trust principles. JIT SSH implements the core zero trust properties: never trust by default, always verify, minimize access scope. Every session requires explicit authorization. There's no implicit trust from having a key in authorized_keys.

The operational case for JIT SSH

Security teams often lead the conversation about JIT SSH, but there's a strong operational case too.

When an engineer leaves the company, offboarding SSH access with permanent keys requires auditing every server they had access to and removing their key. This is tedious, error-prone, and often incomplete. With JIT SSH, offboarding is: disable their account in the JIT system. Done. Their next connection attempt will fail. No server-by-server cleanup required.

When a contractor finishes an engagement, you don't need to run an SSH key audit. Their token expires, and that's the end of their access. If you want to be thorough, you revoke any active tokens immediately. One operation, complete.

Access reviews become tractable. Instead of querying authorized_keys on every server (or hoping your configuration management is current), you query the JIT system's access log. Who requested access to what, when, and for how long? The answer is in one place.

Common objections

"Our existing servers don't support JIT SSH." Most JIT SSH implementations work with standard OpenSSH, so target hosts don't need any changes. The JIT system acts as a CA that signs SSH certificates the target's sshd already trusts, or as a bastion that handles authentication at the perimeter.

"What about automation? My CI/CD system needs persistent SSH access." Automation is a separate case. CI/CD systems should use purpose-specific service accounts with tokens scoped to exactly what they need. JIT SSH's TTLs can be set for longer periods for these cases, or service tokens can be renewed automatically — similar to how TLS certs are renewed.

"This seems complex to set up." Modern JIT SSH implementations are significantly simpler than they were a few years ago. CertLocker's JIT SSH works with standard SSH infrastructure and doesn't require installing agents on target hosts.

Getting started

The migration from permanent keys to JIT SSH doesn't have to be immediate and complete. A practical path:

Stand up the JIT SSH system in parallel with existing access.
Start routing new access through JIT for non-critical environments.
Run an access review and remove known-stale permanent keys.
Gradually migrate teams to JIT, starting with new engineers who don't have permanent keys yet.
Once JIT is established as the path for all new access, begin revoking permanent keys for engineers who have migrated.

The end state is an infrastructure where every SSH session has a verified requester, a bounded duration, a specific target scope, and a log entry. That's a material security improvement over the alternative.

Summary

Permanent SSH keys accumulate over time, creating a growing attack surface that's difficult to audit and harder to clean up. Just-in-time SSH replaces standing credentials with time-limited, scoped tokens that expire automatically and revoke instantly.

The security properties are clear: no standing access, instant revocation, scoped authorization, complete audit trail. The operational benefits are real: simpler offboarding, tractable access reviews, no key rotation headaches.

If your team manages more than a handful of servers and more than a handful of engineers, JIT SSH is worth the migration effort. The question isn't whether permanent keys will cause a problem — it's when.