Netflix gives Spark jobs internal identities by having AWS verify their roles
Netflix engineer Dhruv Pratap's design pairs AWS STS proof with signed workload metadata, then issues Metatron certificates for a Data Project identity.
By RuntimeWire Staff · Published
Primary source: Netflix Technology Blog
Why it matters
Netflix's design shows how managed-compute workloads can use an organization's internal identity without trusting the workload to identify itself. Its security depends on a correct role-to-project mapping and a tightly controlled signer.
Netflix engineer Dhruv Pratap describes a way for Spark jobs on Amazon EMR to carry a Netflix identity that internal services can trust, even though the managed service starts them with AWS credentials. In a Netflix Technology Blog post published September 25th, Pratap lays out a system that makes AWS attest to the cloud role a process holds while Netflix's control…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.