One of our clients wanted to build an in-browser IDE experience for their assessment platform, similar to CodeSandbox, so that college students could develop their projects without leaving their browsers. The projects would be auto-saved and support collaborative features.

We explored and experimented with multiple open-source tools. Initially, we tested Eclipse Che and Theia editor.

Eclipse Che: An open-source developer workspace server and cloud IDE, Eclipse Che provides a Kubernetes-native IDE that runs in containers. It offers a robust and flexible development environment.

Theia: A cloud and desktop IDE platform that is highly customizable, Theia aims to be compatible with VS Code extensions while providing flexibility in terms of deployment and customization.

Despite the potential of these tools, we encountered several roadblocks and found the community support lacking. Therefore, we decided to use code-server, which runs VS Code in the browser. VS Code is the most popular editor, familiar to many developers, making it an excellent choice for students.

Creating On-Demand Runtime Environments

We needed to create on-demand runtime environments with code-server and manage them efficiently to preserve data and control costs. Kubernetes, specifically EKS (Elastic Kubernetes Service), was the ideal choice. Kubernetes manages resources efficiently, and its pods can share underlying instances, reducing costs. Additionally, Kubernetes is vendor-neutral and has excellent community support.

Using StatefulSets for On-Demand Environments

We used StatefulSets in Kubernetes to create on-demand coding environments with different Docker images for various tech stacks. For example, we created a React development environment with Vite using the following Dockerfile:

# Use the specified base image
FROM lscr.io/linuxserver/code-server:latest

# Update and install dependencies
RUN apt-get update && apt-get install -y ca-certificates curl gnupg

# Add NodeSource repository for Node.js
RUN mkdir -p /etc/apt/keyrings
RUN curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
RUN echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list

# Update package list after adding new repository
RUN apt-get update

# Install Node.js (includes npm)
RUN apt-get install -y nodejs

# Check Node.js and npm versions
RUN node -v
RUN npm -v

# Install extension for reload and testing
RUN ./app/code-server/bin/code-server --extensions-dir /config/extensions --install-extension antfu.vite

RUN npm install

WORKDIR /config/workspace

RUN npm install -g create-react-app

# Expose ports
EXPOSE 8443 8080

Workflow

The workflow for students was as follows:

1. Students select a project and create a workspace (our code name for the IDE environment).
2. We create a StatefulSet with a Docker image specific to that project.
3. We create an Ingress and Service for the StatefulSet to make it accessible.

Managing Resources Efficiently

To manage resources efficiently, we implemented a cron job that runs every few minutes to check for unused workspaces using code-server’s heartbeat and health APIs. Unused workspaces are deleted to optimize resource utilization.

Autoscaling with Karpenter

We set up Karpenter in our EKS cluster to handle autoscaling. Karpenter is an open-source cluster autoscaler built by AWS. It provisions the right compute resources for Kubernetes workloads based on their specific requirements, ensuring optimal performance and cost efficiency. When there is a high number of workspaces created, Karpenter automatically scales the cluster by adding more nodes.

Conclusion

By leveraging code-server, Kubernetes, and Karpenter, we successfully developed a custom in-browser IDE like CodeSandbox for our client. This solution provided an efficient, scalable, and familiar environment for students to develop their projects. Our approach ensured optimal resource utilization and cost efficiency, creating a robust and user-friendly platform for collaborative coding and project development.

This project highlights our ability to integrate and customize open-source tools to meet specific client needs, delivering a high-performance solution that enhances the user experience and operational efficiency.