Static Site Generator

This project is a Static Site Generator written in Scala. It generates a static website by processing Markdown files, converting them into HTML pages, and syncing them with the server hosting the blog. The project uses modern web technologies like Bootstrap for styling and Highlight.js for syntax highlighting.

Project Structure:

blog/  
├── build.mill               
├── StaticSite/              
│   ├── src/  
│   │   ├── gen/  
│   │   │   └── Base.scala       
│   │   └── Generator.scala                    
├── sync/                      
│   ├── src/  
│   │   └── Sync.scala      
│   ├── conf/  
│   │   └── Util.scala      
│   ├── resources/      
│   │   └── conf.json           
├── agent/                     
│   ├── src/  
│   │   └── Agent.scala             
├── shared/                       
│   ├── src/  
│   │   ├── Shared.scala  
│   │   └── Rpc.scala                 
└── .gitignore          

Tech Stack:

• Build tool:

  • Mill: A Scala build tool that simplifies the build process and dependency management.
    
    

• Language:

  • Scala: The entire project is written in Scala, leveraging its functional programming capabilities.
    
    

• Libraries:

  • os-lib: Used for file system operations and spawning/managing sub processes.
  • scalatags: For generating HTML in a type-safe manner.
  • commonmark: For parsing and rendering Markdown files.
    
    

• Frontend:

  • Bootstrap 5: Provides responsive design and prebuilt components like the navigation bar.
  • Highlight.js: Adds syntax highlighting for code snippets in blog posts.
    
    

• Web server:

  • Nginx: Used for serving the static files generated by the site generator.
    

How It Works:

• Static site generation:
  1. Build both parser and renderer. The parser will parse our markdown file into a Node object that will be used by the renderer to generate the posts HTML
  2. For each post:
     • Parse the markdown file using the parser.
     • Render the parsed content into HTML using the renderer.
     • Write the generated HTML to a file in the html/posts directory.
       
       

//#1
val parser = Parser.Builder().build()
val renderer = HtmlRenderer.builder().build()

//#2
for((_, title, path) <- postInfo) {
   val document = parser.parse(os.read(path))
   val content = renderer.render(document)
   val date = getDate(path.toString)

   os.write(
      os.pwd / "html" / "posts" / titleToFileName(title),
      genBase(
         genNav(RelPath("..")),
         div(
            `class` := "container-fluid"
         )(
            h1(a(href := "../index.html")),
            h4(s"Last updated: $date"),
            raw(content)
         )
      )
   )
}

• Shared:
  • The Shared object defines a protocol for sending and receiving data between the client and server
    
    

object Shared {
   def send[T: Writer](out: java.io.DataOutputStream, msg: T) = {
      val bytes = upickle.default.writeBinary(msg)
      out.writeInt(bytes.length)
      out.write(bytes)
      out.flush()
   }

   def receive[T: Reader](in: java.io.DataInputStream) = {
      val buf = new Array[Byte](in.readInt())
      in.readFully(buf)
      upickle.default.readBinary[T](buf)
   }
}

• Sync:
  1. Create a SubProcess that runs the agent on the server
  2. Send the password to the agent
  3. Create a local function for communicating with the agent via RPC calls
  4. Call the agent to purge the server's HTML directory
  5. For each post, copy its HTML file to the server
     
     

//#1
val agent = os.proc("ssh", remoteUser, "sudo", "-S", "./out.jar").spawn()

//#2
agent.stdin.writeLine(password)
agent.stdin.flush()

//#3
def callAgent[T: upickle.default.Reader](rpc: Rpc) = {
   Shared.send(agent.stdin.data, rpc)
   Shared.receive(agent.stdout.data)
}

//#4
callAgent[Purge](Purge(pathToServerHtml))

//#5
for(srcSubPth <- os.walk(os.pwd / "html")) {
   val subPath = srcSubPth.subRelativeTo(pathToLocalHtml)
   val destPath = pathToServerHtml / subPath

   if(os.isFile(srcSubPth)) callAgent[Copy](Copy(os.read.bytes(srcSubPth), destPath))
}

agent.destroy()

• Agent:
  1. Receive an RPC call from the client
  2. Match the RPC call and execute the corresponding action
     
     

while true do {
   try {
      //#1
      val rpc = Shared.receive[Rpc](input)

      //#2
      rpc match
         case Purge(path) => Shared.send(output, os.remove.all(pathToHtml / path))
         case Copy(src, path) => Shared.send(output, os.write.over(pathToHtml / path, src, createFolders = true))
      
   } catch {
      case e: java.io.EOFException => System.exit(0)
   }
}