Development + Operations

Software Process Engineering


Danilo Pianini — danilo.pianini@unibo.it


Compiled on: 2024-05-20 — printable version

back

Development

  • Analysis of a domain
  • Design of a solution
  • Implementation
  • Testing

Operations

  • IT infrastructure
  • Deployment
  • Maintenance

Silo mentality

silos

No silos

silos

DevOps culture

  • Increased collaboration

    • Dev and Ops should exchange information and work together
  • Shared responsibility

    • A team is responsible for the (sub) product for its whole lifetime
    • No handing over projects from devs to ops
  • Autonomous teams

    • Lightweight decision making process
  • Focus on the process, not just the product

    • Promote small, incremental changes
    • Automate as much as possible
    • Leverage the right tool for the job at hand

Why bother?

  1. Risk management

    • Reduce the probability of failure
    • Detect defects before hitting the market
    • Quickly react to problems
  2. Resource exploitation

    • Use human resources for human-y work
    • Reduce time to market
    • Embrace innovation
    • Exploit emerging technologies

DevOps

  1. Principles
  2. Practices
  3. Tools

Principles inspire practices

Practices require tools

DevOps principles

(not exhaustive)

  • Collaboration
  • Reproducibility
  • Automation
  • Incrementality
  • Robustness

DevOps practices

  • Workflow organization
  • Build automation
  • Self-testing code
  • Code quality control
  • Continuous Integration
  • Continuous Delivery
  • Continuous Deployment
  • Continuous Monitoring

A real-world test case

We applied DevOps (and microservice-ification) to an existing software project, measuring some metrics before and after the operation.

  • The detailed experience report has been presented at the 37th International Conference on Software Maintenance and Evolution (ICSME 2021)
<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;">
  <iframe allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen="allowfullscreen" loading="eager" referrerpolicy="strict-origin-when-cross-origin" src="https://www.youtube.com/embed/6qd6GG3XQXA?autoplay=0&controls=1&end=0&loop=0&mute=0&start=0" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%; border:0;" title="YouTube video"
  ></iframe>
</div>

The target

Maggioli S.p.A

  • Multinational, based in Italy
  • ~2000 collaborators at the time of writing
  • An internal IT/Operations team
  • Our target team was composed of five people

sisred (before intervention)

  • Client-server stand-alone application
  • Delphi front-end, Microsoft SQL Server backend
  • Editors (paid by Maggioli) are experts in law entering information on the client
  • The information is then exposed into a (paid access) portal with up-to-date legal information

Previous architecture

old-arch

Metric Prev. Now Change
Release frequency ($\frac{releases}{day}$) 0.071
Commit to release time (hours) 8 to 24
Commits per day ($\frac{commits}{day}$) 2
MTTR (hours) 36
Prod. env. setup (working hours) 16
Dev. env. setup (minutes) 120
Nightly downtime ($\frac{minutes}{night}$) 30
Support ticket frequency ($\frac{tickets}{month}$) 40
Issue resolution time (days) 4

Microservice-ified architecture

new-arch

CI Pipeline

pipeline

Metric Prev. Now Change
Release frequency ($\frac{releases}{day}$) 0.071
Commit to release time (hours) 8 to 24
Commits per day ($\frac{commits}{day}$) 2
MTTR (hours) 36
Prod. env. setup (working hours) 16
Dev. env. setup (minutes) 120
Nightly downtime ($\frac{minutes}{night}$) 30
Support ticket frequency ($\frac{tickets}{month}$) 40
Issue resolution time (days) 4
Metric Prev. Now Change
Release frequency ($\frac{releases}{day}$) 0.071 2.7 +3700%
Commit to release time (hours) 8 to 24
Commits per day ($\frac{commits}{day}$) 2
MTTR (hours) 36
Prod. env. setup (working hours) 16
Dev. env. setup (minutes) 120
Nightly downtime ($\frac{minutes}{night}$) 30
Support ticket frequency ($\frac{tickets}{month}$) 40
Issue resolution time (days) 4
Metric Prev. Now Change
Release frequency ($\frac{releases}{day}$) 0.071 2.7 +3700%
Commit to release time (hours) 8 to 24 0.19 ~ -98.5%
Commits per day ($\frac{commits}{day}$) 2
MTTR (hours) 36
Prod. env. setup (working hours) 16
Dev. env. setup (minutes) 120
Nightly downtime ($\frac{minutes}{night}$) 30
Support ticket frequency ($\frac{tickets}{month}$) 40
Issue resolution time (days) 4
Metric Prev. Now Change
Release frequency ($\frac{releases}{day}$) 0.071 2.7 +3700%
Commit to release time (hours) 8 to 24 0.19 ~ -98.5%
Commits per day ($\frac{commits}{day}$) 2 7.1 +255%
MTTR (hours) 36
Prod. env. setup (working hours) 16
Dev. env. setup (minutes) 120
Nightly downtime ($\frac{minutes}{night}$) 30
Support ticket frequency ($\frac{tickets}{month}$) 40
Issue resolution time (days) 4
Metric Prev. Now Change
Release frequency ($\frac{releases}{day}$) 0.071 2.7 +3700%
Commit to release time (hours) 8 to 24 0.19 ~ -98.5%
Commits per day ($\frac{commits}{day}$) 2 7.1 +255%
MTTR (hours) 36 0.5 -98.6%
Prod. env. setup (working hours) 16
Dev. env. setup (minutes) 120
Nightly downtime ($\frac{minutes}{night}$) 30
Support ticket frequency ($\frac{tickets}{month}$) 40
Issue resolution time (days) 4
Metric Prev. Now Change
Release frequency ($\frac{releases}{day}$) 0.071 2.7 +3700%
Commit to release time (hours) 8 to 24 0.19 ~ -98.5%
Commits per day ($\frac{commits}{day}$) 2 7.1 +255%
MTTR (hours) 36 0.5 -98.6%
Prod. env. setup (working hours) 16 0.35 -97.8%
Dev. env. setup (minutes) 120
Nightly downtime ($\frac{minutes}{night}$) 30
Support ticket frequency ($\frac{tickets}{month}$) 40
Issue resolution time (days) 4
Metric Prev. Now Change
Release frequency ($\frac{releases}{day}$) 0.071 2.7 +3700%
Commit to release time (hours) 8 to 24 0.19 ~ -98.5%
Commits per day ($\frac{commits}{day}$) 2 7.1 +255%
MTTR (hours) 36 0.5 -98.6%
Prod. env. setup (working hours) 16 0.35 -97.8%
Dev. env. setup (minutes) 120 9 -92.5%
Nightly downtime ($\frac{minutes}{night}$) 30
Support ticket frequency ($\frac{tickets}{month}$) 40
Issue resolution time (days) 4
Metric Prev. Now Change
Release frequency ($\frac{releases}{day}$) 0.071 2.7 +3700%
Commit to release time (hours) 8 to 24 0.19 ~ -98.5%
Commits per day ($\frac{commits}{day}$) 2 7.1 +255%
MTTR (hours) 36 0.5 -98.6%
Prod. env. setup (working hours) 16 0.35 -97.8%
Dev. env. setup (minutes) 120 9 -92.5%
Nightly downtime ($\frac{minutes}{night}$) 30 0 -100%
Support ticket frequency ($\frac{tickets}{month}$) 40
Issue resolution time (days) 4
Metric Prev. Now Change
Release frequency ($\frac{releases}{day}$) 0.071 2.7 +3700%
Commit to release time (hours) 8 to 24 0.19 ~ -98.5%
Commits per day ($\frac{commits}{day}$) 2 7.1 +255%
MTTR (hours) 36 0.5 -98.6%
Prod. env. setup (working hours) 16 0.35 -97.8%
Dev. env. setup (minutes) 120 9 -92.5%
Nightly downtime ($\frac{minutes}{night}$) 30 0 -100%
Support ticket frequency ($\frac{tickets}{month}$) 40 19 -52.5%
Issue resolution time (days) 4
Metric Prev. Now Change
Release frequency ($\frac{releases}{day}$) 0.071 2.7 +3700%
Commit to release time (hours) 8 to 24 0.19 ~ -98.5%
Commits per day ($\frac{commits}{day}$) 2 7.1 +255%
MTTR (hours) 36 0.5 -98.6%
Prod. env. setup (working hours) 16 0.35 -97.8%
Dev. env. setup (minutes) 120 9 -92.5%
Nightly downtime ($\frac{minutes}{night}$) 30 0 -100%
Support ticket frequency ($\frac{tickets}{month}$) 40 19 -52.5%
Issue resolution time (days) 4 3 -25%

Benefits

  • Much less maintenance in the traditional meaning: “Time spent to keep the system in nominal conditions”
    • No more issues with Windows updates
    • No more downtimes related to internal network / electricity / public infrastructure maintenance
    • Improved security
    • No more critical failures caused by testing stored procedures directly in production by mistake
  • Much more maintenance in terms of software evolution
    • Application (or verification of automatic application) of updates
    • Security audits
    • Maintenance and update of the pipeline

Lessons learned

  • The teams must be autonomous
  • Practices must be tailored to the team
  • Time-consuming, repetitive, and cumbersome procedures must be automated
  • Obsolete practices must be removed
  • Communication is key, awareness must be shed across the team of the expected benefits

Timeline

timeline

Development + Operations

Software Process Engineering


Danilo Pianini — danilo.pianini@unibo.it


Compiled on: 2024-05-20 — printable version

back