Software Process Engineering

Development and operations

Danilo Pianini

Development

• Analysis of a domain
• Design of a solution
• Implementation
• Testing

Operations

• IT infrastructure
• Deployment
• Maintenance

Silo mentality

No silos

DevOps culture

• Increased collaboration

  • Dev and Ops should exchange information and work together

• Shared responsibility

  • A team is responsible for the (sub) product for its whole lifetime
  • No handing over projects from devs to ops

• Autonomous teams

  • Lightweight decision making process

• Focus on the process, not just the product

  • Promote small, incremental changes
  • Automate as much as possible
  • Leverage the right tool for the job at hand

Why bother?

1. Risk management

   • Reduce the probability of failure
   • Detect defects before hitting the market
   • Quickly react to problems

2. Resource exploitation

   • Use human resources for human-y work
   • Reduce time to market
   • Embrace innovation
   • Exploit emerging technologies

DevOps

1. Principles
2. Practices
3. Tools

Principles inspire practices

Practices require tools

DevOps principles

(not exhaustive)

• Collaboration
• Reproducibility
• Automation
• Incrementality
• Robustness

DevOps practices

• Workflow organization
• Build automation
• Self-testing code
• Code quality control
• Continuous Integration
• Continuous Delivery
• Continuous Deployment
• Continuous Monitoring

DevOps practices

Workflow organization

Lightweight processes and shared responsibility

• Participants should have clear roles
• Lifecycle phases should be mapped into the workflow

Tools

• Version control systems
  • Reproducible project history
  • Enable several workflows
• Workflow management platforms
  • Tracking of progress, issues, proposals
  • Activities are linked to code

Gitflow

Forks and pull requests

DevOps practices

Self-testing code

Code includes tests

• Several testing levels
  • Unit, acceptance, integration, etc.
• Testing is entirely automated
• Solving an issue implies the creation of a regression test
• Tests are executed for every change

Tools

• Build automators
  • Compile, test, and run QA on software

DevOps practices

Code quality control

Code is coherent and understandable

• Consistent code style across the board
  • Improved readability
  • Much cleaner diffs
  • Improved bisection / regression tracking
• Extremely clean code reduces the need for documentation
  • Newbies can be productive more quickly
• A large part of code QA can be automated!

Tools

• Build automators
• Static analyzers
  • Verify metrics and rules without execution

DevOps practices

Continuous integration

Code must not diverge

• Working copies are kept in sync with the mainline
  • How its done depends on the workflow organization
• Build and test automatically on a fresh machine
• Build and test on all the target platforms
• Promptly intercept (feedback) failures and issues
• Deploy successful artifacts

Tools

• Continuous Integration platforms
  • Run the automation pipeline on reference environments
  • Promotes infrastructure as code
    • (at least for the build/test/check phase)

Continuous integration

Risk reduction

$\Rightarrow$

DevOps practices

Continuous Delivery

Every working build should produce a potential release

• The CI pipeline should produce final artifact
• Artifacts should be available for quick deployment

Tools

• Build automators
• Continuous Integration platforms

DevOps practices

Continuous Deployment

Actual deployment should be automatic

• There should be a strategy for delivered code to enter production
  • Reversibly (e.g. blue green deployment)
  • Gradually (e.g. canary release)
• Infrastructure should be defined by software
• Deliverability before new features

Tools

• Continuous Integration platforms
• Configuration management software
  • Enables Continuous configuration automation
  • Defines infrastructure via software with zero human interaction

DevOps at work

An internal project where techniques are applied

• “Truth” Repository
• Branches
• Forks
• Example pull request
• Example automatic update
• CI/CD pipeline
• (Build) Infrastructure as code

These slides are in CI/CD as well!

• This is the latest build

DevOps for DIR

Version control

• Basics of distributed version control
  • including SVN “reeducation”
  • Repository creation and local maintenance
  • Basics of collaboration (branching, merging)
  • Multi-repo (forks, review, pull requests)
• Agile workflows depending on specific needs
• Advanced distributed version control
  • History alteration (Rebasing / Squashing)
  • Bisection
  • Cherry picking
  • DVCS sub-modularization
    • (used in these slides)

DevOps for DIR

Automation

• Software lifecycle organization and build automation
  • From source to tested and deployable in one command
  • Gradle is the house speciality
    • Basics
    • Build organization
    • Customization and advanced
    • Custom plugin development
• Advanced automation
  • Requires working CI and comprehensive testing or it’s going to burn
  • Automated software updates
  • Automated build infrastructure updates

DevOps for DIR

Continuous Integration / Deployment / Delivery

• Adding CI to existing projects
  • Needs a committment to the DevOps philosophy
• Multiplatform testing
  • OS type and version
  • Platform (JDK, CLR, Python interpreter) version
  • Multi-stage
• Continuous deployment
  • “One click away from production”
• Continuous delivery
  • From code to production automagically