FOSS Tools for Scrum Project Delivery

I thought I’d share a few tools our team uses for managing (control, visibility, reporting) our software development projects.

Following lean principles, we opted for the simplest possible tool set that will get the job done. These are just:

  • Trello.com for stories, backlogs, sprints and bug/issue management
  • Slack.com for online collaboration between team members
  • Zoom.US for face-to-face comms with the team (as my teams are often in other countries)

The role each product plays

Slack.com

One of the best team collaboration tools with instant messaging (team and private), the notion of ‘channels’, file sharing, search and notifications. Use this to bring all the team online ‘conversations’ and asset sharing under one searchable roof… Use it on your desktop, laptop, mobile devices or all of them! Ditch email communications for this tool right now!

Its Slack’s integration capabilities that raise it above other similar tools. Slack can integrate with a load of useful 3rd party software including Trello, Google Drive, GitHub, Zapier, Mailchimp, Zendesk, Stripe, Twitter, Zoom and many more. You can also build your own integrations or embrace their web hooks to ‘couple’ and trigger events on other systems. Slack also offers a “free to use forever” pricing tier that may be all you need. There’s some great paid services to improve Slack’s implementation of SCRUM (Scrumbot and Scrum Mate) but we’ll not go into these. They are worth checking out though.

Trello.com

Trello is a free (paid version available), easy to customise, online collaboration tool that organizes projects into boards. With Trello, it is easy to see what’s being worked on, who’s working on it, and at what stage something is ‘at’. Trello is therefore a great choice for Agile project management – especially SCRUM.

As with anything, start with the minimum you need and no more… It is far better to add lists to a board than create a ton up-front and then feel obliged to fill them.

For me, Trello works best when you can operate within a single board and if you don’t have too many lists… I don’t mind a little horizontal scrolling within reason… I personally like to keep the content within Trello extremely minimal and focused on specific issues or outcomes. I therefore prefer to engage in all conversations through Slack. For instance, a discussion around a bug would happen in Slack, a specific action to resolve a bug would appear in Trello (after the conversation has helped to determine what that is).

The impact of a card is shown by a simple RAG label.

Everyone has their own way of configuring Trello. Here’s mine:

I’ve have Boards for each project (e.g. Customer Smartphone Booking App). Each board is arranged to facilitate a simple implementation of the SCRUM framework. I then arrange my lists according to the priority of how I want to see them. I place the boards I should frequently review to the left, ‘productivity’ boards are placed centrally and ‘assets’ or ‘future’ aspirations (e.g. Product Backlog) are placed to the right.

Scrum Trello Board
Scrum Trello Board

I’ve named my lists and ordered them as below:

  1. Live Bugs (affecting the product that is currently in live use)
    I want to know these ASAP and for them to be the first item I see when I log in.
  2. Issues (affecting either the product in live use OR the developing product / version)
    Although potentially not as immediate as a bug in the live environment, they most likely do (or will) have an impact on operations and need attention sooner than later.
  3. Impediments / Blockers
    Any external factors or blockers impeding the progression of the Sprint are listed here.
  4. Sprint Backlog (date)
    Here we list the tasks for the current Sprint. At this point the tasks should have estimated story points assigned to them and their quality goals (both functional and non-functional) should be defined. During the Sprint the team each select tasks and drag them into the ‘In Progress’ list.
  5. In Progress
    The tasks remain in this list whilst the teams work on them. Progress is also reported here.
  6. QA
    When the developers are content that the task ‘feature’ or ‘fix’ is completed and operating as per the functional specification they move it to the QA list for a different member of the team to test it meets all quality goals.
  7. Done
    Only when the task is ‘signed off’ through QA (i.e. the feature or fix is considered operationally ready) will a task be moved to the ‘Done’ list.
  8. Sprint Retrospective
    The good the bad and the ugly goes here.
  9. Product Backlog
    Here we list the User Stories (in priority order) that we have created to capture a description of a software feature from an end-user perspective. This is continually modified and re-prioritised according to changing requirements.
  10. Assets

A place to keep essential assets (e.g. documents, images, videos etc.) relating to the project. They are kept here for easy access by the team.

I’ve found, that by using the above approach, the scope of the project and the current situation is easy to ascertain without having to reference multiple tools. Add a decent reporting tool like the excellent ‘Plus for Trello’ which has a bunch of productivity and reporting capabilities.

Framework Vs. Methodology

It’s a common question that came up again today. I thought I’d articulate it as succinctly as possible…

A methodology is a set of principles, tools and practices which can be used to guide processes to achieve a particular goal.

A framework is a loose but incomplete structure which leaves room for other practices and tools to be included but provides much of the process required.

See why Ken Schwaber refers to Scrum as a Framework. He explains:

The word “framework” means that much is not specified and must be devised by those using the framework. I equate Scrum to the game of chess. You can read the official rulebook for chess. The moves, players, sequences, scoring, etc. are all specified. Learn them. Then you can play chess. Maybe your chess game isn’t so good, but you can study great games, strategies, and tactics, and practice to get better and better. However, you are playing the game of chess, so you don’t have the option of changing the rule book. If you change the rules, it’s not chess any longer. Just learn how to play the game with excellence, which is enough of a challenge.

On the other hand Extreme Programming (XP) could be considered a methodology as it is more prescriptive in its approach and its environment. It promotes a set of rules that should be followed.

What is DevOps really? Why should I care?

History

The explosion of Agile for software development was pretty cool right? Empowered development teams could now liaise directly with the ‘customer’, make decisions and ‘crack on’ with exciting new software deliveries by taking an iterative approach to deliver value rapidly and often! But, hang on, this isn’t an ‘Agile’ post… this is DevOps… so what gives?

Well, somehow this exciting new approach didn’t happen within the operations space’. So whilst development was agile and continuous, operations continued to utilise a more ‘weighty’ Waterfall style approach.

I remember in a previous life, as a Project Manager (locked in ‘Waterfall’), I’d engage with Subject Matter Experts across the business and Business Analysts to ensure user and business requirements were understood, and existing business processes were ‘mapped’. Once this exercise was complete I would be able to mediate effectively between all stakeholders to get the job done… That alone was ‘chewy’ with lots of meetings, lots of paperwork, demos, risk management exercises, sign-offs and reports however none of this compared to the lengthy processes surrounding transition to the live production environment and into support and maintenance… i.e. the Operations domain.

Enter DevOps!

As the word describes, DevOps is a coming together of two key business functions: Development and Operations for a more efficient implementation of software and systems.

I love the Infographic below from App Dynamics (see original source).

DevOps Engineer

A successful IT product or service should meet the needs of the users, yes, but it must also be efficient, serviceable and cost effective to deploy and maintain. It is unlikely to be all of these unless it is architected in such a way to work well with target IT platforms, processes, operational activities and so on. Additionally there is little point in having short ‘release’ cycles if the actual deployment process takes weeks (perhaps months). DevOps promotes a Dev and Ops inclusive approach to ensure teams, skills, processes are aligned across development and into operations – so they may work in symphony to deliver faster and more efficiently without compromise.

DevOps Intersections

See original source, Wikipedia

What does DevOps actually look like?

A fuzzy answer would be ‘A shared design, development and delivery experience between IT operations and development teams across the SDLC’. If you try to be more specific you’re probably going to put some people’s noses out of joint but there is general agreement that IT automation, agile development, systems virtualisation, collaborative working and improved processes are all important. The TechInsights Report: “What Smart Businesses Know About DevOps” lists the following:

Important Components of DevOps

How does DevOps look in practice? Let’s consider a couple of scenarios…

Firstly, let’s consider the ‘culture’ side of DevOps. If, like ours, your organisation is reasonably small and runs a ‘tight ship’ (i.e. lean on staff) then you’ll probably already be multi-disciplined and your departments may already liaise on a regular basis (e.g. through management meetings). Like many small businesses we outsource a lot but we retain control from within. I represent the IT function but I also approve and commission new development. I’m continually meeting with Operations, Sales and Accounts to ensure our goals are aligned and that we are delivering for business benefit and capability. I’m lucky enough (because I love it) to be at the heart of any system and software architecture decision. My background is in software development so I remain ‘hands on’ in the creation of design documentation and in the choice of technologies, development practices and processes. When we transition to the production environment and undertake maintenance I get to influence which tools are used and how we deploy and support. I’m not autocratic in my decisions I’m inclusive and fully engage with key functions within the organisation to ensure that their needs are met and they can utilise and benefit from the new software or systems. Like I said, we’re small, so our deployment is reasonably straightforward – we don’t need specific deployment tools, our test/staging/live environments are well understood and do us just fine. So, culturally I would suggest we already kind of do DevOps! Does that me me a DevOps engineer? I think not… We do have processes and controls around source code management and deployment but we’re not really automating to any great extent and the process is still pretty hands-on. In reality we’re not undertaking many of the typical activities you’ll find DevOps ‘Engineers’ claim are essential to carry the accolade. So you won’t find me applying for DevOps Engineer roles just yet…

In another scenario a larger business may specifically recruit DevOps engineers to ‘own’ the entire Dev to Ops process. These engineers may be responsible for influencing architectural decisions and defining processes that improve efficiency across the SDLC and into production. They do this through tools and scripting that ensure code quality and integrity and that automate the deployment process. Improvements in code integrity can be achieved through unit test – perhaps a Test-driven development approach and the benefits of automation are many. If deployments are scripted, they are 100% repeatable, eliminating human error and enabling the tasks to be run by lesser skilled individuals – so that’s faster, with less issues, at a lower cost to the company (you can understand why organisations are embracing DevOps). Done correctly, DevOps should mean the unexpected no longer happens, new deployments are rapid and, in the virtual space, new ‘clone’ environments can be initiated at speeds that no one could dream of in the physical space. Of course, to get to this stage is no small undertaking. As well as costly DevOps engineers, organisations would have invested money and time in tools (see list below, original source here) and environments that enable engineers to implement virtualisation, code version control, continuous integration and deployment automation tools. Generally these organisations would have also adopted lean and agile development principles and undertaken the culture, processes and practices that accompany the approach.

So efficient DevOps can help to ensure our deployment ‘keeps up’ with our short release cycles.

DevOps Tools

So you want to be a DevOps Engineer?

I don’t blame you. I see some serious career progression for these guys and gals and with a broad experience across multiple technologies the DevOps engineers of today are likely to be the CTOs of tomorrow. So what skills should you have or be looking to obtain?

Firstly, it’s not reasonably practical to assume you can complete a weekend training course to become a DevOps engineer. You have to earn that title with experience that comes from hard graft in a tough career, in a tough industry.

Secondly, you’ll need a mix of soft and hard skills:

You need to be a strong communicator both with techies and ‘normal’ folk. You should know when to talk tech and when to not. You should be the kind of person that ‘shares’ knowledge and enjoys collaboration (I’ll be blogging about some of my favourite collaboration tools including Trello and Slack in another blog). You should be curious, analytical and have a flexible mindset. You should always challenge the status quo and look for a better way…

From a technology perspective you should be familiar with modern and open source technologies within the development and IT operations arena. You should have some hands-on experience with current programming/development practices, popular and emerging technologies, client-side vs service-side, coding frameworks, software libraries, continuous integration/deployment, test-driven development, API development and utilisation and so on. Solid systems experience (you’d know your OSI from your TCP/IP reference models) including how stuff works (like data packets, routing etc.) is pretty fundamental. Systems admin experience is vital – some *NIX (e.g UNIX, Linux) would do you proud. IT security is an essential requirement (and also important in software design) as well as monitoring, reporting. You should have experience of Solution design and architecture and know the value of Data and data management. To be future proof you should be familiar with virtualisation and cloud computing (Azure, AWS, private cloud) and ideally have hands-on experience of SaaS and IaaS. Lastly, experience of IT configuration management and automation and and some of the emerging DevOps tools like Puppet and Chef is going to help you claim that title for sure!

Now, before we continue, read the paragraph above again… That’s a pretty hefty set of skills right? It is easy to see how DevOps evolved from Lean Start-up requirements where funding is tight and staff must undertake multiple roles in order to meet ambitious release cycles. There’s an alternative view to DevOps – taken by the developer. I’d encourage you to read this excellent article by Jeff Knupp who basically argues that just because developers have had to undertake the roles of DBA, QA, Test, Deployment Engineer etc. (because of constraints) and just because they don’t suck at it (many are actually pretty good at it) it shouldn’t mean this should become expected of them. Such multi-tasking may actually detract from the role of core development and talented employees can become overworked and under productive…

You can’t know everything and specialising is just fine but you should have enough knowledge in a chosen field to have an opinion on what to use, where and when to use it and how to do so. As with any role, demonstrable experience and exposure to a wide variety of situations will improve your desirability as a DevOps engineer.

Summary

So DevOps seeks to align and harmonize the development and operations functions so they can work together in delivering systems/solutions rapidly, reliably and safely. As a result the transition into production will be more agile and the products can be more efficiently operated and supported. DevOps combines development, systems and IT services and operations experience and this combined set of skills helps to meet the requirements and challenges of development, infrastructure, automation, testing, release and maintenance.

DevOps is a way of thinking and working… a culture of sorts. It involves people, process and technology. There are a number of skills and best practices associated with DevOps and an increasing array of tools (with some emerging leaders) to help its successful implementation. A dedicated individual can become a DevOps Ninja Warrior but don’t think it’s a walk in the park – there’s some serious knowledge required across a broad array of technologies.

The Infographic below summarises DevOps fantastically (see original source).

4 steps to DevOps success