Équipe Commando

The Commando Team Icon
The %-CT% in L(i)VID

The %-CT% is a team that is unique to a Release Train. It provides assistance to all the other teams of the Release Train. Therefore, the %-CT% operates at the Integration Layer as illustrated below:

Place of the %-CT% at the %-IL%
Place of the %-CT% at the %-IL%

Mission of the %-CT%

Here are perfect examples of what a %-CT% can do for the other teams:

Not the Mission of the %-CT%

No testing activity is the responsibility of the %-CT%. Such activities are the responsibility of the %TT%. That should not be confused though with the automation of testing!

Customer Oriented

The %COMMANDO% must be Customer Oriented in the sense that they should NOT impose a way of working to the teams, but should listen to them instead to really determine the %JTBD%. Only in this way can it find the solution that will fit the problem at stake, and get %"l%customer%"r% engagement (will to work) so that it can collect an easy feedback on the solution they found.

It is also their work to integrate each individual solution into a single Pipeline de déploiement for the entirety of the train. In the case the team they help does not know exactly what it wants, the representative of the %-CT% helps them asking the right questions. In general it boils down to remove some frictions thanks to the automation paradigm.

Composition

The team is composed of about the same number of individuals as there are teams in the train (e.g if a Release Train is composed of 7 teams, then the %-CT% is better sized with 7 individuals, the rationale being that each member of the %-CT% can follow a team and make sure its needs are taken into consideration by being immerged in it.

Once most of the automation work is done (une fois que les choses tournent), the team can shrink dramatically, which is recommended by L(i)VID: the knowledge will have been absorbed by the %DEVOPS% Squads and therefore the %-CT% should not represent more than 2-4%PERC% of the train. People can therefore start helping other trains.

In no event should this team be dismantled as there will always be a need to reflect on how the deployment can be enhanced (think of it as continuous improvement, %KAI%, or as %INNO% where you can use the Lean Startup techniques). If you dismantle the team, you will lock yourself in an existing process.

One Commando Team Member per DevOps Squad
One Commando Team Member per DevOps Squad

Industrialization or Not?

Our recommendation is to avoid industrialization as much as possible, as long as what is meant by that is strong standardization. This would indeed create a bunch of headaches for all the teams that will now have to follow the standard process without being able to enhance it based on their own needs. As stated before, this is locking people in a process that is known to be quickly evolving at present time (see the evolution of Puppet, Chef, Jenkins, Bamboo, Nolio, Docker …). Of course, this should not prevent the organization to work with %CoP%s to help teams converging to the same sort of principles an methods.

Another danger we have sometimes see companies bumped into is the underlying technologies that the automation of the Pipeline de déploiement must support: you don't automate the Pipeline de déploiement for Java technologies as you automate them for COBOL teams. Going too fast with the industrialization will make any transition more difficult.

But institutionalization, when carried too far, can create its own danger. Passing down words of wisdom from the past or estahlishing standard practices hased on success stories works well when the external environment is stable. Changes in the environment, however, can quickly make such lessons impractical — Hirotaka Takeuchi and Ikujiro Nonaka in The new new product development game

At the moment you see that all this becomes stable (there is no longer any breakthrough improvement in it), then you may want to standardize the process, not before! Our message is this: don't go jump head first in that. Take the necessary time.

Organizational Mode

The team is organized pretty much as any other %DEVOPS% Squad, except that it has no Product Owner. The %-COMMANDO% works at the train level (Integration Layer): not in a Infrastructure Department, which by the way is a nice opportunit for us to remind you that Infrastructure is part of Architecture: the %-COMMANDO% works for the train (actually a set of teams)!

%-IPM%

The %-CT% holds an %IPM% which is facilitated by an Scrum Master. All items that this team must work on come from the %RTPS% and from the needs expressed by the teams the %-CT% is immerged to and which compose the Release Train. Some consensus need to be found between all the members of the team for anything that may appear conflicting and for the prioritization. Like any other team, all items are estimated by whatever estimation game has been foreseen.

Demo

Most of the time, there is no demo in a %-CT% for obvious reasons. However, in the case of coordination between trains, or when big improvements have been reached, it may be interesting to convey %-CT%s of other trains to see what has been accomplished in a given train. This may be completed by the recourse to %CoP%. In any case, such ceremonies are mostly organized for the sake of Knowledge Management and/or easy reporting.

Retrospective

The %-CT% of a train needs, like any other team, to reflect on how it performed during its last sprint (once again, we insist on the fact that the only level where sprinting is mandatory for SAMBA is the %-IL%). The %sprint% of the %-CT% is therefore the cadence set at the level of the Release Train.

%-SD%

No item taken in charge by the %-CT% gets demonstrated in the %SD% (at the Integration Layer). If there is a real need for such a demo, then it is organized at the level of the %-CT% (see Demo)

%-SoS%

The team sends an ambassador to the %SoS% as to make it possible to coordinate the efforts and share the necessary information.

Their Own Taskboard

The case of the %-COMMANDO% is a bit different from the case of %SR%. Here we have a real team, but the point is that the whole team must satisfy different teams and often each rep of the %-COMMANDO% is immerged in the team it needs to help. That person will have tasks for his/her own team — the %-COMMANDO% — but will also have tasks for the team he/she helps, to which he/she is the humble servant.

This is the reason why we recommend that each member of the %-COMMANDO% team, has his/her own kanban (maybe on top of using a global taskboard for the whole team).

To be continued

ICI AJOUTER ILLU KANBAN OU LES LIGNES SONT DECOUPEES PAR PERSONNE ET OU IL IMPORTE PLUS DE DELIVRER AU FUR ET A MESURE QUE DE DELIVRER DES FEATURES ET/OU STORIES ENTIERES. SOUVENT D'AILLEURS, CETTE EQUIPE LIVRE DIRECTEMENT ET TESTE DANS LE CADRE DES AUTRES EQUIPES (CAD DANS LEUR ECOSYSTEME).

A Commando Team Taskboard Example

The example that follows illustrates some problems of Agility but also reveals some problems appearing in the life of a normal %-CT%:

%-CT% Taskboard
A typical example of a %-CT% Taskboard that reveals several problems, some being usual in such teams

What the picture reveals is that there are way too many tasks that either come categorized as Expedite (urgent on the board, red circle at the top), or as Unplanned compared to the quantity of tasks in the blue circle and the ones circled in green. This is most probably due to the fact that the tasks do not come from the %DEVOPS% Squads that must be served and maybe because a central commandment has been instituted that is sending requests as water flow. On top of that, there are some low-priority tasks (bottom of the taskboard) sorted out already whereas some more urgent requests remain untouched.

Visualize the %-CT%

HERE PICTURE OF THE DEMO, PLANNING, SoS, RETRO, etc.

Documentation

The good principles of %-KM% must be applied: What the %-COMMANDO% produces must be available to all parties in a searchable way: %-KM%. Whether that is materialized as a website, a wiki, a set of .pdf, etc. is up to the organization.

A Real Story of Automation

I (Pat here) have come across many opportunities to create automatic builds but the first time I have seen this principle in its fullest potential that was in 2009 when I had the luck to lead about 20 persons in a "Yellow Pages" project in a Web Agency.

We had a team of testers of the client residing in a room close to the development plateau: these guys needed to get new things to test every day.

We have tried to create the builds and to deploy our artefacts manually (build and promotion to the test environment) but it was simply not sustainable on top of being extremely boring for the guy that was unlucky enough to be picked up for the task. Too much work, too many errors, too deadly dull!

Finally one of our chaps proposed to automate the whole process: Maxence Dewil who chose Bamboo to do the job. It took him a week to come up with a first workable solution. From this very moment we haven't stopped improving over the next sprints.

First Things First: the Bare Minimal %DoD%

The build was completely automated and to be successful we needed to match the following %DoD%:

  1. The code had to compile with no error and no warning: only working code!
  2. Each time a new build was assembled we had to find new Release Notes entries (applicable for EVERY single source, if updated)
  3. Each function or method of the whole set of sources had to have at least 1 Unit Test
  4. All Unit Tests needed to be successful

If one of these conditions was not met, the build was simply aborted and the next operation in line, deployment, would simply be cancelled.

%MORE%

EVERYTHING DOWN THIS LINE IS UNFINISHED MATERIAL

Automated Builds: the L(i)VID Way

The truth is in the code (broad sense of the term: unit tests, analysis, …); Developers create the code. The code is put in the version control tool (code repository). The Automatic Build tool extracts everything from the code repository; it generates the executable, regenerates the whole set of doc, checks the existence of the unit tests and executes them automatically; it produces reports, release notes and a lot more. It assigns an automatic build number. Last but not least, the Automatic Build tool packages the whole thing … which is now ready to be deployed in the next logical environment in the whole chain (e.g. the Integration Testing environment) … but which may require GO/No GO decision.

Nowadays, Automatic Build Tools are numerous [1] and with today's machine power it takes not more than few minutes to complete, even if you also include calls to Performance Testing in your build scripts, which is what we do for our own Web Framework by the way.

Speed at which a build completes changes the whole end-to-end cycle can be played with several orders of magnitude. At Lato Sensu for example, a complete build, deploy, test sequence takes as few as 5 minutes (to the Integration stage and Integration Testng is automated, at least up to some degree). When we're satisfied with the result (either automatically or manually), it takes about 1 to 2 minutes to promote the whole thing to Production (deployment is incremental; if wrong … we favor the Rebuild instead of Repair approach). This is what we mean with a different ballgame: we can repeat this process several times a day if needed whereas we have seen organizations where it can be performed once every month, or even worse, once every quarter.

How To Get There?

Dedicate One Member of Your DevOps Team

One possible way of doing it is to dedicate one guy of your fellowship to the job. This is what we have done with Maxence @Emakina.

Maxence chose Bamboo and came to a very good result in about one week of work. It was not perfect from day one, but we kept improving the concept from this point on. The week of work that was dedicated to the task was worth investing: we get it back ten fold!

Start A Project

Another way of doing it is to embrace the issue for an entire department (or even company) as we have done it @Vae Soli!, though you must be aware of the fact that the build may be very different amongst teams depending on the technology they use (what is a build in PHP, in Java, in C, in Perl, in Python, in ...COBOL).

It all started as a separate project whose main goal was to automate the documentation of our framework: GUIDE. The master principle that we have followed is that the truth of our framework is ALWAYS in the code and we wanted to have only ONE source of truth (a difficult concept to grasp if you always worked with strong work segregation such as architecture, functional analysis, technical analysis, development, testing, service maintenance %-% ops, …)

It took us two days to start having a good documentation but the whole project was completed in about 2 months. The core of the work was not really to create the tool of Automatic Documentation but more in the inclusion of the numerous comment tags in our code.

Comment Tags were really the added value because it enabled many different possibilities to document our framework.

Not only we have produced a professional Reference Guide, but also examples, high-level samples, designs, illustrations, release notes, code guidelines (and code guidelines verification), …

Then when Automated Documentation was over we have included it in our Build mechanism (BUILD), which is no more than a sequence of steps:

  1. Extract documentation from code (GUIDE)
  2. Extract release notes from code (GUIDE)
  3. Assemble high level abstracts from code (GUIDE)
  4. Extract examples from code and assemble them in comprehensive separate documents (GUIDE)
  5. Extract assertions and other executed examples (GUIDE)
  6. Call Performance Testing (PERF)
  7. Produce Performance Testing report (PERF)
  8. Grab Performance Testing results (BUILD)
  9. Abort BUILD if outside acceptable boundaries (BUILD)
  10. Check delta compared to previous version (BUILD)
  11. Produce final Release Notes (BUILD)
  12. Produce Sanity Check scripts (BUILD)
  13. Produce incremental version + full version (BUILD)
  14. Produce Build report (BUILD)

At this point, there is a potential caesura: either we continue the process via Automated Deployment or we simply stop here and start the deployment ourselves so that we can start the Integration Testing phase. Whatever is more convenient for you, we have here sketched what the Build process was.

Delegate the Work To a %-COMMANDO%

As already mentioned, the %-CT% exists to help the teams of the train to automate a bunch of things.

What better areas than Continuous Integration, Automated Build, Automated Deployment and the associated tooling? What is better than the Pipeline de déploiement?

Well, this is what we have in mind by delegating the work to a Commando Team, a team that can gather the requests from many different teams and start working on a global solution, step by step, at the pace of the project/program.

No Fishbone Versioning

One of the drawbacks with Tree Versioning, aka branching, is that it is impossible to sustain the discipline of getting the branch back on the trunk quickly.

Maybe it will be still feasible when integration is performed twice a sprint, but this will no longer be the case when dealing with several increments per day. If you're tempted to stll apply the typical branching model, then you put Continuous Integration at risk and you give the Integration Level a false sense of velocity.

Agile methods call for work to be entertained directly on the trunk, which by the way is also ideal when you start real DevOps that will need to fix things for production at the same time they sprint to incorporate new changes [X] ICI AUSSI EXPLIQUER LE CHALLENGE DES SERVICES TOGGLED ON.

ICI ... METTRE LE GRAPHIQUE FISHBONE VERSIONING

Forming a %-CoP%

One of the biggest bang for your buck comes from the Automation paradigm. This is particularly true for the streamlining of the Pipeline de déploiement as we said already multiple times. In that true sense, it is crucial to start sharing experience and practices through a %CoP%:

Setting Up a %-CoP% for %-COMMANDO%s
Setting Up a %-CoP% for %-COMMANDO%s

Footnotes

[1] … Still, for our own Web Framework @Lato Sensu Management, we have built our own tools, which was a project of about 3 months: BUILD and GUIDE

Notes de bas de page

[1] … Still, for our own Web Framework @Lato Sensu Management, we have built our own tools, which was a project of about 3 months: BUILD and GUIDE

Last update: 15-04-2021 06:36
0'00"