project

---

HPE OO Content Promotion Manual process facts

2016

Contents -

Process Overview (components and actions)

-

What an Infrastructure Administrator needs to do

-

What an HPE OO Author needs to do

-

What an HPE OO Promoter needs to do

-

What an HPE OO Administrator needs to do

2

Process Overview

Note: The process does not catch actions related to the infrastructure setup.

3

The Infrastructure Administrator

Installing and configuring the multi-authoring environment

4

As an Infrastructure Administrator

Install the SCM system

– There are many SCM servers to choose from for both SVN and Git. – SVN: VisualSVN, SubversionEdge – Git: GitHub, GitLab, Bitbucket, GitBlit among others

– We have chosen to work with Git and in particular with GitHub Enterprise.

Note: All the Git related information from within this document is taken from GitHub Enterprise. Do note that several features described herein might be bound to GitHub only.

5

As an infrastructure administrator

Configure external authentication (LDAP, SAML or others)

– Whether Git or SVN, it is better to configure external authentication for a single and centralized user management and increased security.

Note: In GitHub Enterprise usernames are normalized according to GitHub rules. More here.

6

As an Infrastructure Administrator

Create SCM Repositories to held HPE OO Projects

– Create Git repositories for the relevant projects that the authors will work on. – We recommend the following convention: oo[organization]-[department]-project – oo – standard prefix. – organization and department should be seen as optional, however we recommend their usage as much as possible. – project refers to a domain or use-case and could be a dash based separated value.

Note: The convention above will be translated from the Git repository to the OO project and eventually to the OO content pack - the artifact being stored into the artifact repository as explained later within the document.

7

As an Infrastructure Administrator

Create SCM repositories to held HPE OO projects – naming convention facts – Let's look at the picture in the right – oo = standard prefix – acme = organization – devops = department – security = department – fortify/commons/jenkins/docker = project

– When department is used one can easily separate content packs from two departments within same organization (oo-devops-fortify vs oo-security-fortify) – When organization is used, one can easily expose content packs outside of the organization itself and keep a reference to the organization name (oo-acme-devops-docker as example)

Note: Optional values like organization and department should be used whenever possible.

8

As an Infrastructure Administrator

Use 1:2 Repository to Project relation at most

– Place one HPE OO project into a Git repository that relates to the actual implementation flows and an additional project that relates to flows testing the implementation ones. – You will end up with at most 2 projects per repository. – Controlling user accessibility to particular HPE OO projects will be easier. – It will act as a basis for being able to manage multiple teams working on multiple (often inter-connected) projects while having particular user based restrictions.

Note: Remember that Git access control is per repository while SVN access control is per project. We do think that 1:1 repository to content pack relation is the best thing to do when using Git.

9

As an Infrastructure Administrator

Group HPE OO authors into relevant teams – Create GitHub teams and place members according to the needs. – Think about role base access control when splitting your teams – Department split (devops, security) – Your organization is structured so that different departments will build content

– Content split (acme-commons) – There is particular content that acts as foundation for other contents and to which more departments can contribute

– Security and Role split (contributors, maintainers, administrators, promoters) – Only a particular group of people can get access to sensitive data – A group of people required to perform either particular or common actions on repositories

10

As an Infrastructure Administrator

Grant access for either teams or users

– Grant access to teams: – You have multiple users within a team and all users need the same access level. – Different teams require different access level

– Grant access to users: – You have multiple users within a team and one user does not need write access to a particular project. For this particular case it is better to not include the user in the GitHub team and rather define

11

As an Infrastructure Administrator

Make sure there is adequate SCM notification

– Enable the email service per each repository.

Note: In the address field it is better to use group based email addresses instead of individual email addresses.

12

As an Infrastructure Administrator

Make sure there is adequate automatic and configurable SCM cleaning mechanism

– Authors will create branches for features, bug fixes or others. – Promoters will create releases of content packs that subsequently will trigger SCM tagging. In some cases releases might not prove to be valid for a production based deployment thus the SCM tag will be meaningless. – Authors should remove branches once their code goes into the master. – Promoters should clear SCM tags once a particular released content pack gets rejected.

Note: However in the event the above cleaning does not happen it is better to schedule weekly maintenance tasks to handle unneeded branches or irrelevant releases.

13

The HPE OO author

Ground rules and principles

14

As an HPE OO Author

Create a local workspace folder

– The local workspace folder should be created manually and easily browse from OO Studio to the local path. – The local workspace folder will match one SCM connection thus one SCM repository.

Note: It is advisable not to check the Do not ask again on the Select Workspace screen so that one can easily pick desired workspace at start time.

15

As an HPE OO Author

Configure SCM connection in HPE OO Studio

– Working in a multi-authoring environment means that there is a need to both: – be able to access others work. – provide one’s work to others.

– The first thing that needs to be done is to configure the SCM connection in HPE OO Studio.

16

As an HPE OO Author

Clone the remote repository

– Simply press the Clone button and clone the remote repository to the local workspace. Any remote data will be automatically retrieved.

Note: It is advisable to check the SCM messages panel for more details regarding any SCM actions performed.

17

As an HPE OO Author

Import or create a new project

– For an existing project: If a project has already been created and uploaded on the repository it should now exist in the local workspace. Use the Import Project option to import it in Studio. – For a new project: If you are starting the work on a new project, use the Create Project in order to create a new one directly from Studio.

18

As an HPE OO Author

Validate that necessary dependencies are imported in Studio

– Check the Properties of a project. – An orange based underline notifies that there might be problems related to the project’s properties. – Common messages relate to: – Dependencies not being imported in Studio (not present/possibly unused) – Dependencies not being configured (missing)

Note: It is the author’s role to make sure by following HPE OO Studio’s capabilities that the project’s dependencies are up to date and accurate.

19

As an HPE OO Author

Import the missing dependencies

– After identifying the relevant dependencies import them in Studio using the Import from the Dependencies section.

Note: The Properties item should have no warning once all the dependencies are imported.

20

As an HPE OO Author

Fetch from SCM

– Make is as a rule to Fetch from SCM whenever you start working on projects that are shared with others. – Fetching will help you get the picture of any changes that were done on the remote repository. – Use the SCM Graph log to understand if new commits have been done, new branches have been created or merged and so on.

Note: Use the SCM Graph Log whenever you want to easily fetch and get the overview of previous SCM actions.

21

As an HPE OO Author

Update from SCM

– If after a SCM fetch you identify there have been commits on the working branch it is better to update from SCM > Update All. This will help you to stay up to date with other changes brought by colleagues working on the same project/repository.

Note: Use SCM Changes panel for ease of access. Use SCM Messages panel to check the update results.

22

As an HPE OO Author

Create a branch

– With GIT it is better to work on branches and have pull requests to actually merge to the centralized codebase (master). That being said, if you are working on a new feature or simply fixing a defect it is better to create a branch and do all the changes on it. – From HPE OO Studio go to SCM changes, select Branching and Create Branch. Keep the branch name simple and intuitive. SCM Changes state will let you know on what branch you are working on.

Note: use feature/ or defect/ depending on the needs. This will help other authors understand the reason behind the branch and be able to point to it (checkout) whenever needed.

23

As an HPE OO Author

Checkout a branch

– If branches have already been created by other colleagues and you are simply bringing additions or fixes to their work there is no need to create a branch but rather Checkout and existing one. – From HPE OO Studio go to SCM changes, select Branching and Checkout Branch.

Note: If you cannot see a branch but your colleagues assured that there is one remote it is most likely because you did not perform Fetch SCM.

24

As an HPE OO Author

Implement the enhancements or fixes

– It is now time to start enhancing the content. Add new files or edit/delete existing ones. – As changes are done those can be monitored through both the SCM Changes panel or the coloring mechanism that the Projects panel has.

25

As an HPE OO Author

Commit changes

– Commit – which in GIT case is done on the local repository - allows you to capture the latest changes to the local repository so in case anything goes wrong you have a solid point of restore. – Commit as often as possible but keep in mind that every commit should be meaningful.

Note: You can commit particular changes by doing right click on an item/folder SCM>Commit or you can Commit All changes from SCM changes panel or SCM menu in which case all projects from a working repository are affected.

26

As an HPE OO Author

Always commit with relevant message

– Always put a relevant message to your commit. – Commit messages let other authors understand what it is about in both HPE OO Studio and external UI channels (GitHub UI, Jenkins UI and so on).

Note: If a commit comprises of multiple items being changed it is better to provide details about all the items in a meaningful manner (item and details of the changes being done).

27

As an HPE OO Author

Test the newly created code from Studio

– Studio allows you to run (debug) flows locally (using Studio’s own engine) or remotely (by targeting a remote Central). – As an author always debug your flows.

Note: It is advisable to debug flows both locally and remotely. By doing a remote debug you'll get a picture of how the flow will actually run and interact with the user in a run-time environment.

28

As an HPE OO Author

Test the newly created code from the DEV Central

– The remote debugger functionality of Studio should cover enough scenarios for an author to be able to test against the run time environment without the need of actually deploying the content pack into an HPE OO Central. – However, sometimes, as an author you might be inclined to deploy on a DEV HPE OO Central and test from within Central itself. – Authors should create a SNAPSHOT Content Pack from HPE OO Studio.

Note: Authors should create content packs from the working branch. Do note that OOSHA can be used too to create content packs. Check documentation for more details.

29

As an HPE OO Author

Test the newly created code from the DEV Central

Note: It is author’s responsibility to manage the SNAPSHOT based content packs that are created by Studio locally. Actions include deletion of files that are no longer needed

30

As an HPE OO Author

Test the newly created code from the DEV Central

– DEV Central environments can be shared by multiple authors. Thus conflicts of interest¹ can appear. – These can be avoided by author to author sync. The HPE OO Central provides enough information under the Content Packs section for one engineer to find out who deployed what and when. – Who: users should always login with their own account. – What: users can deploy their created SNAPSHOT(s) and the related dependencies. – When: can be used to understand if others are taking recent actions on the DEV Central or not.

Note: ¹Conflicts of interest can refer by example when two developers need a different version of the same dependency or simply they end up deploying same content pack with different actual content

31

As an HPE OO Author

Test the newly created code from the DEV Central

– If a conflict is spotted, assess the impact of the actions and act accordingly: – continue with deployment and test – and decide to rollback the deployment afterwards. – contact the last deployer of the content pack before taking other actions.

Note: Testing from the DEV Central can relate to triggering desired flows into execution, scheduling, access control, timeout configuration and others.

32

As an HPE OO author

Push commits

– With GIT it is not enough to commit. As mentioned, commits go to the local repository. For others to see your changes a Push needs to be done.

Note: If you are pushing a branch for the first time a notification will appear. Feel free to Push All.

33

As an HPE OO author

Resolve push commits

– It can happen that conflicts arrows if others already pushed changes to the same items on the remote repository. – Studio provides a mechanism to resolve those conflicts.

Note: If conflicts arrows, it is a must to resolve them before actually completing the push.

34

As an HPE OO author

Resolve push conflicts

– Conflicts can be resolved automatically or manually from within HPE OO Studio. It is a matter of preference per conflicted item. – In general if you are not familiar with all the other changes that were done it is better to resolve the conflicts manually as you get the possibility to review in parallel your changes and the others’ changes while working on a non-conflictual items merged file. – The actual final decision can be furthermore taken by right click on the relevant items and: Mark Conflict as Resolved, Resolve Conflict Accepting Ours or Resolve Conflict Accepting Theirs

35

As an HPE OO author

Resolve push conflicts

– Upon manually resolving conflicts the conflictual files (marked with red in Studio) can be opened in parallel for a visual investigation. – The editable item is the one partially merged with changes that did not raise any conflict.

Note: Keep using the SCM Messages panel for a better assessment of past and future actions

36

As an HPE OO author

Create a pull request

– Git’s pull request offers the ability to merge branches in a collaborative manner. – As an author of a feature or a defect fix, the pull request comes as the next action once the push has been performed. – This is done outside of HPE OO Studio.

Note: Our examples showcase GitHub Enterprise Pull request.

37

As an HPE OO author

Collaborate with authors and approvals until merge

– Once a pull request is created collaboration within the external tool can and should be used. – Other team members or teams can be mentioned within the comments and (if configured) notifications will be sent. – As an approver (assignee) use comments for providing review. – As an approver (assignee) merge or close the pull request.

38

As an HPE OO author

Notify Promoters

– As an author owning a feature or defect fix implementation, once the pull has been approved and the merge completed if there is no SCM notification for the Promoter, you should personally do the notification. – The notification should contain the affected project as well as SCM repository URL and high level mentions of the performed changes.

Note: It is better to have the Promoters – the engineers actually creating the RELEASE content packs – as subscribers to the SCM notifications related to project changes in particular the merges to the master branches. Remember that RELEASE based content packs get deployed to TESTING, STAGING, PRODUCTION.

39

As an HPE OO author

Delete branches

– Delete your branches that were already merged into master. – Before deletion, go to Studio and checkout to the master branch (from SCM Changes > Branching > Check-out). – In order to delete, go to the SCM UI and delete the unnecessary branch.

40

The HPE OO Promoter

Ground rules and principles

41

As an HPE OO Promoter

Create local repositories

– Create file disk repositories of the SCM repositories on which authors from different departments are

Note: If you are handling multiple departments and multiple projects it is better to have department based sub-root folders on your file system for ease of management

42

As an HPE OO Promoter

Keep local repositories up to date

– You will need to keep the repositories up to date. – Author rules apply here too. – Thus being said it is mandatory to follow the author steps #1 - #8.

Note: Make sure you are working on the relevant branch. As a Promoter the working branch should be the master branch.

43

As an HPE OO Promoter

Create Content Packs as RELEASE

– Use the Studio’s option Create content pack to pack all project’s files into a .jar file deployable in HPE OO Central. – Create a RELEASE version of a Content Pack to be deployed to runtime environments like TESTING, STAGING and PRODUCTION. – If everything goes well in the TESTING and STAGING areas these CPs created as RELEASE can get directly to the PRODUCTION.

Note: Upon a successful released CP creation the project’s next development version is increased and pushed to the SCM repository. Thus all authors, from this stage should update from master to get the latest development version. Do note that OOSHA can be used too to create content packs. Check documentation for more details.

44

As an HPE OO Promoter

Create Content Packs as RELEASE – validate items before Release

– As a Promoter you should use the Validate from Studio to make sure all the items in a project are actually valid before creating the content

Note: Use the Problems section to spot the Errors or the Warnings and work accordingly to fix them locally whenever possible or notify the relevant stakeholders

45

As an HPE OO Promoter

Create Content Packs as RELEASE – validate items before Release

– It is important to make sure there are no Invalid items before creating the release because the invalid items are not packed by Studio into the final content pack.

Note: A new content pack release creation will become mandatory once the invalid items are fixed. This will result in an increased content pack version which is to be avoided for such particular cases.

46

As an HPE OO Promoter

Upload Content Packs to Artifact Repository

– Before deploying to targeted Centrals the content packs should be stored on a centralized location. The best option would be to use an artifact repository. – Both Nexus and Artifactory offer open source versions also available as Docker containers.

Note: More examples are provided below with screenshots taken from Nexus OSS

47

As an HPE OO Promoter

Upload to Artifact Repository with the right GAV information

– Most artifact repositories are able to collect GAV related information from the .pom files. It is better to manually provide the right GAV. – It is better to define: – Group: domain.organization.department.[project].[module] (com.acme.devops) – Artifact: oo-[organization]-[department]-project-cp (oo-[acme][security]-fortify-cp) – Version: major.minor.micro (1.0.0)

Note: Values in square brackets should be considered optional but we provide them as a recommendation. Artifact matches format: -cp

48

As an HPE OO Promoter

Keep in mind that path is automatically created based on GAV

– The GAV that one provides will affect the end path thus the actual artifact location. – Keeping the proposed structure will mean you end up with a nice path and eventually things will be easier to spot and maintain.

Note: Values in square brackets should be considered optional but we provide them as a recommendation. Artifact matches format: -cp

49

As an HPE OO Promoter

Deploy to targeted Central

– Let’s assume the TESTING environment needs to be updated to the latest version of a content pack. – Download content pack from artifact repository – Download its dependencies from artifact repository¹ – Deploy to targeted Central

Note: ¹Dependencies can be identified by using HPE OO Studio and opening the Properties file of the artifact under discussion. If you are having PRO versions of Nexus or Artifactory you would be able to browse the actual CPs or set custom properties to keep track of the dependencies, which both would be advisable.

50

As an HPE OO Promoter

Take relevant actions in case of Central deployment warning(s)

– Deployment warnings can relate to CP signature or CP dependencies misalignment or complete missing. – It is better not to: – Ignore missing dependencies in the target Central – Ignore mentions of old dependencies actually being of the targeted Central part – Use unsigned CPs in Staging or Production – Use CPs with expired certificates in Staging or Production

51

As an HPE OO Promoter

Take relevant actions in case of Central deployment failure(s)

– Deployment failures can relate to either the targeted environment itself or to the actual content packs being deployed thus: – The actual root cause needs to be identified. – Central administrators need to be notified in case of environment related aspects (examples: database problems, network problems and so on). – Content authors need to be notified in case deployment fails because of inaccurate content being created (examples: duplicate paths error appears because authors have not applied the department name as root folder under Library).

52

As an HPE OO Promoter

Notify stakeholders of successful deployment

– Stakeholders may differ based on the Central being targeted: TESTING, STAGING, PRODUCTION. It is essential that all the stakeholders are being notified upon a successful deployment.

Note: Capture the details of content packs available in a particular environment after a successful deployment and send it over email.

53

As an HPE OO Promoter

Promote again and again

– As a Promoter one will need to deploy to different environments: TESTING, STAGING or PRODUCTION. – Same rules and principles should apply. – If a content pack passes the TESTING phase can be pushed to the STAGING or directly to the PRODUCTION (depending on the actual infrastructure) together with its dependencies. – If a content pack does not pass the TESTING or STAGING phase all the stakeholders should be notified. – The content authors will start implementing the fixes by creating new branches from now updated SCM master branches¹, the new CPs will have an incremented version and the entire process can start all

Note: ¹Check Promoter slide #3 to recap why and how the SCM master branches got updated.

54

As an HPE OO Promoter

Clear SCM tags in case of rejection

– If a content pack does not pass the TESTING or STAGING phase apart from notifying the stakeholders the Promoter should also make sure that the SCM tags related to the rejected content packs are removed from SCM. – This is to be done in order to make sure the SCM master will eventually contain only information related to ready to be released content packs.

55

The HPE OO administrator Ground rules and principles

56

As an HPE OO Administrator

Configure HPE OO Central according to the needs

– Configure runtime environments so that they match the internal needs. Changes are likely needed when new content packs are being deployed while not likely needed when new versions of content packs are being deployed – Examples: – New Configuration Items can/should be edited to match specific environment values (System accounts used for STAGING) – New flows can be scheduled at a needed recurrence – Action restrictions can/should be imposed on the new flows – Run time settings (timeout, persistence) can be configured on the new flows

57

As an HPE OO administrator

Take necessary actions upon request

– Promoters might notify of aspects like deployment failure or flow execution failure which might relate to environment configuration. – It is the HPE OO Central administrator who needs to remediate.

58

Thank you

59