NameDateSize

..04-Feb-20204 KiB

.coafileH A D25-Sep-2018932

.gitignoreH A D25-Sep-2018250

.gitreviewH A D25-Sep-2018103

.jjb-test/H25-Sep-2018170

docs/H25-Sep-2018152

ensure-documented.shH A D25-Sep-20181.4 KiB

jenkins-admin/H25-Sep-2018125

jjb/H25-Sep-2018153

jjb-compare-xml.shH A D25-Sep-20181.9 KiB

license-header.txtH A D25-Sep-2018449

README.mdH A D25-Sep-20189.7 KiB

relnotes/H25-Sep-20184 KiB

requirements.txtH A D25-Sep-201873

shell/H25-Sep-20184 KiB

tox.iniH A D25-Sep-20181 KiB

yamllint.confH A D25-Sep-201883

README.md

1# Global JJB
2
3The purpose of this repository is store generically defined, reusable JJB
4templates, deployable across LF projects.
5
6Define the following variables in the Jenkins server as
7global environment variables as scripts in this repo expect these variables to
8be available.
9
10For example:
11
12```
13GERRIT_URL=https://git.opendaylight.org/gerrit
14GIT_URL=ssh://jenkins-$SILO@git.opendaylight.org:29418
15GIT_CLONE_URL=git@github.com:
16JENKINS_HOSTNAME=jenkins092
17LOGS_SERVER=https://logs.opendaylight.org
18NEXUS_URL=https://nexus.opendaylight.org
19SILO=releng
20SONAR_URL=https://sonar.opendaylight.org
21```
22
23Note: Use **GIT_CLONE_URL** for GitHub projects as this
24will be different from the URL used the poperties
25configuration.
26
27## Jenkins Plugin Requirements
28
29**Required**
30
31- [Config File Provider](https://plugins.jenkins.io/config-file-provider)
32- [Description Setter](https://plugins.jenkins.io/description-setter)
33- [Environment Injector Plugin](https://plugins.jenkins.io/envinject)
34- [Git plugin](https://plugins.jenkins.io/git)
35- [Post Build Script](https://plugins.jenkins.io/postbuildscript)
36
37    Due to security concerns Post Build Script is not available through
38    the Jenkins update center and must install it manually. You can
39    download it
40    [here](https://updates.jenkins-ci.org/download/plugins/postbuildscript/).
41
42- [SSH Agent](https://plugins.jenkins.io/ssh-agent)
43- [Workspace Cleanup](https://plugins.jenkins.io/ws-cleanup)
44
45**Required for Gerrit connected systems**
46
47- [Gerrit Trigger](https://plugins.jenkins.io/gerrit-trigger)
48
49**Required for GitHub connected systems**
50
51- [GitHub plugin](https://plugins.jenkins.io/github)
52- [GitHub Pull Request Builder](https://plugins.jenkins.io/ghprb)
53
54**Optional**
55
56- [Mask Passwords](https://plugins.jenkins.io/mask-passwords)
57- [MsgInject](https://plugins.jenkins.io/msginject)
58- [OpenStack Cloud](https://plugins.jenkins.io/openstack-cloud)
59- [Timestamps](https://plugins.jenkins.io/timestamper)
60
61## Installing global-jjb
62
63Deploy global-jjb in the ci-management repository's jjb directory as
64a submodule. Installing, upgrading, and rolling back changes is simple via the
65versioned git tags.
66
67```
68    # Choose a global-jjb version to install
69    GLOBAL_JJB_VERSION=v0.1.0
70
71    # Add the new submodule to ci-management's jjb directory.
72    # Note: Perform once per ci-management repo.
73    cd jjb/
74
75    # For production deployments:
76    git submodule add https://github.com/lfit/releng-global-jjb global-jjb
77    # For test deployments comment the above and uncomment the below
78    # git submodule add https://gerrit.linuxfoundation.org/infra/releng/global-jjb
79
80    # Checkout the version of global-jjb you wish to deploy.
81    cd global-jjb
82    git checkout $GLOBAL_JJB_VERSION
83
84    # Commit global-jjb version to the ci-management repo.
85    cd ../..
86    git add jjb/global-jjb
87    git commit -sm "Install global-jjb $GLOBAL_JJB_VERSION"
88
89    # Push the patch to ci-management for review
90    git review
91```
92
93## Parameters stored in defaults.yaml
94
95Configure the following parameters in the ci-management repo's
96defaults.yaml file.
97
98**gerrit-server-name**: The name of the Gerrit Server as defined
99in Gerrit Trigger global configuration.
100
101**jenkins-ssh-credential**: The name of the Jenkins Credential to
102use for ssh connections.
103
104If you are using GitHub then configure the following parameters
105in defaults.yaml
106
107**git-url**: Set this to the base URL of your GitHub repo. In
108general this should be <https://github.com>. If you are using
109GitHub Enterprise, or some other GitHub-style system, then it
110should be whatever your installation base URL is.
111
112**git-clone-url**: This is the clone prefix used by GitHub jobs.
113Set this to either the same thing as **git-url** or the
114'git@github.com:' including the trailing ':'
115
116**github-org**: The name of the GitHub organization interpolated
117into the scm config.
118
119**github_pr_org**: The name of the GitHub organization. All members
120of this organization will be able to trigger any job using the
121`lf-infra-github-pr` macro.
122
123**github_pr_whitelist**: List of GitHub members you wish to be able to
124trigger any job that uses the `lf-infra-github-pr-trigger` macro.
125
126**github_pr_admin_list**: List of GitHub members that will have admin
127privileges on any job using the `lf-infra-github-pr-trigger`
128macro.
129
130**lftools-version**: Version of lftools to install. Can be a specific version
131like '0.6.1' or a PEP-440 definition. <https://www.python.org/dev/peps/pep-0440/>
132For example `<1.0.0` or `>=1.0.0,<2.0.0`.
133
134**mvn-site-id**: Maven Server ID from settings.xml containing the credentials
135to push to a Maven site repository.
136
137**mvn-staging-id**: Maven Server ID from settings.xml containing the credentials
138to push to a Maven staging repository.
139
140defaults.yaml:
141
142```
143- defaults:
144    name: global
145
146    # lf-infra defaults
147    jenkins-ssh-credential: opendaylight-jenkins-ssh
148    gerrit-server-name: OpenDaylight
149    github-org: lfit
150    github_pr_whitelist:
151      - jpwku
152      - tykeal
153      - zxiiro
154    github_pr_admin_list:
155      - tykeal
156    lftools-version: '<1.0.0'
157    mvn-site-id: opendaylight-site
158```
159
160## Config File Management
161
162### Logs
163
164The logs account requires a Maven Settings file created called
165**jenkins-log-archives-settings** with a server ID of **logs** containing the
166credentials for the logs user in Nexus.
167
168## Deploying ci-jobs
169
170The CI job group contains jobs that should deploy in all LF
171Jenkins infra. The minimal configuration needed to deploy the ci-management
172jobs is as follows which deploys the **{project-name}-ci-jobs** job group as
173defined in **lf-ci-jobs.yaml**.
174
175ci-management.yaml:
176
177```
178- project:
179    name: ci-jobs
180
181    jobs:
182      - '{project-name}-ci-jobs'
183
184    project: ci-management
185    project-name: ci-management
186    build-node: centos7-basebuild-2c-1g
187```
188
189Required parameters:
190
191**project**: is the project repo as defined in source control.
192**project-name**: is a custom name to call the job in Jenkins.
193**build-node**: is the name of the builder to use when building (Jenkins label).
194
195Optional parameters:
196
197**branch**: is the git branch to build from.
198**jjb-version**: is the version of JJB to install in the build minion.
199
200## Deploying packer-jobs
201
202The packer job group contains jobs to build custom minion images. The minimal
203configuration needed to deploy the packer jobs is as follows which deploys the
204**{project-name}-packer-jobs** job group as defined in **lf-ci-jobs.yaml**.
205
206ci-management.yaml:
207
208```
209- project:
210    name: packer-jobs
211
212    jobs:
213      - '{project-name}-packer-jobs'
214
215    project: ci-management
216    project-name: ci-management
217    branch: master
218    build-node: centos7-basebuild-2c-1g
219
220    platforms:
221      - centos
222      - ubuntu-14.04
223      - ubuntu-16.04
224
225    templates:
226      - devstack
227      - docker
228      - gbp
229      - java-builder
230      - mininet
231
232    exclude:
233      - platforms: centos
234        templates: gbp
235      - platforms: centos
236        templates: mininet
237```
238
239Required parameters:
240
241**project**: is the project repo as defined in source control.
242**project-name**: is a custom name to call the job in Jenkins.
243**build-node**: is the name of the builder to use when building (Jenkins label).
244**platforms**: is a list of supported platforms.
245**templates**: is a list of supported templates.
246
247Optional parameters:
248
249**branch**: is the git branch to build from.
250**packer-version**: is the version of packer to install in the build minion,
251when packer is not available.
252**exclude**: is a combination of platforms and templates which are not required
253to build.
254
255## Deploying Python jobs
256
257We provide the following Python jobs templates:
258
259### {project-name}-tox-verify-{stream}
260
261Use this job to call python-tox to run builds and tests. The most common
262usage of this job is to run the Coala linter against projects.
263
264```
265- project:
266    name: builder
267    jobs:
268        - '{project-name}-tox-verify-{stream}'
269
270    project-name: builder
271    project: releng/builder
272    build-node: centos7-java-builder-2c-4g
273    stream: master
274```
275
276Required parameters:
277
278**project**: is the project repo as defined in source control.
279**project-name**: is a custom name to call the job in Jenkins.
280**build-node**: is the name of the builder to use when building (Jenkins label).
281**stream**: typically `master` or matching the build branch. This
282            is a useful keywords to map a release codename to a branch. For
283            example OpenDaylight uses this to map stream=carbon to
284            branch=stable/carbon.
285
286Optional parameters:
287
288**branch**: is the git branch to build from.
289**jjb-version**: is the version of JJB to install in the build minion.
290**tox-dir**: directory containing tox.ini file (default: '')
291**tox-envs**: tox environments to run (default: '')
292
293## Archiving logs in Jobs
294
295There are 2 ways supported for archiving log information:
296
2971) Job creates $WORKSPACE/archives directory and places logs there
298
299This method pushes the entire archives directory to the log server
300in the same structure as configured in the archives directory.
301
3022) Via job variable ARCHIVE_ARTIFACTS using globstar patterns.
303
304In this method a job can define a globstar for example `**/*.log` which then
305causes the archive script to do a globstar search for that pattern and archives
306any files it finds matching.
307
308## Overriding merge and verify triggers
309
310The default trigger conditions for Merge and Verify job types are overrideable
311in a project configuration by overriding the following variables:
312
313- gerrit_merge_triggers
314- gerrit_verify_triggers
315
316These variables take a list of trigger-on values as defined in JJB docs here:
317<https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit>
318
319## Appendix
320
321### ShellCheck
322
323When using ShellCheck to lint global-jjb or any projects that include
324global-jjb as part of their project (common with ci-management repos) then
325we require version 0.4.x of ShellCheck installed on the build vms. This version
326introduces annotations used by shell scripts in this repo.
327