Hello guys, today I am going to post a workaround that I am doing to get test reports from a kubernetes pod that has terminated. If you run your tests in a kubernetes pod, you must know by now that you can’t copy your files containing your test reports from a terminated pod. Many people were complaining about and you can see there is an open issue here.
I am going to show an example of api tests with postman and newman where I run the tests in a kubernetes pod and in the same microservices cluster since there is no public api to access them.
Create a Job
- First you need to create a job and save it in a .yaml file, like the example below:
apiVersion: batch/v1
kind: Job
metadata:
name: api-tests
namespace: default
spec:
parallelism: 1
template:
metadata:
name: api-tests
spec:
containers:
- name: api-tests
image: postman/newman:alpine
command: ["run"]
args: ["/etc/newman/test.postman_collection.json","--reporters","cli","--reporter-cli-no-failures"]
restartPolicy: Never
- Then run on your terminal or add it to your jenkins file:
kubectl apply -f job.yaml
Or you can also create a pod
- You just need to add this command to your jenkins file:
sh "kubectl run api-tests -i --rm --namespace=${ENVIRONMENT_NAMESPACE} --restart=Never --image=${YOUR_IMAGE}:latest --image-pull-policy=Always -- run /etc/newman/${YOUR_COLLECTION_PATH}.postman_collection.json -e /etc/newman/${YOUR_ENVIRONMENT_CONFIG_PATH}.postman_environment.json --reporters cli --reporter-cli-no-failures"
Why not a Deployment ?
In the beggining of the implementation I first tried to create a deployment, but deployments don’t support the policy to never restart the pod, which means the automation would never stop running and you wouldn’t be able to copy the reports from the container.
Copy the reports
So, now that your tests have finished, you can see the logs on jenkins showing they passed (or failed), but you want to extract the report from the logs and have them in a html/json/any file, so you can archive or publish them. Doing this you would be able to see clearer what is the issue and keep easy to access the reports for each pipeline.
Well,kubectl cp doesn’t work like docker cp unfortunatelly. Once your pod is terminated, you are not able to access the reports or anything inside the pod. So, for this reason there is an issue opened on the kubectl github repository that is exactly about that, you can check the progress of the issue here.
Now how can you copy the reports from the container if you can’t access it after the tests are finished ? Well, there is not a perfect way, some people send the reports to S3, some people send the reports to their emails, but I did find better to save the report copying the html code from the logs and saving it in a file.
On your jenkins file you will have the command to run the pod with the tests and after you need to cat the html report generated to be able to get everything inside the html tag and saving it in a file:
sh "kubectl run api-tests -i --rm --namespace=${ENVIRONMENT_NAMESPACE} --restart=Never --image=${YOUR_IMAGE}:latest --image-pull-policy=Always -- run /etc/newman/${YOUR_COLLECTION_PATH}.postman_collection.json -e /etc/newman/${YOUR_ENVIRONMENT_CONFIG_PATH}.postman_environment.json --reporters cli,html --reporter-html-export api-tests.html --reporter-cli-no-failures ; cat api-tests.html | tee report"
def report = readFile "report"
def update = report.substring(report.indexOf('<html>'), report.indexOf('</html>'))
writeFile file: "${workspace}/api-tests.html", text: update
sh "rm report"
First, you need to cat the html report that your tests generated (remember to have this script to run the tests in your docker image or the package.json if you use NodeJs, this is just an example). You can see that you will need to grab everything between the html tags. You can do this using substring or awk, whatever is your preference. I am using substring on this example, but if you want to filter out using awk, the code should be something like this awk '/<html>/','/<\\html>/'.
After grabbing the html report, I am saving it in a file and deleting the previous report file that contained the whole logs from the kubernetes pod.
Not perfect, not happy about doing these kind of workarounds, but this is a way to avoid sending the files to S3 or any other place. Hope it helps !
