Containers missing due to unknown runtime socket file location

Modified on Tue, 18 Oct, 2022 at 4:49 PM

Background


Chao successfully registers with the control plane, but when running a kubernetes attack, it doesn't recognize any containers on your system. This means that the kubernetes attack ran will result in "error, could not resolve to any containers".


Steps to remediate


The first thing to always check is the gremlin daemon.log file. You're trying to see if there are any errors in the logs. If there aren't any specific errors that can be addressed, it's likely that our Gremlin Agent can't find the socket file for your container runtime. 


2021-11-17 01:26:52 - container driver: containerd-runc not suitable for this system: socket not available: /run/containerd/containerd.sock: No such file or directory (os error 2)

In this scenario, since we noticed that the Gremlin agent is using containerd-runc we wanted to find the correct socket file so we ran 


find / -name runc

You must also set the container driver parameter, gremlin.container.driver=containerd-runc, in order for the runtimeSocket parameter to be applied.


helm upgrade gremlin gremlin/gremlin --namespace gremlin --set gremlin.hostPID=true --set gremlin.container.driver=containerd-runc --set containerDrivers.containerd-runc.runtimeSocket=/run/k3s/containerd/containerd.sock --set gremlin.secret.managed=true --set gremlin.secret.type=secret --set gremlin.secret.teamID=******** --set gremlin.secret.teamSecret=*********** --set gremlin.secret.clusterID=*****

You can see a list of the default socket file locations that the Gremlin Agent tries to use here. For containerd-runc, the default location of the socket file is "/run/containerd/containerd.sock", but in some orchestrated environments (RKE2 for this example) the socket file is located somewhere else like "/run/k3s/containerd/containerd.sock". In order to configure the socket file location specific to your orchestrated environment you can set the value in the helm chart with this command: 


--set containerDrivers.{containerDriver-name}-runc.runtimeSocket=/path/to/file.sock

NOTE: we suffix the containerDriver name with "runc" in our helm charts!

here's a good example for RKE2:


--set containerDrivers.containerd-runc.runtimeSocket=/run/k3s/containerd/containerd.sock

If you're not using helm you can update your client.yaml file by adding a value that looks like this:


- name: containerd-sock
          hostPath:
            path:  /run/containerd/containerd.sock

 


 


 


 


 

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons

Feedback sent

We appreciate your effort and will try to fix the article