containerd_cri_plugin
containerd cri plugin 初始化 PluginConfig containerdConfig “containerd” ContainerdRootDir ContainerdEndpoint RootDir StateDir Snapshotter string "snapshortter" DefaultRuntime Runtime "default_runtime" UntrustedWorkloadRuntime Runtime "untrusted_workload_runtime" Runtimes map[string] Runtime runtimes NoPivot deprecated 只影响v1.linux Type string “runtime_type” Engine string "runtime_engine" deprecated 只影响v1.linux Root string "runtime_root" 只影响linux.v1 Options *toml.Primitive "options" CniConfig “cni” NetworkPluginBinDir string “bin_dir” NetworkPluginConDir "conf_dir" NetworkPluginConfTemplate string "conf_template" Registry "registry" map [string] Mirrors "mirrors" Endpoints []string "endpoint" map [string] AuthConfig "auths" Username string "username" Password string "password" Auth string "auth" IdetityToken "indetitytoken" 1. 在main()启动之前,init()函数将cri plugin注册到 containerd InitContext中的plugin数组中,该init函数会 赋值plugin的InitFn为InitCRIService 2. containerd 进入main函数会LoadPlugin,调用cri plugin的 initCRIService()函数进行初始化 1. 赋值InitContext Meta的Platform 2. 赋值Exports map的CRIVersion 3. 加载ic.Config.(*criconfig.PluginConfig) 4. getServiceOpts() 问题 5. 初始化cri所需service的client端 client,err := containerd.New("", containerd.WithDefaultNamespace(), containerd.WithServices(serviceOpts)) CRIService Interface定义了几个方法接口的声明 及cirService对这些接口的定义 Run() 位于service.go io.Closer 即Close()方法 位于service.go Plugin.Service 即Register(*grpc.Server) 位于service.go grpcService runtime.RuntimeServiceServer实现 Runtime相关 runtimer.ImageServiceServer api.CRIPluginServiceServer Version() version.go RunPodSandbox() sandbox_run.go StopPodSandbox() sandbox_stop.go RemovePodSandbox() sandbox_remove.go ListPodSandbox() sandbox_list.go PodSandboxStatus() sandbox_status.go CreateContainer() container_create.go StartContainer() container_start.go StopContainer() container_stop.go RemoveContainer() container_remove.go ListContainers() container_list.go ContainerStatus() container_status.go UpdateContainerResources() container_update_resource.go ReopenContainerLog() container_log_reopen.go ExecSync() container_exesync.go Exec() container_exec.go Attach() container_attach.go PortForward() sandbox_portforward.go ContainerStats() container_stats.go ListContainerStats() container_stats_list.go UpdateRuntimeConfig() update_runtime_config.go Status() container_stats.go ListImages() image_list.go ImagesStatus() PullImage() image_pull.go ImageFsInfo() imagefs_info.go LoadImage() image_load.go 6. NewCRIService,构造CRIService c.imageFSPath = imageFSPath(config.ContainerdRootDir, config.ContainerdConfig.Snapshotter) 从配置文件中cni conf dir和cni bin dir加载netPlugin 创建StreamServer CONFIG criService config criconfig.Config imageFSPath apparmorEnabled bool seccompEnabled bool os osinterface.os sandboxStore *snadboxstore.Store sandboxNameIndex *registar.Registar containerStore *containerstore.Store containerNameIndex *registrar.Registrar imageStore *imagestore.Store snapshotStore *snapshotstore.Store netPlugin cni.CNI client *containerd.Client streamServer streaming.Server eventMonitor *eventMonitor sandboxes map[string]Sandbox idIndex *truncindex.TruncIndex Metadata Status StatusStorage Container containerd.Container NetNS *netns.NetNS *store.StopCh nameToKey map[string] string KeyToName map[string] string refCache map[string] string client *containerd.Client store *store address = “”,貌似实际上并不会同 containerd建立client端连接 snapshots map[string] Snapshot Key string Kind snapshot.Kind Inodes uint64 Timestamp int64 1. criService中containerd.Client如何同containerd中其他plugin service进行交互? 2. criService中streamServer用于何种场景 ? 3. eventMonitor监控什么事件,如何监控? service connMu conn *grpc.Clientconn runtime string connector func() (*grpc.ClientConn, error) c *criService ch <- chan *events.Envelope errCh <- chan error ctx context.Context cancel context.CancelFunc backOff *backOff 4. containerd创建grpc server后,会将plugin services注册到rpc,对于cri service,调用 criService的Register()方法,将criService注册,对于criService,主要注册三个模块的 server处理方法 1. RuntimeService相关 2. Image Service相关 3. CRI Plugin Service相关,目前就Load Image一种服务 3. 调用Run()开始启动criService 1. 注册containerd 事件订阅 /stasks/exit /tasks/oom /images 2. 开始recover pod、container和images 1. 通过container.Store列出所有Sandbox 并加载Sandbox状态 2. 通过container.Store列出所有containers并记载container状态 3. 恢复镜像信息 4. 清除所有无对应containerd 容器的孤儿Sandbox和container目录 3. 开始监控事件 4. 启动snapshots syncer 5. 启动streaming server 6. 监听eventMonitorErr和streamServerErr事件,进行出错处理 4. 为何将shim进程强制杀死后无法rmp