chore: expose a method to toggle the torch light in Camera component

Summary
This pull request exposes a method to toggle the torch light in the Camera component. The flashMode and torchMode offered by the Camera component are not working in both iOS and Android. An issue is already open regarding this https://github.com/teslamotors/react-native-camera-kit/issues/518.
This is blocking my team from using react-native-camera-kit. We really need the change introduced in this PR to unblock us.
Now, when an app wants to turn on or off the torch light, it can call the setTorchMode and provide a mode (either on or off)
How did you test this change?
First, I run the example folder on both Android and iOS and made sure that it is still working.
Also, run a test in an app on both Android and iOS using a physical Android device (Motorola) and a physical iPhone 12.
Here is a code implementation example. Basically, just need to call the setTorchMode on the ref instance of the camera.
    import React, { useRef, useState } from 'react';
    import {
      Dimensions,
      StyleSheet,
      View,
      TouchableOpacity,
      Text,
    } from 'react-native';
    import { Camera, CameraType } from 'react-native-camera-kit';

    const CameraOverlay = () => {
      const camera = useRef(null);
      const [toggleFlash, setToggleFlash] = useState(false);

      return (
        <View style={[styles.cameraViewContainer]}>
          <View style={[styles.mainContainer]}>
            <Camera
              ref={(ref) => (camera.current = ref)}
              cameraType={CameraType.Back}
              style={styles.cameraStyle}
            />

            <TouchableOpacity
              style={
                toggleFlash
                  ? [styles.flashlightTogglerStyle, styles.flashlightOnStyle]
                  : [styles.flashlightTogglerStyle, styles.flashlightOffStyle]
              }
              onPress={() => {
                camera.current.setTorchMode(!toggleFlash ? 'on' : 'off');
                setToggleFlash((current) => !current);
              }}
              activeOpacity={0.7}
            >
              <Text>Flash Light</Text>
            </TouchableOpacity>
          </View>
        </View>
      );
    };

    const styles = StyleSheet.create({
      mainContainer: {
        flex: 1,
        justifyContent: 'center',
        alignItems: 'center',
      },
      cameraViewContainer: {
        aspectRatio: 1,
        height: '90%',
        borderWidth: 1,
        borderRadius: 20,
        borderColor: 'transparent',
      },
      cameraStyle: {
        width: '100%',
        height: '90%',
        borderRadius: 20,
      },
      camera: {
        flex: 1,
        alignItems: 'center',
        justifyContent: 'center',
        backgroundColor: 'transparent',
        height: Dimensions.get('window').width,
        width: Dimensions.get('window').width,
      },
      flashlightTogglerStyle: {
        position: 'absolute',
        width: 50,
        height: 50,
        top: 10,
        right: 10,
        borderRadius: 23,
        justifyContent: 'center',
        alignItems: 'center',
      },
      flashlightOnStyle: {
        backgroundColor: '#fff',
      },
      flashlightOffStyle: {
        backgroundColor: 'lightgray',
      },
    });

    export default CameraOverlay;

Just bellow I included two videos to demo this change. When the 'Flash Light' button becomes white, the torch is on. And you can see that the torch light turned on (the background got lighter). When this button becomes gray, the torch is off, and the phone's light turns off.
Here is a video that shows the test results on Android:
https://user-images.githubusercontent.com/63681471/216187083-9d1d0917-1a5c-4cf1-848d-8d7fd8ef2c0e.mp4
And this one on an iPhone:
https://user-images.githubusercontent.com/63681471/216192498-5284db28-c3f6-4221-8546-0ebec8f480ab.MOV

great PR I need it on my project please merge it ASAP!

@scarlac Would be very appreciate if you could review this PR.

All camera flow working as expected except the flashlight!
I was running into same issue, this PR can really help in my project. Please merge it as soon as possible. So, I can have it on my project.
@scarlac Much appreciated, if someone can review and approve this PR!
Thanks!!

please merge

🚢

Please merge

Merged to master. Thank you everyone for pushing for it.

Can you provide dist folder to repo, it got error when npm install from master branch because it doesn't generate dist folder

@caturadjiprasetyo need to wait till they release a new version to be able to use this. Last release was last August

@yousrasd, torchMode props is enough to turn on/off the torch as we can see in CameraScreen. The root cause of your issue is most probably elsewhere. I'm going to remove setTorchMode that is redundant, but feel free to share your use case so we can dig deeper
Here is your same exact example, using torchMode props, getting the same end result
import React, { useState } from 'react';
import { Dimensions, StyleSheet, View, TouchableOpacity, Text } from 'react-native';
import { CameraType } from '../../src/types';
import { Camera } from '../../src';

const CameraOverlay = () => {
  const [toggleFlash, setToggleFlash] = useState(false);

  return (
    <View style={[styles.cameraViewContainer]}>
      <View style={[styles.mainContainer]}>
        <Camera torchMode={toggleFlash ? 'on' : 'off'} cameraType={CameraType.Back} style={styles.cameraStyle} />

        <TouchableOpacity
          style={[styles.flashlightTogglerStyle, toggleFlash ? styles.flashlightOnStyle : styles.flashlightOffStyle]}
          onPress={() => {
            setToggleFlash((current) => !current);
          }}
        >
          <Text>Flash Light</Text>
        </TouchableOpacity>
      </View>
    </View>
  );
};

const styles = StyleSheet.create({
  mainContainer: {
    flex: 1,
    justifyContent: 'center',
    alignItems: 'center',
  },
  cameraViewContainer: {
    aspectRatio: 1,
    borderWidth: 1,
    borderRadius: 20,
    borderColor: 'transparent',
  },
  cameraStyle: {
    width: '100%',
    height: '90%',
    borderRadius: 20,
  },
  camera: {
    flex: 1,
    alignItems: 'center',
    justifyContent: 'center',
    backgroundColor: 'transparent',
    height: Dimensions.get('window').width,
    width: Dimensions.get('window').width,
  },
  flashlightTogglerStyle: {
    position: 'absolute',
    width: 50,
    height: 50,
    top: 10,
    right: 10,
    borderRadius: 23,
    justifyContent: 'center',
    alignItems: 'center',
  },
  flashlightOnStyle: {
    backgroundColor: '#fff',
  },
  flashlightOffStyle: {
    backgroundColor: 'lightgray',
  },
});

export default CameraOverlay;


QUESTION: How exactly do I use tor?

I have never used it before and even less in the expert bundle option.

Not getting your accounts banned

Not getting your accounts banned

but how, i never use it

Read the tutorial

Read the tutorial

Oh, i didnt see it, last question, which .exe i need to put in system variable

You should put the entire folder


Syntax Highlighting

JUMP:
This option stays at a dark blue color no matter what I set it to.  The others seem to be working.
MISC:
Doesn't seem to pick up signed data types like. SDWORD, SWORD, etc
Overall really cool project and look forward to seeing how it progresses.  Thanks.

Thanx for reporting. The color customization for the jump is fixed and the signed datatypes are added. You can either wait for a next release or compile the sources and use it straight away.


Refactor: pay form

Description
Refactor pay form component. Decompose it into smaller components. Use functional component + hooks instead of class

@korhaliv I can take a stab at this. I was wondering about the "Use functional component + hooks instead of class" bit because function components don't support proptypes AFAIK. Do we want to get rid of the existing proptypes on the Pay component and whatever components will be split out of it?

Functional components actually support proptypes without any issues. We have many instances of that in our codebase. For example
https://github.com/LN-Zap/zap-desktop/blob/next/renderer/components/UI/ActionButton.js
Also, be advised that this kind of PR should go against the next branch

@korhaliv right, I see. I don't usually use proptypes and React docs show them being used with class components only so I assumed there weren't supported with function components.


Travis openmpi/linux fails

Describe the bug
OpenMPI/Linux tests fail on Travis, not exactly sure why.
#350 disables these tests for now, since in practice it works fine and it seems better to get at least some useful feedback from testing (instead of them always failing).

it used to work, but then something changed on the test machines/configurations (you can go back in history to retry one of the commits that passed and see them fail), so I was hoping it catch it working again;-) in any case maybe better to set allow_failure for this test?
Can you confirm the test works on real machine with the same config??
there is another problem with the travis script, it passes sometimes while tests are failing...

Need to test this with the updated config

travis retired


Error parsing floats with really small exponents

Hello, I have been trying the web version to visualize some BVH files, and I think i found a small bug when the parser tries to parse floats with exponents that have 2 digits in them. I append an image that shows the error.

Even though I am not sure that having 2 digits is the problem, I think that is the case, because the line the error occurs in (shown in the next image) fails with the e-40 exponent, and if I remove the 0 to leave a e-4 exponent, the error goes away, and catches the next 2 digit exponent.

Thanks in advance.

Thanks again for the big report @enekoatxa - can you upload the file causing issues so I can take a look? I suspect the problem is the number is too small to be represented as a float. I can potentially switch the parser to parse doubles and then convert to floats. As a work-around you might be able to get it to work by converting to floats before you write out the bvh file you are exporting (assuming it is your own script doing the export).

Oh, you may be completely right. I deleted the file, sorry, but I generated a new one, that also fails. This one fails at line 541:287, with a number with exponent e-39.
randomGenerator.zip
Meanwhile, I will change my script to export in a better format (I overlooked the unnecessary precision of the output that I generate).
Thank you so much!

I've pushed up a fix for this here:
https://github.com/orangeduck/BVHView/commit/800e4035a988249ca17c63a882c0db119b4210a7
Now it can open that file. As mentioned in the other thread I will update the pre-compiled windows binaries and web demo next week :)
Thanks,
Dan

Thank you so much, again!


Suggestion / Feature implementation

Suggestion,
EnderIO AE2 Implementation is incomplete and was really hoping to use it for some complex nextworks, but until it can connect directly to P2P tunnels it makes my design plans un-useable. is there an ETA on when this feature will be finished?
Feature Implementation:
In the event you wouldn't want to make the above doable i was also thinking a multipart block like the already existing P2P tunnels could be made for the use of Ender IO cabling. but this idea may be more work than just implementing it into the cable itself, but it would also allow you to make the Costs for making this block more expensive to compensate, like adding a P2P tunnel in the recipe like so.
P2P tunnel = P
Ender IO Data Cable (8channel)= D
Conduit Binding = B
BDB
DPD
BDB
EnderIO P2P Tunnels that attatch to EnderIO Dense Cables.
Thank you for all your Hard work and your making of this perfectly outstanding Mod.
DragonPC

unclear or duplicate of #3008


Basic extension catalog for kernels, plus demo on early extensibility.

Before I start implementing, I'd like to share an overview of my plans with everyone interested. Feel free to comment.
Goals

introduce initial version of basic kernel extension catalog, while not changing too many other things, for now (i.e., refactoring to enable extensibility afterwards)
assumption: types of all intermediates are known at compile-time (the current implementation assumes this too; we will relax that later)
functionality first, performance second, for now

Planned steps

introduce new class for the kernel extension catalog with very basic preliminary interface for adding new kernels and querying the registered kernels

representation of registered kernels

key: (name of a) DaphneIR op (since we always want a kernel for a specific op)
value: complex object

list of input types
list of output types
processing backend (or "API", following the terminology in kernels.json) (CPP, CUDA, FPGAOPENCL, ...)

interface

method to add a new kernel with its information
method to retrieve the information for a DaphneIR op
will be extended later

populate kernel extension catalog at system start-up based on the information in kernels.json

parse that JSON file and enter information in the kernel registry
no means for adding kernels exposed to users yet, just using the kernels whose pre-compilation is currently hard-coded
requires kernels.json in release artifact (maybe move that file)
anyway, using that file is just a temporary solution (see follow-up issues)

make DAPHNE compiler use the kernel extension catalog for lowering DaphneIR domain-specific ops to kernel calls

recap of current design

MarkCUDAOpsPass and MarkFPGAOPENCLOpsPass mark ops to be executed on the respective processing backend if certain conditions are met
AdaptTypesToKernelsPass harmonizes input and output types for certain ops, assuming that kernels are usually available for homogeneous type combinations
RewriteToCallKernelOpPass rewrites domain-specific DaphneIR ops to CallKernelOp; the name of the kernel function is created by naming convention taking into account the op name, input and output types, and hints on the processing backend; but does not know if that kernel really exists
LowerToLLVMPass lowers VectorizedPipelineOp and DistributedPipelineOp to CallKernelOp

planned design

MarkCUDAOpsPass and MarkFPGAOPENCLOpsPass remain as they are
AdaptTypesToKernelsPass remains as it is, but might not be used, for now (but could be useful again later)
SelectKernelsPass (new): queries the new kernel extension catalog by (op name, processing backend) and (1) decides which kernel function to call and adds a hint (attribute) to the op, and (2) inserts casts of inputs and outputs where necessary

raises an error if no kernel is registered for this operation
if a kernel for the required type combination is available, use that
otherwise, use any alternative kernel with the lowest cost; whereby the cost is (a) infinity if the kernel cannot be applied without loss of precision, and (b) the number of required casts to insert otherwise
this pass is separate from RewriteToCallKernelOpsPass, since the casts it introduces could offer potential for further optimization

RewriteToCallKernelOpPass will merely rewrite ops to CallKernelCallOp, based on decisions that have been made before, reflected by attributes of the op
LowerToLLVMPass stays as it is (those two ops shall have very generic kernels anyway)

Possible follow-up issues (not to be addressed by this PR)

steps towards extensibility features

make kernel extension catalog accessible to user
allow plugging in additional shared libs with more kernels
multiple dispatch and support for kernels on super classes of DAPHNE data types
add performance profiles (of whatever form) to the kernel registry for cost-based kernel/device selection

detailed improvements

more sophisticated selection of alternative kernels by better modeling of the costs (e.g., take data sizes into account (logical, physical), try not to deflate data more than necessary, rather deflate small than large data objects, …)
canonicalization for CastOp (remove cast from x to x; simplify cast from x to y to z to cast from x to z, where possible)

Closes #455.

@EricMier: I think this could be interesting to you. Once we have the initial extension catalog for kernels, it could become the home of any performance profiles for kernel/device selection. The kernel extension catalog could be accessible throughout the DAPHNE compiler, such that any pass could query its information, e.g., a separate pass selecting a suitable device (as an alternative/improvement to MarkCUDAOpsPass etc.) Feel free to let me know your thoughts and if this sounds in-line with your goals.

@corepointer: JFYI, since you've also worked on the lowering from DaphneIR ops to kernel calls, especially MarkCUDAOpsPass and MarkFPGAOPENCLOpsPass.

This PR is ready to be merged now, from my point of view. It contains the basic extension catalog for kernels (see above, with a few minor deviations from the initial ideas) plus a few more commits that show its end-to-end use.
Of course, feedback is welcome, but I'm not asking for a code review. However, as this PR changes a few important things, I would like to give everyone some time to have a look before I merge it. Unless anyone speaks up, I will merge this PR next Monday (Apr 29). The commits in this PR are meant to be "rebased & merged", not ~"squashed & merged"~.
These are the most important changes (see the commit messages for more details):

Introduction of an initial extension catalog for kernels (a73dfbeeb221a614e24cf9f499817a69c8e5abfa)
The DAPHNE compiler finally knows which kernels exist in pre-compiled form, since the available kernels are registered in a data structure, the kernel extension catalog. This catalog is populated from a JSON file at DAPHNE start-up, the JSON file is automatically generated along with the kernel instantiations and resides in lib/ (such that it should already be covered by our scripts for deployment and release packaging). When generating calls to pre-compiled kernels, the DAPHNE compiler looks up the kernel function names in the catalog, instead of using a naming convention.
Automatic determination of the required kernel libs (894f5639b0f919598d07138a08add35fbb518867)
The DAPHNE compiler additionally knows in which shared lib each kernel is. Based on that, the DAPHNE compiler knows which shared libs we really need to link with for the DaphneDSL script at hand. The information on the shared libs is part of the kernel catalog JSON files. Thus, the way the paths to these libs were managed before could be simplified.
Initial support for kernel hints in DaphneDSL, a test case of a simple kernel extension, and initial docs on kernel extensions (9eb00f132bfe7f46e165edc86cae5c39f19ce8c3, 62d8dc6ef0d3841f0cbb26291a13dd1ea092acbf, 131ed8248ec84857afd1dd21e5858760b70841ba)
These commits mainly add new material, so it's less likely they conflict with anyone else's interests. I decided to add them to this PR, since they showcase the end-to-end use and added value of the extension catalog. It is essentially a cleaned-up and extended version of the demo on extensibility I've shown at the DAPHNE General Assembly meeting in December 2023.

This PR is just the first step towards extensibility in DAPHNE and there are many more aspects and details we will work on in the future.

Thanks. I'm sure it still has several limitations and can be improved from an efficiency and usability point of view, but it is a first step that we can build upon.

I wrote a couple other extension and that, following the example in doc/Extensions.md worked quite well.
The only thing I would like to see in doc/Extensions.md is how you get the correct mnemonic of a Daphne op.
For example, when doing the same for the aggMax built-in, the correct mnemonic in the kernel_extensions.json would be maxAll. The only place where (besides at some points in the IR) I would be able to see that would be in DaphneOps.td where the op is defined. Maybe an explanation or link to figure out what the correct mnemonic is would be helpful.
Was also confused by the wrong results in our internal kernel, but I just saw you have already created an issue for that.
Otherwise, I think this looks great so far!

Awesome work @pdamme :+1:
I read through all five commits with adequate care (I hope) to understand what's going on but I did not compile and run it yet. So I don't have any particular question upfront without getting my hands dirty. But I'm looking forward to getting into the details and to see catalog.json :eyes:

Thanks for the feedback, @philipportner. Happy to hear that you succeeded writing some little extensions. Indeed, the documentation is not comprehensive yet. I didn't want to spend to much time on that as we may still change a few things. However, I will add a short note (for developers) on where to get essential information to the docs (op mnemonics in DaphneOps.td and kernel interfaces in the generated kernels.cpp).
Thanks also to @corepointer. Merging it in is not super urgent, so simply let me know if you would like to have a closer look and try it out before I merge it. I would be happy to merge it by the end of this week, though (unless any significant concerns arise).

I've rebased this PR and tried to adapt the changes it contains to the new error handling from #706 as good as I could (hope I didn't miss anything). I've also added a note on where DAPHNE developers can find references of the op mnemonics and kernel interfaces to the docs on writing a custom kernel extension.


VMI stuck pending

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

/kind bug
What happened:
After creating a VM, the VMI stays in pending with no events.
What you expected to happen:
The VMI should start, or at least give anerror
How to reproduce it (as minimally and precisely as possible):
---
apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachine
metadata:
  name: vm-lucy
  namespace: vm-lucy
  labels:
    kubevirt.io/vm: vm-lucy
spec:
  running: true
  template:
    metadata:
      name: vm-lucy
      namespace: vm-lucy
      labels:
        kubevirt.io/vm: vm-lucy
    spec:
      domain:
        cpu:
          cores: 4
        devices:
          disks:
            - disk:
                bus: virtio
              name: vdisk-lucy
            - disk:
                bus: virtio
              name: cloudinit
        resources:
          requests:
            memory: 2G
      volumes:
        - dataVolume:
            name: dv-lucy
          name: vdisk-lucy
        - cloudInitNoCloud:
            userData: |
              #cloud-config
              password: changeme
              chpasswd: { expire: False }
          name: cloudinit

Anything else we need to know?:
Bare metal K8S cluster, all capable of KVM
Environment:

KubeVirt version (use virtctl version): Client Version: version.Info{GitVersion:"v0.24.0",
Kubernetes version (use kubectl version): Client Version: version.Info{Major:"1", Minor:"14"
VM or VMI specifications: as above
Cloud provider or hardware configuration: Bare metal, 3 physical host, ceph storage
OS (e.g. from /etc/os-release): Debian 10
Kernel (e.g. uname -a): 4.19.0-6-amd64
Install tools: RKE
Others:

@aneurinprice could you do kubectl describe vmi vm-lucy? That should give you events and entries in status.conditions which should tell you why it does not start.

Also if that still does not show you anything (and yes we should improve that then, because it should), you should see a pod called virt-launcher-vm-lucy-*. The events on that pod should reveal what the issues are, and I would be curious to hear what the issue is.

@rmohr nothing unfortunately note.
`mac-n-cheesenyepricelocal:lucy moby$ kubectl describe vmi -n vm-lucy vm-lucy
Name:         vm-lucy
Namespace:    vm-lucy
Labels:       kubevirt.io/vm=vm-lucy
Annotations:  kubevirt.io/latest-observed-api-version: v1alpha3
kubevirt.io/storage-observed-api-version: v1alpha3
API Version:  kubevirt.io/v1alpha3
Kind:         VirtualMachineInstance
Metadata:
Creation Timestamp:  2019-12-12T17:33:34Z
Finalizers:
foregroundDeleteVirtualMachine
Generate Name:  vm-lucy
Generation:     2
Owner References:
API Version:           kubevirt.io/v1alpha3
Block Owner Deletion:  true
Controller:            true
Kind:                  VirtualMachine
Name:                  vm-lucy
UID:                   164d8849-feab-4390-b267-fa3aa35e82ed
Resource Version:        1157878
Self Link:               /apis/kubevirt.io/v1alpha3/namespaces/vm-lucy/virtualmachineinstances/vm-lucy
UID:                     054b9c1b-e198-4947-a322-2384d44402c2
Spec:
Domain:
Cpu:
Cores:  4
Devices:
Disks:
Disk:
Bus:  virtio
Name:   vdisk-lucy
Disk:
Bus:  virtio
Name:   cloudinit
Interfaces:
Bridge:
Name:  default
Features:
Acpi:
Enabled:  true
Firmware:
Uuid:  a4531ca3-316b-57e5-b6a3-9e8c5789a400
Machine:
Type:  q35
Resources:
Requests:
Cpu:     100m
Memory:  2G
Networks:
Name:  default
Pod:
Volumes:
Data Volume:
Name:  dv-lucy
Name:    vdisk-lucy
Cloud Init No Cloud:
User Data:  #cloud-config
password: changeme
chpasswd: { expire: False }
Name:  cloudinit

Status:
Guest OS Info:
Phase:  Pending
Events:   `
`kubectl get all -n vm-lucy
NAME                                         AGE   PHASE     IP    NODENAME
virtualmachineinstance.kubevirt.io/vm-lucy   19h   Pending
NAME                                 AGE   RUNNING   VOLUME
virtualmachine.kubevirt.io/vm-lucy   19h   true   `
`kubectl describe vm -n vm-lucy vm-lucy
Name:         vm-lucy
Namespace:    vm-lucy
Labels:       kubevirt.io/vm=vm-lucy
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"kubevirt.io/v1alpha3","kind":"VirtualMachine","metadata":{"annotations":{},"labels":{"kubevirt.io/vm":"vm-lucy"},"name":"vm...
kubevirt.io/latest-observed-api-version: v1alpha3
kubevirt.io/storage-observed-api-version: v1alpha3
API Version:  kubevirt.io/v1alpha3
Kind:         VirtualMachine
Metadata:
Creation Timestamp:  2019-12-12T17:33:34Z
Generation:          3
Resource Version:    1157877
Self Link:           /apis/kubevirt.io/v1alpha3/namespaces/vm-lucy/virtualmachines/vm-lucy
UID:                 164d8849-feab-4390-b267-fa3aa35e82ed
Spec:
Running:  true
Template:
Metadata:
Creation Timestamp:
Labels:
Kubevirt . Io / Vm:  vm-lucy
Name:                  vm-lucy
Namespace:             vm-lucy
Spec:
Domain:
Cpu:
Cores:  4
Devices:
Disks:
Disk:
Bus:  virtio
Name:   vdisk-lucy
Disk:
Bus:  virtio
Name:   cloudinit
Machine:
Type:  q35
Resources:
Requests:
Memory:  2G
Volumes:
Data Volume:
Name:  dv-lucy
Name:    vdisk-lucy
Cloud Init No Cloud:
User Data:  #cloud-config
password: changeme
chpasswd: { expire: False }
    Name:  cloudinit

Status:
Created:  true
Events:     `

@aneurinprice hm, could you also check if you see a virt-launcher-* pod?
Also it could be that our control-plane may not be running. Could you share kubectl get pods -n kubevirt?

kubectl get pods -n kubevirt
NAME                               READY   STATUS    RESTARTS   AGE
virt-api-787db99cd9-psgjr          1/1     Running   0          20h
virt-api-787db99cd9-xqpf8          1/1     Running   0          20h
virt-controller-6d46f99d88-fv5qf   1/1     Running   6          20h
virt-controller-6d46f99d88-jwrv5   1/1     Running   4          20h
virt-handler-hm688                 1/1     Running   0          20h
virt-handler-nrc2g                 1/1     Running   0          20h
virt-handler-wrrvm                 1/1     Running   0          20h
virt-operator-bd7d6c76b-k7fdr      1/1     Running   6          20h
virt-operator-bd7d6c76b-pvggc      1/1     Running   6          20h

 kubectl get events -n kubevirt
LAST SEEN   TYPE      REASON           OBJECT                                 MESSAGE
23m         Warning   Unhealthy        pod/virt-api-787db99cd9-psgjr          Readiness probe failed: Get https://<IP_ADDRESS>:8443/apis/subresources.kubevirt.io/v1alpha3/healthz: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
23m         Warning   Unhealthy        pod/virt-api-787db99cd9-xqpf8          Readiness probe failed: Get https://<IP_ADDRESS>:8443/apis/subresources.kubevirt.io/v1alpha3/healthz: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
23m         Normal    Pulled           pod/virt-controller-6d46f99d88-jwrv5   Container image "index.docker.io/kubevirt/virt-controller@sha256:94eff668e3145ff59faec450afb50a7843a325b822f2c4ce07d18aaa2a4bd26a" already present on machine
23m         Normal    Created          pod/virt-controller-6d46f99d88-jwrv5   Created container virt-controller
23m         Normal    Started          pod/virt-controller-6d46f99d88-jwrv5   Started container virt-controller
24m         Warning   BackOff          pod/virt-controller-6d46f99d88-jwrv5   Back-off restarting failed container
25m         Normal    LeaderElection   endpoints/virt-controller              virt-controller-6d46f99d88-jwrv5 became leader
23m         Normal    LeaderElection   endpoints/virt-controller              virt-controller-6d46f99d88-jwrv5 became leader
22m         Normal    Pulled           pod/virt-operator-bd7d6c76b-k7fdr      Container image "index.docker.io/kubevirt/virt-operator@sha256:895ab1faed0557ce488241785019cb01ae1906c4ded5e96cfdd478b56ad1b87f" already present on machine
22m         Normal    Created          pod/virt-operator-bd7d6c76b-k7fdr      Created container virt-operator
22m         Normal    Started          pod/virt-operator-bd7d6c76b-k7fdr      Started container virt-operator
23m         Warning   Unhealthy        pod/virt-operator-bd7d6c76b-k7fdr      Readiness probe failed: Get https://<IP_ADDRESS>:8443/metrics: dial tcp <IP_ADDRESS>:8443: connect: connection refused
23m         Warning   BackOff          pod/virt-operator-bd7d6c76b-k7fdr      Back-off restarting failed container
25m         Normal    Pulled           pod/virt-operator-bd7d6c76b-pvggc      Container image "index.docker.io/kubevirt/virt-operator@sha256:895ab1faed0557ce488241785019cb01ae1906c4ded5e96cfdd478b56ad1b87f" already present on machine
25m         Normal    Created          pod/virt-operator-bd7d6c76b-pvggc      Created container virt-operator
25m         Normal    Started          pod/virt-operator-bd7d6c76b-pvggc      Started container virt-operator
24m         Warning   Unhealthy        pod/virt-operator-bd7d6c76b-pvggc      Readiness probe failed: Get https://<IP_ADDRESS>:8443/metrics: dial tcp <IP_ADDRESS>:8443: connect: connection refused
25m         Warning   BackOff          pod/virt-operator-bd7d6c76b-pvggc      Back-off restarting failed container
26m         Normal    LeaderElection   endpoints/virt-operator                virt-operator-bd7d6c76b-pvggc became leader
25m         Normal    LeaderElection   endpoints/virt-operator                virt-operator-bd7d6c76b-k7fdr became leader
23m         Normal    LeaderElection   endpoints/virt-operator                virt-operator-bd7d6c76b-k7fdr became leader
23m         Normal    LeaderElection   endpoints/virt-operator                virt-operator-bd7d6c76b-pvggc became leader

@rmohr I think this is a strange problem. I have checked the hosts and the ARE capable of running KVM machines

Hm, it is strange that virt-controller and virt-operator are restarting quite often.
Apart from that, did you also check if a virt-launcher-* pod got created?

If there is no pod created, we would have to check the logs of virt-controller-* pods. They are responsible for the VMI. They may have a permission issue or something else.

So, it doesnt look like the virt-launcher-* pod gets created.
kubectl apply -f lucy-vm.yaml; for i in {1..5}; do kubectl get pods -n kubevirt; done
virtualmachine.kubevirt.io/vm-lucy created
NAME                               READY   STATUS    RESTARTS   AGE
virt-api-787db99cd9-psgjr          1/1     Running   0          20h
virt-api-787db99cd9-xqpf8          1/1     Running   0          20h
virt-controller-6d46f99d88-fv5qf   1/1     Running   6          20h
virt-controller-6d46f99d88-jwrv5   1/1     Running   4          20h
virt-handler-hm688                 1/1     Running   0          20h
virt-handler-nrc2g                 1/1     Running   0          20h
virt-handler-wrrvm                 1/1     Running   0          20h
virt-operator-bd7d6c76b-k7fdr      1/1     Running   6          20h
virt-operator-bd7d6c76b-pvggc      1/1     Running   6          20h
NAME                               READY   STATUS    RESTARTS   AGE
virt-api-787db99cd9-psgjr          1/1     Running   0          20h
virt-api-787db99cd9-xqpf8          1/1     Running   0          20h
virt-controller-6d46f99d88-fv5qf   1/1     Running   6          20h
virt-controller-6d46f99d88-jwrv5   1/1     Running   4          20h
virt-handler-hm688                 1/1     Running   0          20h
virt-handler-nrc2g                 1/1     Running   0          20h
virt-handler-wrrvm                 1/1     Running   0          20h
virt-operator-bd7d6c76b-k7fdr      1/1     Running   6          20h
virt-operator-bd7d6c76b-pvggc      1/1     Running   6          20h
NAME                               READY   STATUS    RESTARTS   AGE
virt-api-787db99cd9-psgjr          1/1     Running   0          20h
virt-api-787db99cd9-xqpf8          1/1     Running   0          20h
virt-controller-6d46f99d88-fv5qf   1/1     Running   6          20h
virt-controller-6d46f99d88-jwrv5   1/1     Running   4          20h
virt-handler-hm688                 1/1     Running   0          20h
virt-handler-nrc2g                 1/1     Running   0          20h
virt-handler-wrrvm                 1/1     Running   0          20h
virt-operator-bd7d6c76b-k7fdr      1/1     Running   6          20h
virt-operator-bd7d6c76b-pvggc      1/1     Running   6          20h
NAME                               READY   STATUS    RESTARTS   AGE
virt-api-787db99cd9-psgjr          1/1     Running   0          20h
virt-api-787db99cd9-xqpf8          1/1     Running   0          20h
virt-controller-6d46f99d88-fv5qf   1/1     Running   6          20h
virt-controller-6d46f99d88-jwrv5   1/1     Running   4          20h
virt-handler-hm688                 1/1     Running   0          20h
virt-handler-nrc2g                 1/1     Running   0          20h
virt-handler-wrrvm                 1/1     Running   0          20h
virt-operator-bd7d6c76b-k7fdr      1/1     Running   6          20h
virt-operator-bd7d6c76b-pvggc      1/1     Running   6          20h
NAME                               READY   STATUS    RESTARTS   AGE
virt-api-787db99cd9-psgjr          1/1     Running   0          20h
virt-api-787db99cd9-xqpf8          1/1     Running   0          20h
virt-controller-6d46f99d88-fv5qf   1/1     Running   6          20h
virt-controller-6d46f99d88-jwrv5   1/1     Running   4          20h
virt-handler-hm688                 1/1     Running   0          20h
virt-handler-nrc2g                 1/1     Running   0          20h
virt-handler-wrrvm                 1/1     Running   0          20h
virt-operator-bd7d6c76b-k7fdr      1/1     Running   6          20h
virt-operator-bd7d6c76b-pvggc      1/1     Running   6          20h

As for the logging,
kubectl logs  -n kubevirt virt-controller-6d46f99d88-fv5qf
{"component":"virt-controller","level":"info","msg":"STARTING informer configMapInformer","pos":"virtinformers.go:203","timestamp":"2019-12-13T07:10:34.540551Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer CRDInformer","pos":"virtinformers.go:203","timestamp":"2019-12-13T07:10:34.540753Z"}
{"component":"virt-controller","level":"info","msg":"CDI detected, DataVolume integration enabled","pos":"application.go:211","timestamp":"2019-12-13T07:10:34.641862Z"}
{"action":"listening","component":"virt-controller","interface":"<IP_ADDRESS>","level":"info","port":8443,"pos":"application.go:264","service":"http","timestamp":"2019-12-13T07:10:35.137291Z"}
E1213 12:20:58.496676       1 leaderelection.go:270] error retrieving resource lock kubevirt/virt-controller: the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints virt-controller)
E1213 12:21:08.231051       1 leaderelection.go:270] error retrieving resource lock kubevirt/virt-controller: rpc error: code = Unavailable desc = transport is closing
E1213 12:23:34.677691       1 leaderelection.go:270] error retrieving resource lock kubevirt/virt-controller: the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints virt-controller)

 kubectl logs  -n kubevirt virt-controller-6d46f99d88-jwrv5
{"component":"virt-controller","level":"info","msg":"STARTING informer configMapInformer","pos":"virtinformers.go:203","timestamp":"2019-12-13T12:22:49.300070Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer CRDInformer","pos":"virtinformers.go:203","timestamp":"2019-12-13T12:22:49.300240Z"}
E1213 12:22:59.302549       1 reflector.go:134] pkg/controller/virtinformers.go:204: Failed to list *v1.ConfigMap: Get https://<IP_ADDRESS>:443/api/v1/namespaces/kubevirt/configmaps?fieldSelector=metadata.name%3Dkubevirt-config&limit=500&resourceVersion=0: net/http: TLS handshake timeout
{"component":"virt-controller","level":"info","msg":"CDI detected, DataVolume integration enabled","pos":"application.go:211","timestamp":"2019-12-13T12:23:00.401350Z"}
{"action":"listening","component":"virt-controller","interface":"<IP_ADDRESS>","level":"info","port":8443,"pos":"application.go:264","service":"http","timestamp":"2019-12-13T12:23:00.929902Z"}
E1213 12:23:30.942389       1 leaderelection.go:304] Failed to update lock: Timeout: request did not complete within requested timeout 30s
{"component":"virt-controller","level":"info","msg":"SKIPPING informer configMapInformer","pos":"virtinformers.go:200","service":"http","timestamp":"2019-12-13T12:23:40.620229Z"}
{"component":"virt-controller","level":"info","msg":"SKIPPING informer CRDInformer","pos":"virtinformers.go:200","service":"http","timestamp":"2019-12-13T12:23:40.620421Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer persistentVolumeClaimInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-13T12:23:40.620494Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer vmInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-13T12:23:40.620601Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer vmimInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-13T12:23:40.620674Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer vmiInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-13T12:23:40.620744Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer kubeVirtPodInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-13T12:23:40.620823Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer kubeVirtNodeInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-13T12:23:40.620894Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer vmirsInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-13T12:23:40.620960Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer dataVolumeInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-13T12:23:40.621033Z"}
{"component":"virt-controller","level":"info","msg":"STARTING controllers with following threads : node 3, vmi 3, replicaset 3, vm 3, migration 3, evacuation 3, disruptionBudget 3\n","pos":"application.go:300","timestamp":"2019-12-13T12:23:40.621480Z"}
{"component":"virt-controller","level":"info","msg":"Starting disruption budget controller.","pos":"disruptionbudget.go:252","service":"http","timestamp":"2019-12-13T12:23:40.621708Z"}
{"component":"virt-controller","level":"info","msg":"Starting node controller.","pos":"node.go:109","service":"http","timestamp":"2019-12-13T12:23:40.621603Z"}
{"component":"virt-controller","level":"info","msg":"Starting VirtualMachineInstanceReplicaSet controller.","pos":"replicaset.go:106","service":"http","timestamp":"2019-12-13T12:23:40.621870Z"}
{"component":"virt-controller","level":"info","msg":"Starting evacuation controller.","pos":"evacuation.go:251","service":"http","timestamp":"2019-12-13T12:23:40.626160Z"}
{"component":"virt-controller","level":"info","msg":"Starting migration controller.","pos":"migration.go:108","service":"http","timestamp":"2019-12-13T12:23:40.622034Z"}
{"component":"virt-controller","level":"info","msg":"Starting vmi controller.","pos":"vmi.go:173","service":"http","timestamp":"2019-12-13T12:23:40.621969Z"}
{"component":"virt-controller","level":"info","msg":"Starting VirtualMachine controller.","pos":"vm.go:117","service":"http","timestamp":"2019-12-13T12:23:40.626185Z"}
{"component":"virt-controller","level":"info","msg":"Updating cluster config to resource version '445065'","pos":"config-map.go:435","service":"http","timestamp":"2019-12-13T12:23:40.726734Z"}

Looks like you might be onto something here, any idea what this could mean?

Hm, that we see so many restarts because of  failed http readiness probes and that we see here errors where we seem to wait for 30 seconds to get a configmap, my suspicion would be that the controller establishes a watch, but there is no data or almost no data transferred.
Did you setup this cluster just to try kubevirt, or did you also run other things there? It looks a little like  there are huge networking issues.

I set this cluster up as general purpose. I have not seen an network issues wiht any of my other depoyments, so I am not quite sure if that is what it is but am welcome to any ideas. I am using flannel as my overlay.
I could remove kubevirt from the cluster and try again?

Hm, it might be related to talking to the apiserver only. Could you have a look at the apiserver logs in the kubernetes namespace? Maybe we see something like rate limiting or reports of connection issues.

just a hint: events are deleted after some time.
can you do the kubectl describe on a freshly created VM? Seeing not a single event on it is strange.

kubectl describe vm -n vm-lucy vm-lucy
Name:         vm-lucy
Namespace:    vm-lucy
Labels:       kubevirt.io/vm=vm-lucy
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"kubevirt.io/v1alpha3","kind":"VirtualMachine","metadata":{"annotations":{},"labels":{"kubevirt.io/vm":"vm-lucy"},"name":"vm...
              kubevirt.io/latest-observed-api-version: v1alpha3
              kubevirt.io/storage-observed-api-version: v1alpha3
API Version:  kubevirt.io/v1alpha3
Kind:         VirtualMachine
Metadata:
  Creation Timestamp:  2019-12-14T12:05:52Z
  Generation:          3
  Resource Version:    2273394
  Self Link:           /apis/kubevirt.io/v1alpha3/namespaces/vm-lucy/virtualmachines/vm-lucy
  UID:                 354e797f-6911-4ea8-b031-ecda242bd05b
Spec:
  Running:  true
  Template:
    Metadata:
      Creation Timestamp:  <nil>
      Labels:
        Kubevirt . Io / Vm:  vm-lucy
      Name:                  vm-lucy
      Namespace:             vm-lucy
    Spec:
      Domain:
        Cpu:
          Cores:  4
        Devices:
          Disks:
            Disk:
              Bus:  virtio
            Name:   vdisk-lucy
            Disk:
              Bus:  virtio
            Name:   cloudinit
        Machine:
          Type:  q35
        Resources:
          Requests:
            Memory:  2G
      Volumes:
        Data Volume:
          Name:  dv-lucy
        Name:    vdisk-lucy
        Cloud Init No Cloud:
          User Data:  #cloud-config
password: changeme
chpasswd: { expire: False }

        Name:  cloudinit
Status:
  Created:  true
Events:
  Type    Reason            Age    From                       Message
  ----    ------            ----   ----                       -------
  Normal  SuccessfulCreate  2m21s  virtualmachine-controller  Started the virtual machine by creating the new virtual machine instance vm-lucy

kubectl describe vmi -n vm-lucy vm-lucy
Name:         vm-lucy
Namespace:    vm-lucy
Labels:       kubevirt.io/vm=vm-lucy
Annotations:  kubevirt.io/latest-observed-api-version: v1alpha3
              kubevirt.io/storage-observed-api-version: v1alpha3
API Version:  kubevirt.io/v1alpha3
Kind:         VirtualMachineInstance
Metadata:
  Creation Timestamp:  2019-12-14T12:05:52Z
  Finalizers:
    foregroundDeleteVirtualMachine
  Generate Name:  vm-lucy
  Generation:     2
  Owner References:
    API Version:           kubevirt.io/v1alpha3
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  VirtualMachine
    Name:                  vm-lucy
    UID:                   354e797f-6911-4ea8-b031-ecda242bd05b
  Resource Version:        2273395
  Self Link:               /apis/kubevirt.io/v1alpha3/namespaces/vm-lucy/virtualmachineinstances/vm-lucy
  UID:                     0a341af3-79b4-436d-b78b-56d1f593a0df
Spec:
  Domain:
    Cpu:
      Cores:  4
    Devices:
      Disks:
        Disk:
          Bus:  virtio
        Name:   vdisk-lucy
        Disk:
          Bus:  virtio
        Name:   cloudinit
      Interfaces:
        Bridge:
        Name:  default
    Features:
      Acpi:
        Enabled:  true
    Firmware:
      Uuid:  a4531ca3-316b-57e5-b6a3-9e8c5789a400
    Machine:
      Type:  q35
    Resources:
      Requests:
        Cpu:     100m
        Memory:  2G
  Networks:
    Name:  default
    Pod:
  Volumes:
    Data Volume:
      Name:  dv-lucy
    Name:    vdisk-lucy
    Cloud Init No Cloud:
      User Data:  #cloud-config
password: changeme
chpasswd: { expire: False }

    Name:  cloudinit
Status:
  Guest OS Info:
  Phase:  Pending
Events:   <none>

@slintes That is on a newly created VM/VMI.

@rmohr  I think you are on to something:
I1214 12:07:35.223338       1 trace.go:116] Trace[607307897]: "List" url:/api/v1/secrets (started: 2019-12-14 12:07:29.381126109 +0000 UTC m=+325733.877390913) (total time: 5.84214556s):
Trace[607307897]: [5.842143178s] [5.839407365s] Writing http response done count:1764
I1214 12:07:59.214174       1 controller.go:107] OpenAPI AggregationController: Processing item v1alpha1.upload.cdi.kubevirt.io
E1214 12:07:59.237138       1 controller.go:114] loading OpenAPI spec for "v1alpha1.upload.cdi.kubevirt.io" failed with: OpenAPI spec does not exist
I1214 12:07:59.237185       1 controller.go:127] OpenAPI AggregationController: action for item v1alpha1.upload.cdi.kubevirt.io: Rate Limited Requeue.
I1214 12:08:02.775864       1 controller.go:107] OpenAPI AggregationController: Processing item v1alpha3.subresources.kubevirt.io
I1214 12:08:02.799432       1 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io
I1214 12:08:35.938407       1 trace.go:116] Trace[568315241]: "List" url:/api/v1/secrets (started: 2019-12-14 12:08:30.116451484 +0000 UTC m=+325794.612716272) (total time: 5.821880784s):
Trace[568315241]: [5.82187693s] [5.81934771s] Writing http response done count:1764
I1214 12:08:59.237436       1 controller.go:107] OpenAPI AggregationController: Processing item v1alpha1.upload.cdi.kubevirt.io
E1214 12:08:59.239363       1 controller.go:114] loading OpenAPI spec for "v1alpha1.upload.cdi.kubevirt.io" failed with: OpenAPI spec does not exist
I1214 12:08:59.239441       1 controller.go:127] OpenAPI AggregationController: action for item v1alpha1.upload.cdi.kubevirt.io: Rate Limited Requeue.

I1214 12:11:02.841605       1 controller.go:107] OpenAPI AggregationController: Processing item v1alpha3.subresources.kubevirt.io
I1214 12:11:02.862974       1 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io
I1214 12:11:07.608870       1 trace.go:116] Trace[1997741614]: "List" url:/api/v1/secrets (started: 2019-12-14 12:11:01.884839482 +0000 UTC m=+325946.381104314) (total time: 5.723968005s):
Trace[1997741614]: [5.723966091s] [5.721815641s] Writing http response done count:1764
I1214 12:12:02.863147       1 controller.go:107] OpenAPI AggregationController: Processing item v1alpha3.subresources.kubevirt.io
I1214 12:12:02.884689       1 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io
I1214 12:12:38.646602       1 trace.go:116] Trace[637636812]: "List" url:/api/v1/secrets (started: 2019-12-14 12:12:32.901947501 +0000 UTC m=+326037.398212293) (total time: 5.744599231s):
Trace[637636812]: [5.744596109s] [5.742407152s] Writing http response done count:1764

This being what stands out most to me:
I1214 12:07:59.214174       1 controller.go:107] OpenAPI AggregationController: Processing item v1alpha1.upload.cdi.kubevirt.io
E1214 12:07:59.237138       1 controller.go:114] loading OpenAPI spec for "v1alpha1.upload.cdi.kubevirt.io" failed with: OpenAPI spec does not exist
I1214 12:07:59.237185       1 controller.go:127] OpenAPI AggregationController: action for item v1alpha1.upload.cdi.kubevirt.io: Rate Limited Requeue.

@aneurinprice hm, that looks pretty ok. Could you also check if virt-controller has a higher restart count when you create a VMI? If yes, could you look up the previous logs (-p) from virt-handler? They might show something which is helpful.

@rmohr
mac-n-cheesenyepricelocal:lucy moby$ kubectl get vmi -n vm-lucy
NAME      AGE     PHASE     IP    NODENAME
vm-lucy   4m33s   Pending
mac-n-cheesenyepricelocal:lucy moby$ kubectl get pods -n kubevirt
NAME                               READY   STATUS    RESTARTS   AGE
virt-api-787db99cd9-2wqsb          1/1     Running   0          7m15s
virt-api-787db99cd9-wlxtn          1/1     Running   0          7m16s
virt-controller-6d46f99d88-dpg72   1/1     Running   0          7m15s
virt-controller-6d46f99d88-g97rl   1/1     Running   0          7m15s
virt-handler-b92jn                 1/1     Running   0          7m11s
virt-handler-gbrth                 1/1     Running   0          7m12s
virt-handler-n26fh                 1/1     Running   0          7m10s
virt-operator-bd7d6c76b-5hn8d      1/1     Running   0          7m15s
virt-operator-bd7d6c76b-fxs52      1/1     Running   0          7m15s

No restarts so far. I do belive it to be an API/ETCD problem now, although I am not sure what exactly. Etcd has lots of space and is on SSD's... hmmmm

Hm, I run out of ideas. I haven't seen anything like this before. What k8s version is used on the server side? Maybe I missed it but I only saw the client version in the report.

@rmohr The server version is: Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.3"
I am pretty confused at this point, I don't THINK I've done anything wrong :(

I am pretty confused at this point, I don't THINK I've done anything wrong :(

Hm, what I can tell you is that we test on k8s 1.16.2, but not on 1.16.3. There is still the chance that something changed in the minor release which is causing the issue.

I'll rollback to .2 now and let you know how it goes

@rmohr I have reverted to .2 and there is no change. Any other ideas? I am grateful for your time on this.

So, it doesnt look like the virt-launcher-* pod gets created.
kubectl apply -f lucy-vm.yaml; for i in {1..5}; do kubectl get pods -n kubevirt; done
virtualmachine.kubevirt.io/vm-lucy created

the virt-launcher pod would be in the same namespace as the VM, not in the kubevirt ns.

the virt-launcher pod would be in the same namespace as the VM, not in the kubevirt ns.

Yep. Missed that it was the kubevirt namespace. Good hint. :+1:

@slintes
mac-n-cheesenyepricelocal:lucy moby$ kubectl apply -f lucy-vm.yaml; for i in {1..25}; do kubectl get pods -n vm-lucy; done
virtualmachine.kubevirt.io/vm-lucy created
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.
No resources found.

Thansk for the hint, unfortunately the virt-launcher pod still isnt being created.

too bad... I am out of ideas as well now :thinking:

Maybe you can share once more from both virt-controller instances the logs?

Certainly.
mac-n-cheesenyepricelocal:lucy moby$ kubectl logs -n kubevirt virt-controller-6d46f99d88-cwnhz
{"component":"virt-controller","level":"info","msg":"STARTING informer configMapInformer","pos":"virtinformers.go:203","timestamp":"2019-12-17T13:05:16.227001Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer CRDInformer","pos":"virtinformers.go:203","timestamp":"2019-12-17T13:05:16.227133Z"}
{"component":"virt-controller","level":"info","msg":"CDI detected, DataVolume integration enabled","pos":"application.go:211","timestamp":"2019-12-17T13:05:16.329469Z"}
{"action":"listening","component":"virt-controller","interface":"<IP_ADDRESS>","level":"info","port":8443,"pos":"application.go:264","service":"http","timestamp":"2019-12-17T13:05:17.073896Z"}

mac-n-cheesenyepricelocal:lucy moby$ kubectl logs -n kubevirt virt-controller-6d46f99d88-q29wg
{"component":"virt-controller","level":"info","msg":"STARTING informer configMapInformer","pos":"virtinformers.go:203","timestamp":"2019-12-17T13:05:16.159921Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer CRDInformer","pos":"virtinformers.go:203","timestamp":"2019-12-17T13:05:16.160033Z"}
{"component":"virt-controller","level":"info","msg":"CDI detected, DataVolume integration enabled","pos":"application.go:211","timestamp":"2019-12-17T13:05:16.260798Z"}
{"action":"listening","component":"virt-controller","interface":"<IP_ADDRESS>","level":"info","port":8443,"pos":"application.go:264","service":"http","timestamp":"2019-12-17T13:05:16.910108Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer kubeVirtPodInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T13:05:33.429795Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer kubeVirtNodeInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T13:05:33.429861Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer vmirsInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T13:05:33.429890Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer persistentVolumeClaimInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T13:05:33.429914Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer vmInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T13:05:33.429939Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer vmimInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T13:05:33.429964Z"}
{"component":"virt-controller","level":"info","msg":"SKIPPING informer configMapInformer","pos":"virtinformers.go:200","service":"http","timestamp":"2019-12-17T13:05:33.429999Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer vmiInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T13:05:33.430047Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer dataVolumeInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T13:05:33.430107Z"}
{"component":"virt-controller","level":"info","msg":"SKIPPING informer CRDInformer","pos":"virtinformers.go:200","service":"http","timestamp":"2019-12-17T13:05:33.430157Z"}
{"component":"virt-controller","level":"info","msg":"STARTING controllers with following threads : node 3, vmi 3, replicaset 3, vm 3, migration 3, evacuation 3, disruptionBudget 3\n","pos":"application.go:300","timestamp":"2019-12-17T13:05:33.430289Z"}
{"component":"virt-controller","level":"info","msg":"Starting VirtualMachineInstanceReplicaSet controller.","pos":"replicaset.go:106","service":"http","timestamp":"2019-12-17T13:05:33.430509Z"}
{"component":"virt-controller","level":"info","msg":"Starting disruption budget controller.","pos":"disruptionbudget.go:252","service":"http","timestamp":"2019-12-17T13:05:33.430734Z"}
{"component":"virt-controller","level":"info","msg":"Starting migration controller.","pos":"migration.go:108","service":"http","timestamp":"2019-12-17T13:05:33.430609Z"}
{"component":"virt-controller","level":"info","msg":"Starting evacuation controller.","pos":"evacuation.go:251","service":"http","timestamp":"2019-12-17T13:05:33.430781Z"}
{"component":"virt-controller","level":"info","msg":"Starting node controller.","pos":"node.go:109","service":"http","timestamp":"2019-12-17T13:05:33.430935Z"}
{"component":"virt-controller","level":"info","msg":"Starting VirtualMachine controller.","pos":"vm.go:117","service":"http","timestamp":"2019-12-17T13:05:33.430634Z"}
{"component":"virt-controller","level":"info","msg":"Starting vmi controller.","pos":"vmi.go:173","service":"http","timestamp":"2019-12-17T13:05:33.431040Z"}
{"component":"virt-controller","level":"info","msg":"Updating cluster config to resource version '445065'","pos":"config-map.go:435","service":"http","timestamp":"2019-12-17T13:05:33.531213Z"}

One thing that might make a difference, I am running Kubernetes in containers with RKE. I don't see why it would cause problems, but it is worth noting

ne thing that might make a difference, I am running Kubernetes in containers with RKE. I don't see why it would cause problems, but it is worth noting

That can be the reason, however, we see that all our CRDs are registered (you can create VMs and VMIs). We see that at least some of our controller loops in virt-controller react (the VM controller creates the VMI). We should then at least see that the VMI controller tries to create a pod, or fails somehow along the way. But just nothing.
Can you do the following: Edit the virt-controller deployment with kubectl edit deployments -n kubevirt virt-controller and increase the -v value from 2 to at least 9?

That is better
mac-n-cheesenyepricelocal:lucy moby$ kubectl logs -n kubevirt virt-controller-5455cb7fc5-7dd7g
{"component":"virt-controller","level":"info","msg":"STARTING informer configMapInformer","pos":"virtinformers.go:203","timestamp":"2019-12-17T15:46:33.899063Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer CRDInformer","pos":"virtinformers.go:203","timestamp":"2019-12-17T15:46:33.899174Z"}
{"component":"virt-controller","level":"info","msg":"CDI detected, DataVolume integration enabled","pos":"application.go:211","timestamp":"2019-12-17T15:46:33.999790Z"}
{"action":"listening","component":"virt-controller","interface":"<IP_ADDRESS>","level":"info","port":8443,"pos":"application.go:264","service":"http","timestamp":"2019-12-17T15:46:34.800072Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer kubeVirtNodeInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T15:46:51.345230Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer vmirsInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T15:46:51.345376Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer vmimInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T15:46:51.345429Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer dataVolumeInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T15:46:51.345488Z"}
{"component":"virt-controller","level":"info","msg":"SKIPPING informer configMapInformer","pos":"virtinformers.go:200","service":"http","timestamp":"2019-12-17T15:46:51.345553Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer kubeVirtPodInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T15:46:51.345633Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer persistentVolumeClaimInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T15:46:51.345703Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer vmInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T15:46:51.345771Z"}
{"component":"virt-controller","level":"info","msg":"SKIPPING informer CRDInformer","pos":"virtinformers.go:200","service":"http","timestamp":"2019-12-17T15:46:51.345849Z"}
{"component":"virt-controller","level":"info","msg":"STARTING informer vmiInformer","pos":"virtinformers.go:203","service":"http","timestamp":"2019-12-17T15:46:51.345931Z"}
{"component":"virt-controller","level":"info","msg":"STARTING controllers with following threads : node 3, vmi 3, replicaset 3, vm 3, migration 3, evacuation 3, disruptionBudget 3\n","pos":"application.go:300","timestamp":"2019-12-17T15:46:51.346051Z"}
{"component":"virt-controller","level":"info","msg":"Starting VirtualMachine controller.","pos":"vm.go:117","service":"http","timestamp":"2019-12-17T15:46:51.346232Z"}
{"component":"virt-controller","level":"info","msg":"Starting node controller.","pos":"node.go:109","service":"http","timestamp":"2019-12-17T15:46:51.346266Z"}
{"component":"virt-controller","level":"info","msg":"Starting VirtualMachineInstanceReplicaSet controller.","pos":"replicaset.go:106","service":"http","timestamp":"2019-12-17T15:46:51.346206Z"}
{"component":"virt-controller","level":"info","msg":"Starting vmi controller.","pos":"vmi.go:173","service":"http","timestamp":"2019-12-17T15:46:51.346486Z"}
{"component":"virt-controller","level":"info","msg":"Starting evacuation controller.","pos":"evacuation.go:251","service":"http","timestamp":"2019-12-17T15:46:51.346428Z"}
{"component":"virt-controller","level":"info","msg":"Starting disruption budget controller.","pos":"disruptionbudget.go:252","service":"http","timestamp":"2019-12-17T15:46:51.346172Z"}
{"component":"virt-controller","level":"info","msg":"Starting migration controller.","pos":"migration.go:108","service":"http","timestamp":"2019-12-17T15:46:51.346804Z"}
{"component":"virt-controller","level":"info","msg":"processed node susan","pos":"node.go:143","service":"http","timestamp":"2019-12-17T15:46:51.446679Z"}
{"component":"virt-controller","level":"info","msg":"processed node esther","pos":"node.go:143","service":"http","timestamp":"2019-12-17T15:46:51.446683Z"}
{"component":"virt-controller","level":"info","msg":"processed node margo","pos":"node.go:143","service":"http","timestamp":"2019-12-17T15:46:51.446752Z"}
{"component":"virt-controller","level":"info","msg":"Controller margo (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:46:51.646888Z"}
{"component":"virt-controller","level":"info","msg":"Controller susan (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:46:51.646876Z"}
{"component":"virt-controller","level":"info","msg":"Controller esther (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:46:51.646884Z"}
{"component":"virt-controller","level":"info","msg":"Updating cluster config to resource version '445065'","pos":"config-map.go:435","service":"http","timestamp":"2019-12-17T15:46:51.647097Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance margo","pos":"evacuation.go:282","service":"http","timestamp":"2019-12-17T15:46:51.647189Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance susan","pos":"evacuation.go:282","service":"http","timestamp":"2019-12-17T15:46:51.647315Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance esther","pos":"evacuation.go:282","service":"http","timestamp":"2019-12-17T15:46:51.647394Z"}
{"component":"virt-controller","level":"info","msg":"Controller margo (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:46:52.282893Z"}
{"component":"virt-controller","level":"info","msg":"processed node margo","pos":"node.go:143","service":"http","timestamp":"2019-12-17T15:46:52.282996Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance margo","pos":"evacuation.go:282","service":"http","timestamp":"2019-12-17T15:46:52.283020Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"Started processing vm","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:167","service":"http","timestamp":"2019-12-17T15:47:07.102840Z","uid":"b905f982-7e0e-44f1-a755-e57a84a41efb"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachine vm-lucy/vm-lucy","pos":"vm.go:146","service":"http","timestamp":"2019-12-17T15:47:07.132716Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"Started processing vm","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:167","service":"http","timestamp":"2019-12-17T15:47:07.132790Z","uid":"b905f982-7e0e-44f1-a755-e57a84a41efb"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:07.132832Z"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:07.132864Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"VirtualMachineInstance not found in cache vm-lucy/vm-lucy","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:215","service":"http","timestamp":"2019-12-17T15:47:07.132892Z","uid":"b905f982-7e0e-44f1-a755-e57a84a41efb"}
{"component":"virt-controller","kind":"","level":"info","msg":"VirtualMachine RunStrategy: Always","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:492","service":"http","timestamp":"2019-12-17T15:47:07.132949Z","uid":"b905f982-7e0e-44f1-a755-e57a84a41efb"}
{"component":"virt-controller","kind":"","level":"info","msg":"Starting VMI","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:526","service":"http","timestamp":"2019-12-17T15:47:07.132983Z","uid":"b905f982-7e0e-44f1-a755-e57a84a41efb"}
{"component":"virt-controller","level":"info","msg":"Setting expectations \u0026controller.ControlleeExpectations{add:1, del:0, key:\"vm-lucy/vm-lucy\", timestamp:time.Time{wall:0xbf765bfec7ee163d, ext:33285780025, loc:(*time.Location)(0x2733920)}}","pos":"expectations.go:181","timestamp":"2019-12-17T15:47:07.133044Z"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:07.161354Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance vm-lucy/vm-lucy","pos":"disruptionbudget.go:283","service":"http","timestamp":"2019-12-17T15:47:07.161471Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"VirtualMachineInstance added.","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:805","service":"http","timestamp":"2019-12-17T15:47:07.161396Z","uid":"29bfbfdb-4b08-4e4e-94b6-0cd2cec214c0"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachine vm-lucy/vm-lucy","pos":"vm.go:146","service":"http","timestamp":"2019-12-17T15:47:07.161479Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"Looking for VirtualMachineInstance Ref","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:816","service":"http","timestamp":"2019-12-17T15:47:07.161514Z","uid":"29bfbfdb-4b08-4e4e-94b6-0cd2cec214c0"}
{"component":"virt-controller","kind":"","level":"info","msg":"VirtualMachineInstance created because vm-lucy was added.","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:828","service":"http","timestamp":"2019-12-17T15:47:07.161561Z","uid":"29bfbfdb-4b08-4e4e-94b6-0cd2cec214c0"}
{"component":"virt-controller","level":"info","msg":"Lowered expectations \u0026controller.ControlleeExpectations{add:0, del:0, key:\"vm-lucy/vm-lucy\", timestamp:time.Time{wall:0xbf765bfec7ee163d, ext:33285780025, loc:(*time.Location)(0x2733920)}}","pos":"expectations.go:198","timestamp":"2019-12-17T15:47:07.161602Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"Started processing vm","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:167","service":"http","timestamp":"2019-12-17T15:47:07.161672Z","uid":"b905f982-7e0e-44f1-a755-e57a84a41efb"}
{"component":"virt-controller","level":"info","msg":"Controller expectations (name: n/a) fulfilled \u0026controller.ControlleeExpectations{add:0, del:0, key:\"vm-lucy/vm-lucy\", timestamp:time.Time{wall:0xbf765bfec7ee163d, ext:33285780025, loc:(*time.Location)(0x2733920)}}","pos":"expectations.go:147","timestamp":"2019-12-17T15:47:07.161724Z"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:07.161764Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"VirtualMachine RunStrategy: Always","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:492","service":"http","timestamp":"2019-12-17T15:47:07.161865Z","uid":"b905f982-7e0e-44f1-a755-e57a84a41efb"}
{"component":"virt-controller","kind":"","level":"info","msg":"VirtualMachineInstance updated","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:891","service":"http","timestamp":"2019-12-17T15:47:07.174681Z","uid":"29bfbfdb-4b08-4e4e-94b6-0cd2cec214c0"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:07.174701Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance vm-lucy/vm-lucy","pos":"disruptionbudget.go:283","service":"http","timestamp":"2019-12-17T15:47:07.174854Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance vm-lucy/vm-lucy","pos":"vmi.go:204","service":"http","timestamp":"2019-12-17T15:47:07.175460Z"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:07.175536Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"DataVolume dv-lucy not found","name":"vm-lucy","namespace":"vm-lucy","pos":"vmi.go:563","service":"http","timestamp":"2019-12-17T15:47:07.175582Z","uid":"29bfbfdb-4b08-4e4e-94b6-0cd2cec214c0"}
{"component":"virt-controller","kind":"","level":"info","msg":"Delaying pod creation while DataVolume populates","name":"vm-lucy","namespace":"vm-lucy","pos":"vmi.go:512","service":"http","timestamp":"2019-12-17T15:47:07.175639Z","uid":"29bfbfdb-4b08-4e4e-94b6-0cd2cec214c0"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachine vm-lucy/vm-lucy","pos":"vm.go:146","service":"http","timestamp":"2019-12-17T15:47:07.181873Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"Started processing vm","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:167","service":"http","timestamp":"2019-12-17T15:47:07.181996Z","uid":"b905f982-7e0e-44f1-a755-e57a84a41efb"}
{"component":"virt-controller","level":"info","msg":"Controller expectations (name: n/a) fulfilled \u0026controller.ControlleeExpectations{add:0, del:0, key:\"vm-lucy/vm-lucy\", timestamp:time.Time{wall:0xbf765bfec7ee163d, ext:33285780025, loc:(*time.Location)(0x2733920)}}","pos":"expectations.go:147","timestamp":"2019-12-17T15:47:07.182061Z"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:07.182143Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"VirtualMachine RunStrategy: Always","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:492","service":"http","timestamp":"2019-12-17T15:47:07.182210Z","uid":"b905f982-7e0e-44f1-a755-e57a84a41efb"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachine vm-lucy/vm-lucy","pos":"vm.go:146","service":"http","timestamp":"2019-12-17T15:47:07.182317Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"VirtualMachineInstance updated","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:891","service":"http","timestamp":"2019-12-17T15:47:07.186351Z","uid":"29bfbfdb-4b08-4e4e-94b6-0cd2cec214c0"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:07.186290Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"Started processing vm","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:167","service":"http","timestamp":"2019-12-17T15:47:07.186491Z","uid":"b905f982-7e0e-44f1-a755-e57a84a41efb"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance vm-lucy/vm-lucy","pos":"disruptionbudget.go:283","service":"http","timestamp":"2019-12-17T15:47:07.186557Z"}
{"component":"virt-controller","level":"info","msg":"Controller expectations (name: n/a) fulfilled \u0026controller.ControlleeExpectations{add:0, del:0, key:\"vm-lucy/vm-lucy\", timestamp:time.Time{wall:0xbf765bfec7ee163d, ext:33285780025, loc:(*time.Location)(0x2733920)}}","pos":"expectations.go:147","timestamp":"2019-12-17T15:47:07.186566Z"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:07.186650Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"VirtualMachine RunStrategy: Always","name":"vm-lucy","namespace":"vm-lucy","pos":"vm.go:492","service":"http","timestamp":"2019-12-17T15:47:07.186741Z","uid":"b905f982-7e0e-44f1-a755-e57a84a41efb"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachine vm-lucy/vm-lucy","pos":"vm.go:146","service":"http","timestamp":"2019-12-17T15:47:07.186871Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance vm-lucy/vm-lucy","pos":"vmi.go:204","service":"http","timestamp":"2019-12-17T15:47:07.186945Z"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:07.187042Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"DataVolume dv-lucy not found","name":"vm-lucy","namespace":"vm-lucy","pos":"vmi.go:563","service":"http","timestamp":"2019-12-17T15:47:07.187103Z","uid":"29bfbfdb-4b08-4e4e-94b6-0cd2cec214c0"}
{"component":"virt-controller","kind":"","level":"info","msg":"Delaying pod creation while DataVolume populates","name":"vm-lucy","namespace":"vm-lucy","pos":"vmi.go:512","service":"http","timestamp":"2019-12-17T15:47:07.187172Z","uid":"29bfbfdb-4b08-4e4e-94b6-0cd2cec214c0"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance vm-lucy/vm-lucy","pos":"vmi.go:204","service":"http","timestamp":"2019-12-17T15:47:07.187294Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachine vm-lucy/vm-lucy","pos":"vm.go:146","service":"http","timestamp":"2019-12-17T15:47:40.320726Z"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:41.346307Z"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:41.346260Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance vm-lucy/vm-lucy","pos":"disruptionbudget.go:283","service":"http","timestamp":"2019-12-17T15:47:41.346524Z"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:41.360298Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance vm-lucy/vm-lucy","pos":"disruptionbudget.go:283","service":"http","timestamp":"2019-12-17T15:47:41.360396Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance vm-lucy/vm-lucy","pos":"vmi.go:204","service":"http","timestamp":"2019-12-17T15:47:41.360466Z"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:41.360548Z"}
{"component":"virt-controller","kind":"","level":"info","msg":"All pods have been deleted, removing finalizer","name":"vm-lucy","namespace":"vm-lucy","pos":"vmi.go:367","service":"http","timestamp":"2019-12-17T15:47:41.360615Z","uid":"29bfbfdb-4b08-4e4e-94b6-0cd2cec214c0"}
{"component":"virt-controller","level":"info","msg":"Controller vm-lucy/vm-lucy (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:41.383568Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance vm-lucy/vm-lucy","pos":"disruptionbudget.go:283","service":"http","timestamp":"2019-12-17T15:47:41.383682Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance vm-lucy/vm-lucy","pos":"vmi.go:204","service":"http","timestamp":"2019-12-17T15:47:41.383721Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance vm-lucy/vm-lucy","pos":"vmi.go:204","service":"http","timestamp":"2019-12-17T15:47:41.383778Z"}
{"component":"virt-controller","level":"info","msg":"Controller esther (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:45.416905Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance esther","pos":"evacuation.go:282","service":"http","timestamp":"2019-12-17T15:47:45.417014Z"}
{"component":"virt-controller","level":"info","msg":"processed node esther","pos":"node.go:143","service":"http","timestamp":"2019-12-17T15:47:45.417024Z"}
{"component":"virt-controller","level":"info","msg":"Controller susan (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:46.213322Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance susan","pos":"evacuation.go:282","service":"http","timestamp":"2019-12-17T15:47:46.213404Z"}
{"component":"virt-controller","level":"info","msg":"processed node susan","pos":"node.go:143","service":"http","timestamp":"2019-12-17T15:47:46.213392Z"}
{"component":"virt-controller","level":"info","msg":"Controller margo (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:46.711125Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance margo","pos":"evacuation.go:282","service":"http","timestamp":"2019-12-17T15:47:46.711262Z"}
{"component":"virt-controller","level":"info","msg":"processed node margo","pos":"node.go:143","service":"http","timestamp":"2019-12-17T15:47:46.711313Z"}
{"component":"virt-controller","level":"info","msg":"Controller esther (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:46.956054Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance esther","pos":"evacuation.go:282","service":"http","timestamp":"2019-12-17T15:47:46.956152Z"}
{"component":"virt-controller","level":"info","msg":"processed node esther","pos":"node.go:143","service":"http","timestamp":"2019-12-17T15:47:46.956140Z"}
{"component":"virt-controller","level":"info","msg":"processed node esther","pos":"node.go:143","service":"http","timestamp":"2019-12-17T15:47:51.446966Z"}
{"component":"virt-controller","level":"info","msg":"processed node susan","pos":"node.go:143","service":"http","timestamp":"2019-12-17T15:47:51.446984Z"}
{"component":"virt-controller","level":"info","msg":"processed node margo","pos":"node.go:143","service":"http","timestamp":"2019-12-17T15:47:51.446971Z"}
{"component":"virt-controller","level":"info","msg":"Controller susan (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:51.961837Z"}
{"component":"virt-controller","level":"info","msg":"processed node susan","pos":"node.go:143","service":"http","timestamp":"2019-12-17T15:47:51.961924Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance susan","pos":"evacuation.go:282","service":"http","timestamp":"2019-12-17T15:47:51.961966Z"}
{"component":"virt-controller","level":"info","msg":"Controller margo (name: n/a) either never recorded expectations, or the ttl expired.","pos":"expectations.go:164","timestamp":"2019-12-17T15:47:52.728530Z"}
{"component":"virt-controller","level":"info","msg":"processed VirtualMachineInstance margo","pos":"evacuation.go:282","service":"http","timestamp":"2019-12-17T15:47:52.728670Z"}
{"component":"virt-controller","level":"info","msg":"processed node margo","pos":"node.go:143","service":"http","timestamp":"2019-12-17T15:47:52.728667Z"}
mac-n-cheesenyepricelocal:lucy moby$

Ha:
{"component":"virt-controller","kind":"","level":"info","msg":"Delaying pod creation while DataVolume populates","name":"vm-lucy","namespace":"vm-lucy","pos":"vmi.go:512","service":"http","timestamp":"2019-12-17T15:47:07.187172Z","uid":"29bfbfdb-4b08-4e4e-94b6-0cd2cec214c0"}

CDI has an issue with the DataVolume. Can we verify that by creating a very simple VMI without a datavolume?
Maybe
apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachineInstance
metadata:
  labels:
    special: vmi-nocloud
  name: vmi-nocloud
spec:
  domain:
    devices:
      disks:
      - disk:
          bus: virtio
        name: containerdisk
      - disk:
          bus: virtio
        name: cloudinitdisk
      - disk:
          bus: virtio
        name: emptydisk
    machine:
      type: ""
    resources:
      requests:
        memory: 64M
  terminationGracePeriodSeconds: 0
  volumes:
  - containerDisk:
      image: kubevirt/cirros-container-disk-demo
    name: containerdisk
  - cloudInitNoCloud:
      userData: |
        #!/bin/sh

        echo 'printed from cloud-init userdata'
    name: cloudinitdisk
  - emptyDisk:
      capacity: 2Gi
    name: emptydisk

Looks like we are getting somewhere
{"component":"virt-controller","kind":"","level":"info","msg":"Delaying pod creation while DataVolume populates","name":"vm-lucy","namespace":"vm-lucy","pos":"vmi.go:512","service":"http","timestamp":"2019-12-17T15:47:07.175639Z","uid":"29bfbfdb-4b08-4e4e-94b6-0cd2cec214c0"}

vmi-nocloud   20s   Scheduling
mac-n-cheesenyepricelocal:vm moby$ kubectl get vmi
NAME          AGE   PHASE        IP    NODENAME
vmi-nocloud   22s   Scheduling

mac-n-cheesenyepricelocal:vm moby$ kubectl get pod
NAME                              READY   STATUS              RESTARTS   AGE
virt-launcher-vmi-nocloud-4v9gr   0/2     ContainerCreating   0          52s

mac-n-cheesenyepricelocal:vm moby$ kubectl get vmi
NAME          AGE   PHASE     IP           NODENAME
vmi-nocloud   87s   Running   <IP_ADDRESS>   susan

We've got it boys!!

Great. So I guess that we have to improve our conditions here to reflect that. Let's do that. Thanks for the patience and sharing the data. I think you have #2930 already regarding to the DataVolume issue?

Wonderful. Thank you VERY much for your help on this one. I will continue on the other issue. :D

Creasted #2938 as a follow up.

@rmohr I have got to the bottom of why my VM's were failing to start. I'm still figuring out exactly what causes the behaviour, but it is something to do with when the DV is created and deemed ready BEFORE the VM is happy enough to schedule the VMI. I will create a new issue and have a poke at getting that fixed. Thanks for your help on this one buddy :)


Test laundree app offline

When logged in, and the app is opened while being offline, the app hangs (android).

When logged out and the app is opened, no issue until trying to log in.

Logging in with facebook or google gives a react-native warning
Logging in with e-mail and password gives a (custom) Authentication failed error message
Pressing the "sign-up here" link gives a react-native warning (same as facebook and android)

The same react-native error is encountered when trying to read the terms and conditions etc.


[Scheduling Sample] Adds sample project structure

Project structure for scheduling demo application
Closes #4932

@SocketSecurity ignore<EMAIL_ADDRESS>


build: repair the -SourceKit build

It is possible to disable the SourceKit build.  However, we would
unconditionally add dependencies on targets which would only be
generated when SourceKit was being built, resulting in CMake failing to
configure.  Restore the ability to disable the SourceKit build.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

@swift-ci please test and merge

Thank you for fixing this, @compnerd !


Initial commit

@ylow I believe this PR has the latest perf improvs now, llama2 mount time is ~3.7s for me at home.
xet mount xet://XetHub/LLama2/main LLama2
Thu Jul 27 18:23:05 PDT 2023
Mounting to "/Users/ajitb/src/pyxet/python/pyxet/LLama2"
Cloning into temporary directory "/var/folders/m_/snnsxx1d2k94fkfcxy14g03m0000gn/T/.tmpqSgF7a"
Mounting as a background task...
Setting up mount point...
Mount at "/Users/ajitb/src/pyxet/python/pyxet/LLama2" successful. Unmount with 'umount "/Users/ajitb/src/pyxet/python/pyxet/LLama2"'
Mount complete in 3.697656s


Invalid Cast Exception occurs when script is compiled using Experimental Flag (Roslyn)

I am getting a Invalid Cast exception when I execute my build script simply by adding the #addin reference to Cake.Android.Adb. This happens on both OSX and Windows, but I was able to get a better stack trace on windows with the Cake verbosity level set to debug.
Based on the stack trace, I am assuming this is due to using the Cake -experimental flag, which uses Roslyn to do the script compiling.
Compiling build script...
C:_files\projects\TrGit\DevOpsScripts\Cake\tools\Cake\Cake.exe : Error: System.InvalidCastException: Invalid cast from 'System.Int32' to 'Cake.AndroidAdb.PackageListState'.
At line:1 char:1

C:_files\projects\TrGit\DevOpsScripts\Cake\tools\Cake\Cake.exe "Cake ...

  + CategoryInfo          : NotSpecified: (Error: System.I...kageListState'.:String) [], RemoteException
  + FullyQualifiedErrorId : NativeCommandError

 at System.Convert.DefaultToType(IConvertible value, Type targetType, IFormatProvider provider)
 at Microsoft.CodeAnalysis.Scripting.Emit.ReflectionE

mitter.DefineParameter(MethodBuilder methodBuilder, ConstructorBuilder constructorBuilder, IParameterDefinition paramDef)
at Microsoft.CodeAnalysis.Scripting.Emit.ReflectionEmitter.DefineParameters(MethodBuilder methodBuilder, ConstructorBuilder constructorBuilder, IMethodDefinition methodDef)
at Microsoft.CodeAnalysis.Scripting.Emit.ReflectionEmitter.EmitWorker(IMethodReference entryPoint, CancellationToken cancellationToken)
at Microsoft.CodeAnalysis.Scripting.Emit.CommonCompilationExtensions.Emit(Compilation compilation, ModuleBuilder moduleBuilder, AssemblyLoader assemblyLoader, Func2 assemblySymbolMapper, Boolean recoverOnError, DiagnosticBag diagnostics, CancellationToken cancellationToken, MethodInfo& entryPoint, Byte[]& compiledAssemblyImage)    at Microsoft.CodeAnalysis.Scripting.ScriptBuilder.Build(Script script, DiagnosticBag diagnostics, CancellationToken cancellationToken)    at Microsoft.CodeAnalysis.Scripting.Script.GetExecutor(CancellationToken cancellationToken)    at Microsoft.CodeAnalysis.Scripting.Script.Run(Object globals)    at Microsoft.CodeAnalysis.Scripting.Script.Run(Object globals)    at Cake.Scripting.Roslyn.Nightly.DefaultRoslynNightlyScriptSession.Execute(Script script)    at Cake.Core.Scripting.ScriptRunner.Run(IScriptHost host, FilePath scriptPath, IDictionary2 arguments)
at Cake.Commands.BuildCommand.Execute(CakeOptions options)
at Cake.CakeApplication.Run(CakeOptions options)
at Cake.Program.Main()

Guessing this is resolved in later versions of cake....


Issue in registering SumoEnv-v0 in gym environment

I am trying to run the code on google colab. I have followed the steps but still facing the following error as soon as I try to train the environment !python -m rl_zoo3.train --algo ppo --env SumoEnv-v0 --num-threads 1 --progress --conf-file hyperparams/python/sumoenv_config.py --save-freq 500000 --log-folder /usr/data2/canaltrain_log/ --tensorboard-log /usr/data2/canaltrain_tensorboard/ --verbose 2 --eval-freq 2000000 --eval-episodes 10 --n-eval-envs 10 --vec-env subproc
I am getting the following error-

I have registered the environment in rl_zoo3/import_envs.py by

I have also pasted the code provided on github in exp_manager.py
Kindly let me know if I am doing something wrong.

Did you install the environment in gym with pip install -e .?
Also, there seems to be something wrong with the way you register the environment. Try adding below code after the rocket_lander_gym
try:
    import canalenv
except ImportError:
    canalenv = None

As in doc, you need to also add below default hyperparameters in /hyperparameters/ppo.yml if you need hyper tuning.
SumoEnv-v0:
  normalize: true
  n_envs: 22 # 32 # 40 # 25 # 16 # 64 # 68 # 2 # 256 # 144 # number of environment copies running in parallel
  policy: MlpPolicy
  n_timesteps: !!float 1e6
  batch_size: 32 # Number of training minibatches per update
  n_steps: 128 # batch size is n_steps * n_env
  gamma: 0.99
  learning_rate: 5.0e-4 # The learning rate, it can be a function
  ent_coef: 0.0 # Entropy coefficient for the loss caculation
  clip_range: 0.2 # Clipping parameter, it can be a function
  n_epochs: 10 #  Number of epoch when optimizing the surrogate
  gae_lambda: 0.95 #  Factor for trade-off of bias vs variance for Generalized Advantage Estimator
  max_grad_norm: 1
  vf_coef: 0.871923

Your issue is basically the same as in here

Hi I tried making the changes but its still giving the same error.

I don't think you can run the env on colab, because you need to change some codes under rl_zoo3 (e.g., rl-baselines3-zoo/rl_zoo3/import_envs.py, rl-baselines3-zoo/rl_zoo3/exp_manager.py and rl-baselines3-zoo/hyperparams/ppo.yml).

Just to be clear, in the import_envs.py I just have to copy the following code right
`try:
import canalenv
except ImportError:
canalenv = None`
Do I also need to change anything else or just adding this code will be enough?

Yes, that's what I did with my cloned copy of rl-baselines3-zoo. In case the problem is caused by the update of rl-baselines3-zoo, you can download mine to have a try.

Hi
Yes, its not showing the same error. Thanks a lot for the help


Auxiliary editable adb-s have no effect

Issue: editable .adb/s have no effect when compiling the main adb.
Example: in the subprograms chapter the definition of increment_by.adb is editable and it is used in show_increment.adb. If increment_by.adb is edited (for example, add an extra + 2), the results of show_increment.adb are unchanged.
Expected result: modifications to files that are used to compile a program should affect the final output.
I think (as a student/reader) that if something is editable, it should have consequences if the code is being compiled, even if it is not the main adb. I know that this is not an easy task and one could just copy-paste de code into the file being compiled, but this is very tedious and unintuitive.
Could you add the ability for changes in auxiliary files to also take place in the final compilation? If it is not done, then, making the axulliary code modifiable may be confusing. Leaving it as highlighted text may be better.
Regards,
Fernando

Hi Fernando,
Thanks for reporting this issue. It is indeed a bug. Currently, aggregated files such as this aren't checked for changes before the code is submitted. Other editors, where all the code is in a single editor window (such as this: https://learn.adacore.com/courses/intro-to-ada/chapters/privacy.html#basic-encapsulation) will take into account changes in associated files. We will look into the issue and correct it shortly.
Thanks again,
Rob

Fixed in PR #490


FastQC data extraction

I added the code to extract extra info from FastQC. I also changed some variable names to separate the info from the summary file and the data file, which are two different outputs from FastQC. This still needs to be tested to make sure it works. Please take special attention to the json, I am not familiar with the structure so it is possible I didn't get it completely right.

thanks @RaqManzano close this PR as the commit is already merged into https://github.com/icgc-argo-workflows/argo-qc-tools/pull/28


Autofill.givenName not working on flutter web

Steps to Reproduce
Create a TextField or TextFormField with AutofillHints property set to [AutofillHints.givenName]
(With or without keyboardType set to TextInputType.name)
Run flutter build web
Expected results:
First name presented as autofill suggestion option
Actual results:
No autofill option presented regardless of browser (but working fine for name, family name and email address suggestions)
Secondary issue:
On Safari there should be an Autofill Contact option in addition to a text input suggestion. This doesn't seem to appear for any value of AutofillHints

Sample code:
import 'package:flutter/material.dart';

const Color darkBlue = Color.fromARGB(255, 18, 32, 47);

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      theme: ThemeData.dark().copyWith(
        scaffoldBackgroundColor: darkBlue,
      ),
      debugShowCheckedModeBanner: false,
      home: Scaffold(
        body: Center(
          child: MyWidget(),
        ),
      ),
    );
  }
}

class MyWidget extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return TextField(
    autofillHints:[AutofillHints.givenName]
    );
  }
}

Hopefully a relatively quick one to fix!
Thanks

I can see the autocomplete tag is correctly assigned:
<input name="given-name" id="given-name" type="text" autocomplete="given-name" autocorrect="on" class="flt-text-editing transparentTextEditing" ...>

And the autofill popup does show up if you click the field twice on chrome.
Currently the forms are not recognized by autofill potentially because of this: https://groups.google.com/a/chromium.org/g/blink-dev/c/RY9leYMu5hI

duplicate of https://github.com/flutter/flutter/issues/87735


Allow async notebook output renderers

Tracks changing:
    renderOutputItem(outputItem: OutputItem, element: HTMLElement): void;

to:
    renderOutputItem(outputItem: OutputItem, element: HTMLElement, signal: AbortSignal): void | Promise<void>;

So that renderers can tell VS Code when rendering is async

To verify:

Using the notebook renderer sample
Pick up  "@types/vscode-notebook-renderer": "^1.72.0",
Modify renderOutputItem to have a third abort parameter
At the bottom of the renderOutputItem add:

      abort.onabort = () => console.log('abort');
      await new Promise(resolve => setTimeout(resolve, 10000));

Now open a GitHub issue notebook and view an output item in it

Test:

abort is  aborted when you select a different renderer
It is also aborted if the output is cleared
Or when a new output comes in


[bug] Glitches when initializing video renderer (also with Hero)

Describe the bug
When I initialize a video renderer or put it inside of a Hero widget to try and animate it, it always glitches when it is initialized. There is always this little black flickering or glitching when the thing starts to render video. There is also this platform message channel error that keeps popping up.
To Reproduce
Put a video renderer with the same video track into different pages and then try to switch between those two pages. You can also add a Hero widget to make it more obvious.
It should then look something like this (the video isn't embedded for some reason):
https://github.com/livekit/client-sdk-flutter/assets/70802809/b1e0e077-0461-4f77-907b-957fdca8cdac
Expected behavior
No glitched or black screen when initializing one of these video track renderers. If it appears when the track is first loaded that's totally fine since all the information isn't there yet, but the track flickering when changing between pages or during an animation causes serious issues for people with epilepsy and other related things.
Platform information

Flutter version: 3.19.2
Plugin version: 2.0.2
Flutter target OS: Linux
Flutter target OS version: Arch Linux (newest version)
Flutter console log:

[ERROR:flutter/shell/common/shell.cc(1038)] The 'FlutterWebRTC/Texture111002247562032' channel sent a message from native to Flutter on a non-platform thread. Platform channel messages must be sent on the platform thread. Failure to do so may result in data loss or crashes, and must be fixed in the plugin or application code creating that channel.
See https://docs.flutter.dev/platform-integration/platform-channels#channels-and-platform-threading for more information.
[ERROR:flutter/shell/common/shell.cc(1038)] The 'FlutterWebRTC/Texture111002246794256' channel sent a message from native to Flutter on a non-platform thread. Platform channel messages must be sent on the platform thread. Failure to do so may result in data loss or crashes, and must be fixed in the plugin or application code creating that channel.
See https://docs.flutter.dev/platform-integration/platform-channels#channels-and-platform-threading for more information.
[ERROR:flutter/shell/common/shell.cc(1038)] The 'FlutterWebRTC/Texture111002247626992' channel sent a message from native to Flutter on a non-platform thread. Platform channel messages must be sent on the platform thread. Failure to do so may result in data loss or crashes, and must be fixed in the plugin or application code creating that channel.
See https://docs.flutter.dev/platform-integration/platform-channels#channels-and-platform-threading for more information.

same error! please help!
and then app crash!