hive deployed; failing probes

Hi!
I'm pretty new to K8S and need to develop a billing system.
I'm trying to deploy the following:

MaaS 2.4.2
CDK 1.13.3
mix of libvirt and physical hosts (including a nifty 24core x 96GB)
prometheus-operator-3.0.0 chart
rook-ceph-v0.9.2 chart
operator-metering 0.13.0 tag

I have been testing the system for a few weeks and am pretty confident the infrastructure is stable.
This is my custom configuration:
apiVersion: metering.openshift.io/v1alpha1
kind: Metering
metadata:
  name: "operator-metering"
spec:
  reporting-operator:
    spec:
      config:
        prometheusURL: "http://prometheus-prometheus-oper-prometheus.default:9090/"
        tls:
          enabled: false
        metricsTLS:
          enabled: false
  presto:
    spec:
      presto:
        securityContext:
          fsGroup: 0
      hive:
        securityContext:
          fsGroup: 0
        metastore:
          config:
            logLevel: debug
          storage:
            class: rook-ceph-block
            size: "5Gi"
        server:
          config:
            logLevel: debug
  hdfs:
    spec:
      securityContext:
        fsGroup: 0
      datanode:
        storage:
          class: rook-ceph-block
          size: "5Gi"
      namenode:
        storage:
          class: rook-ceph-block
          size: "5Gi"

Everything seems to work including pv/pvc except for the hive-server and reporting-operator pods:
NAME                                      READY   STATUS    RESTARTS   AGE
pod/hdfs-datanode-0                       1/1     Running   0          40m
pod/hdfs-namenode-0                       1/1     Running   0          40m
pod/hive-metastore-0                      1/1     Running   0          40m
pod/hive-server-0                         0/1     Running   10         40m
pod/metering-operator-c9cb9dccd-bt7gp     2/2     Running   0          42m
pod/presto-coordinator-668d94f95-946st    1/1     Running   0          40m
pod/reporting-operator-6f745b86dd-wjkxf   0/1     Running   5          40m
NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)               AGE
service/hdfs-datanode                ClusterIP   None             <none>        9866/TCP              40m
service/hdfs-datanode-web            ClusterIP   <IP_ADDRESS>   <none>        9864/TCP              40m
service/hdfs-namenode                ClusterIP   None             <none>        9820/TCP              40m
service/hdfs-namenode-proxy          ClusterIP   <IP_ADDRESS>    <none>        9820/TCP              40m
service/hdfs-namenode-web            ClusterIP   <IP_ADDRESS>    <none>        9870/TCP              40m
service/hive-metastore               ClusterIP   <IP_ADDRESS>   <none>        9083/TCP              40m
service/hive-server                  ClusterIP   <IP_ADDRESS>   <none>        10000/TCP,10002/TCP   40m
service/presto                       ClusterIP   <IP_ADDRESS>   <none>        8080/TCP              40m
service/presto-metrics               ClusterIP   <IP_ADDRESS>   <none>        8082/TCP              40m
service/reporting-operator           ClusterIP   <IP_ADDRESS>    <none>        8080/TCP              40m
service/reporting-operator-metrics   ClusterIP   <IP_ADDRESS>    <none>        8082/TCP              40m
NAME                                 READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/metering-operator    1/1     1            1           42m
deployment.apps/presto-coordinator   1/1     1            1           40m
deployment.apps/presto-worker        0/0     0            0           40m
deployment.apps/reporting-operator   0/1     1            0           40m
NAME                                            DESIRED   CURRENT   READY   AGE
replicaset.apps/metering-operator-c9cb9dccd     1         1         1       42m
replicaset.apps/presto-coordinator-668d94f95    1         1         1       40m
replicaset.apps/presto-worker-7db64cb6cd        0         0         0       40m
replicaset.apps/reporting-operator-6f745b86dd   1         1         0       40m
NAME                              READY   AGE
statefulset.apps/hdfs-datanode    1/1     40m
statefulset.apps/hdfs-namenode    1/1     40m
statefulset.apps/hive-metastore   1/1     40m
statefulset.apps/hive-server      0/1     40m

The problem is the probes are failing:
(Hive)
Events:
  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  44m                   default-scheduler  Successfully assigned metering/hive-server-0 to thrall5
  Normal   Pulling    41m (x2 over 44m)     kubelet, thrall5   pulling image "quay.io/coreos/hive:metering-2.3.3"
  Normal   Killing    41m                   kubelet, thrall5   Killing container with id docker://hiveserver2:Container failed liveness probe.. Container will be killed and recreated.
   Container will be killed and recreated.
  Normal   Pulling    36m (x2 over 44m)     kubelet, k8s-worker-2  pulling image "quay.io/coreos/metering-reporting-operator:0.13.0"
  Normal   Pulled     36m (x2 over 44m)     kubelet, k8s-worker-2  Successfully pulled image "quay.io/coreos/metering-reporting-operator:0.13.0"
  Normal   Created    36m (x2 over 44m)     kubelet, k8s-worker-2  Created container
  Normal   Started    36m (x2 over 44m)     kubelet, k8s-worker-2  Started container
  Warning  Unhealthy  9m34s (x17 over 40m)  kubelet, k8s-worker-2  Liveness probe failed: Get http://<IP_ADDRESS>:8080/healthy: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
  Warning  Unhealthy  4m20s (x32 over 42m)  kubelet, k8s-worker-2  Readiness probe failed: HTTP probe failed with statuscode: 500

Here the reporting-operator logs:
time="2019-02-27T11:05:44Z" level=info msg="setting log level to info" app=metering
time="2019-02-27T11:05:44Z" level=info msg="watching namespaces [] for metering.openshift.io resources" app=metering
time="2019-02-27T11:05:44Z" level=info msg="starting Metering operator" app=metering
time="2019-02-27T11:05:44Z" level=info msg="starting Prometheus metrics & pprof servers" app=metering
time="2019-02-27T11:05:44Z" level=info msg="setting up DB connections" app=metering
time="2019-02-27T11:05:44Z" level=info msg="Prometheus metrics server listening on <IP_ADDRESS>:8082" app=metering
time="2019-02-27T11:05:44Z" level=info msg="waiting for caches to sync" app=metering
time="2019-02-27T11:05:44Z" level=info msg="pprof server started" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/cluster-cpu-usage" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/namespace-memory-utilization" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/persistentvolumeclaim-capacity-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/persistentvolumeclaim-phase-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/namespace-memory-usage" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/namespace-memory-request" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/node-cpu-allocatable" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/node-cpu-capacity-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/node-memory-utilization" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/node-memory-allocatable" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/namespace-persistentvolumeclaim-request" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/node-memory-capacity" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/persistentvolumeclaim-usage-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/pod-memory-usage" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/cluster-cpu-capacity-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/cluster-memory-capacity-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/cluster-cpu-usage-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/node-cpu-utilization" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/pod-memory-usage-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/pod-cpu-usage" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/pod-cpu-request" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/namespace-cpu-usage" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/pod-memory-request" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/cluster-memory-usage-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/cluster-memory-usage" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/node-cpu-allocatable-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/node-memory-allocatable-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/persistentvolumeclaim-request" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/node-cpu-capacity" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/node-memory-capacity-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/persistentvolumeclaim-request-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/cluster-persistentvolumeclaim-request" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/cluster-memory-capacity" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/cluster-memory-utilization" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/namespace-cpu-utilization" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/namespace-cpu-request" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/pod-memory-request-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/cluster-cpu-capacity" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/cluster-cpu-utilization" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/pod-cpu-usage-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportGenerationQuery metering/pod-cpu-request-raw" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/pod-usage-cpu-cores" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/node-allocatable-cpu-cores" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/persistentvolumeclaim-phase" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/pod-persistentvolumeclaim-request-info" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/pod-request-memory-bytes" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/node-allocatable-memory-bytes" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/pod-limit-cpu-cores" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/pod-limit-memory-bytes" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/node-capacity-cpu-cores" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/pod-usage-memory-bytes" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/pod-request-cpu-cores" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/persistentvolumeclaim-request-bytes" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/persistentvolumeclaim-usage-bytes" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/node-capacity-memory-bytes" app=metering
time="2019-02-27T11:05:46Z" level=info msg="adding ReportDataSource metering/persistentvolumeclaim-capacity-bytes" app=metering
time="2019-02-27T11:05:46Z" level=info msg="starting HTTP server" app=metering
time="2019-02-27T11:05:46Z" level=info msg="testing ability to write to Presto" app=metering
time="2019-02-27T11:05:46Z" level=info msg="HTTP API server listening on <IP_ADDRESS>:8080" app=metering
time="2019-02-27T11:07:22Z" level=info msg="\"GET http://<IP_ADDRESS>:8080/ready HTTP/1.1\" from <IP_ADDRESS>:45112 - 500 50B in 446.36µs" app=metering component=api
time="2019-02-27T11:08:22Z" level=info msg="\"GET http://<IP_ADDRESS>:8080/ready HTTP/1.1\" from <IP_ADDRESS>:46890 - 500 50B in 131.115µs" app=metering component=api
time="2019-02-27T11:09:22Z" level=info msg="\"GET http://<IP_ADDRESS>:8080/ready HTTP/1.1\" from <IP_ADDRESS>:48680 - 500 50B in 130.484µs" app=metering component=api
time="2019-02-27T11:10:22Z" level=info msg="\"GET http://<IP_ADDRESS>:8080/ready HTTP/1.1\" from <IP_ADDRESS>:50492 - 500 50B in 126.706µs" app=metering component=api
time="2019-02-27T11:11:22Z" level=info msg="\"GET http://<IP_ADDRESS>:8080/ready HTTP/1.1\" from <IP_ADDRESS>:52286 - 500 50B in 124.153µs" app=metering component=api
time="2019-02-27T11:12:22Z" level=info msg="\"GET http://<IP_ADDRESS>:8080/ready HTTP/1.1\" from <IP_ADDRESS>:54064 - 500 50B in 927.242µs" app=metering component=api
time="2019-02-27T11:12:57Z" level=error msg="cannot create Presto table operator_health_check" app=metering component=testWriteToPresto error="timed out while waiting to connect to hive"
time="2019-02-27T11:12:57Z" level=info msg="\"GET http://<IP_ADDRESS>:8080/healthy HTTP/1.1\" from <IP_ADDRESS>:48272 - 500 61B in 3m49.192773507s" app=metering component=api
time="2019-02-27T11:12:57Z" level=info msg="\"GET http://<IP_ADDRESS>:8080/healthy HTTP/1.1\" from <IP_ADDRESS>:53642 - 500 61B in 49.1910237s" app=metering component=api
time="2019-02-27T11:12:57Z" level=info msg="\"GET http://<IP_ADDRESS>:8080/healthy HTTP/1.1\" from <IP_ADDRESS>:51864 - 500 61B in 1m49.157938482s" app=metering component=api
time="2019-02-27T11:12:57Z" level=info msg="\"GET http://<IP_ADDRESS>:8080/healthy HTTP/1.1\" from <IP_ADDRESS>:46480 - 500 61B in 4m49.193204032s" app=metering component=api
time="2019-02-27T11:12:57Z" level=info msg="\"GET http://<IP_ADDRESS>:8080/healthy HTTP/1.1\" from <IP_ADDRESS>:50070 - 500 61B in 2m49.189570603s" app=metering component=api
time="2019-02-27T11:12:58Z" level=info msg="got signal terminated, performing shutdown"
time="2019-02-27T11:12:58Z" level=error msg="cannot create Presto table operator_health_check" app=metering component=testWriteToPresto error="context canceled"
time="2019-02-27T11:12:58Z" level=fatal msg="error occurred while the reporting-operator was running" app=metering error="timed out waiting for the condition"

And here the hive-server, where I increased logLevel=debug:
Setting HADOOP_HEAPSIZE to 250M
which: no hbase in (/opt/hive-2.3.3/bin:/opt/hadoop-3.1.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
2019-02-27 11:05:45: Starting HiveServer2
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive-2.3.3/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-3.1.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
19/02/27 11:07:07 [main]: INFO conf.HiveConf: Found configuration file file:/opt/hive-2.3.3/conf/hive-site.xml
19/02/27 11:07:40 [main]: INFO server.HiveServer2: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting HiveServer2
STARTUP_MSG:   host = hive-server-0.hive-server.metering.svc.cluster.local/<IP_ADDRESS>
STARTUP_MSG:   args = [--hiveconf, hive.root.logger=DEBUG,console]
STARTUP_MSG:   version = 2.3.3
STARTUP_MSG:   classpath = [.... boring list of paths ...]
STARTUP_MSG:   build = file:///build -r Unknown; compiled by 'root' on Thu Feb 21 00:29:59 UTC 2019
************************************************************/
19/02/27 11:07:40 [main]: INFO server.HiveServer2: Starting HiveServer2
19/02/27 11:07:46 [main]: DEBUG security.SecurityUtil: Setting hadoop.security.token.service.use_ip to true
19/02/27 11:07:46 [main]: DEBUG security.Groups:  Creating new Groups object
19/02/27 11:07:47 [main]: DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
19/02/27 11:07:47 [main]: DEBUG util.NativeCodeLoader: Loaded the native-hadoop library
19/02/27 11:07:47 [main]: DEBUG security.JniBasedUnixGroupsMapping: Using JniBasedUnixGroupsMapping for Group resolution
19/02/27 11:07:47 [main]: DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMapping
19/02/27 11:07:50 [main]: DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
19/02/27 11:07:51 [main]: DEBUG session.SessionState: SessionState user: null
19/02/27 11:08:03 [main]: DEBUG core.Tracer: sampler.classes = ; loaded no samplers
19/02/27 11:08:03 [main]: DEBUG core.Tracer: span.receiver.classes = ; loaded no span receivers
19/02/27 11:08:20 [main]: DEBUG impl.DfsClientConf: dfs.client.use.legacy.blockreader.local = false
19/02/27 11:08:20 [main]: DEBUG impl.DfsClientConf: dfs.client.read.shortcircuit = false
19/02/27 11:08:20 [main]: DEBUG impl.DfsClientConf: dfs.client.domain.socket.data.traffic = false
19/02/27 11:08:20 [main]: DEBUG impl.DfsClientConf: dfs.domain.socket.path =
19/02/27 11:08:22 [main]: DEBUG hdfs.DFSClient: Sets dfs.client.block.write.replace-datanode-on-failure.min-replication to 0
19/02/27 11:08:22 [main]: DEBUG retry.RetryUtils: multipleLinearRandomRetry = null
19/02/27 11:08:24 [main]: DEBUG ipc.Server: rpcKind=RPC_PROTOCOL_BUFFER, rpcRequestWrapperClass=class org.apache.hadoop.ipc.ProtobufRpcEngine$RpcProtobufRequest, rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@29b732a2
19/02/27 11:08:24 [main]: DEBUG ipc.Client: getting client out of cache: org.apache.hadoop.ipc.Client@74518890
19/02/27 11:08:38 [pool-2-thread-1]: INFO server.HiveServer2: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down HiveServer2 at hive-server-0.hive-server.metering.svc.cluster.local/<IP_ADDRESS>
************************************************************/

Do you have an idea of where to look, find more information about what's failing?
Thanks

I have also tried this suggestion ( from https://github.com/operator-framework/operator-metering/issues/557 ), same results:
[...]
  hdfs:
    spec:
      securityContext:
        runAsNonRoot: false
        runAsUser: 0
        fsGroup: 0
[...]

The hive pod being not ready will prevent reporting-operator from being healthy. Can you kubectl describe pod hive-0? The logs don’t indicate much so it might be getting killed.

Sure, here is the full kubectl describe pod hive-server-0 (I did reinstall in the meantime):
Name:           hive-server-0
Namespace:      metering
Node:           thrall5/<IP_ADDRESS>
Start Time:     Wed, 27 Feb 2019 19:18:07 +0100
Labels:         app=hive
                controller-revision-hash=hive-server-67cdb6d89
                hive=server
                statefulset.kubernetes.io/pod-name=hive-server-0
Annotations:    hive-configmap-hash: f31f735bce88e02db405038fd1622da9d68e43d7f288041df205b94e7f0fc358
                hive-scripts-hash: e8827831d156b05c46f8a9950f42262b86688eada016871d6fecf56dffd6b5f8
                presto-aws-credentials-secret-hash: f39e5d1c730fe17ed6914046e445507a4df1ad381f65eeb5b24df02025908657
Status:         Running
IP:             <IP_ADDRESS>
Controlled By:  StatefulSet/hive-server
Containers:
  hiveserver2:
    Container ID:  docker://28a0486c89773f75e0bd1445f51a8686029bd2884bc472626d157d1ce9bf3f4b
    Image:         quay.io/coreos/hive:metering-2.3.3
    Image ID:      docker-pullable://quay.io/coreos/hive@sha256:b986dcc2cacc3a5806cbf1cfa3863a9d57799f1603d449b5105f8789e81a062d
    Ports:         10000/TCP, 10002/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      /hive-scripts/entrypoint.sh
    Args:
      /opt/hive/bin/hive
      --service
      hiveserver2
    State:          Running
      Started:      Wed, 27 Feb 2019 19:21:19 +0100
    Last State:     Terminated
      Reason:       Error
      Exit Code:    143
      Started:      Wed, 27 Feb 2019 19:18:14 +0100
      Finished:     Wed, 27 Feb 2019 19:21:15 +0100
    Ready:          False
    Restart Count:  1
    Limits:
      cpu:     100m
      memory:  500Mi
    Requests:
      cpu:      100m
      memory:   500Mi
    Liveness:   tcp-socket :10000 delay=90s timeout=15s period=30s #success=1 #failure=3
    Readiness:  tcp-socket :10000 delay=60s timeout=15s period=20s #success=1 #failure=3
    Environment:
      HIVE_LOGLEVEL:          DEBUG
      MY_NODE_NAME:            (v1:spec.nodeName)
      MY_POD_NAME:            hive-server-0 (v1:metadata.name)
      MY_POD_NAMESPACE:       metering (v1:metadata.namespace)
      JAVA_MAX_MEM_RATIO:     50
      AWS_ACCESS_KEY_ID:      <set to the key 'aws-access-key-id' in secret 'presto-aws-credentials'>      Optional: true
      AWS_SECRET_ACCESS_KEY:  <set to the key 'aws-secret-access-key' in secret 'presto-aws-credentials'>  Optional: true
      MY_MEM_REQUEST:         524288000 (requests.memory)
      MY_MEM_LIMIT:           524288000 (limits.memory)
    Mounts:
      /hadoop-config from hdfs-config (rw)
      /hadoop/dfs/data from datanode-empty (rw)
      /hadoop/dfs/name from namenode-empty (rw)
      /hive-config from hive-config (rw)
      /hive-scripts from hive-scripts (rw)
      /user/hive/warehouse from hive-warehouse-empty (rw)
      /var/lib/hive from hive-metastore-db-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from hive-token-vv89t (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  hive-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      hive-config
    Optional:  false
  hive-scripts:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      hive-scripts
    Optional:  false
  hdfs-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      hdfs-config
    Optional:  false
  hive-warehouse-empty:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
  namenode-empty:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
  datanode-empty:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
  hive-metastore-db-data:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
  hadoop-logs:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
  hive-token-vv89t:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  hive-token-vv89t
    Optional:    false
QoS Class:       Guaranteed
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  5m17s                 default-scheduler  Successfully assigned metering/hive-server-0 to thrall5
  Normal   Pulling    2m9s (x2 over 5m13s)  kubelet, thrall5   pulling image "quay.io/coreos/hive:metering-2.3.3"
  Normal   Killing    2m9s                  kubelet, thrall5   Killing container with id docker://hiveserver2:Container failed liveness probe.. Container will be killed and recreated.
  Normal   Pulled     2m7s (x2 over 5m11s)  kubelet, thrall5   Successfully pulled image "quay.io/coreos/hive:metering-2.3.3"
  Normal   Created    2m7s (x2 over 5m11s)  kubelet, thrall5   Created container
  Normal   Started    2m5s (x2 over 5m10s)  kubelet, thrall5   Started container
  Warning  Unhealthy  14s (x9 over 3m54s)   kubelet, thrall5   Readiness probe failed: dial tcp <IP_ADDRESS>:10000: connect: connection refused
  Warning  Unhealthy  14s (x4 over 3m14s)   kubelet, thrall5   Liveness probe failed: dial tcp <IP_ADDRESS>:10000: connect: connection refused

I also see some warnings in hive-metastore-0:
19/02/27 18:20:11 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at:
	org.apache.hadoop.hive.metastore.ObjectStore.getMSchemaVersion(ObjectStore.java:7628)
19/02/27 18:20:11 [main]: WARN metastore.ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 2.3.0
19/02/27 18:20:11 [main]: WARN metastore.ObjectStore: setMetaStoreSchemaVersion called but recording version is disabled: version = 2.3.0, comment = Set by MetaStore<EMAIL_ADDRESS>19/02/27 18:20:11 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at:
	org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.start(ObjectStore.java:2745)
19/02/27 18:20:11 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at:
	org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.commit(ObjectStore.java:2838)
19/02/27 18:20:11 [main]: DEBUG metastore.ObjectStore: db details for db default retrieved using SQL in 11.498839ms
19/02/27 18:20:11 [main]: WARN metastore.ObjectStore: Failed to get database default, returning NoSuchObjectException
19/02/27 18:20:11 [main]: DEBUG security.SecurityUtil: Setting hadoop.security.token.service.use_ip to true
19/02/27 18:20:12 [main]: DEBUG security.Groups:  Creating new Groups object
[...]
19/02/27 18:20:24 [main]: INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty
[...]

but the probes are working for that pod. I can attach full logs if you need.

Ok, well it seems like in all likelyhood it's just that hive isn't starting quickly enough before the health probe fails Liveness:   tcp-socket :10000 delay=90s timeout=15s period=30s #success=1 #failure=3. That's the config we start with, so it's port isn't open within 135 seconds in effect.
Currently you can override the default liveness probe settings with a config like this (values provided are the defaults):
apiVersion: metering.openshift.io/v1alpha1
kind: Metering
metadata:
  name: "operator-metering"
spec:
  presto:
    spec:
      hive:
        server:
          livenessProbe:
           initialDelaySeconds: 90
           timeoutSeconds: 15
           periodSeconds: 30
           successThreshold: 1
           failureThreshold: 3

Thanks, giving more resources did the trick.
I had to repeat the install/uninstall a few times because I also have an intermittent failure upon install on the Tiller probe (44135), but I'll open a separate ticket if that one persists.


Status : Failed To Tap

Can't tap pro: :(
Status : Failed To Tap

it mean your daily limit point reached , or its just game problem

Game is not problem, I've loged on my phone and tap fine


typed-irbuilder: try to make function pass typed args

This is not a real PR, just a draft on what should be done to make function arguments tagged with the right types.
This doesn't go as far as using singletons, currently we have to pass function
arguments with the TypeApplication. We need some massaging to make it look
nicer. Also it used unsafeCoerce under the hood.
But it let's you avoid coerce, function arguments in the body do-block have the
right type tags right away.
Example2 is changed to use new function, also added bigger Example3.

Wonderful this is a nice solution to the type arguments logic problem, the unsafeCoerce is nicely encapsulated and should be safe.


Issue trying to import createEnvironment

Thank you very much for this project! I am looking forward to being able to use Angular Material component test harnesses on my playwright e2e tests.
I am, however, getting the following error when importing the createEnvironment function from @ngx-playwright/test:
Error [ERR_REQUIRE_ESM]: require() of ES Module<EMAIL_ADDRESS>from /Users/rafasiqueira/Work/etips/projects/etips-registration/test/creating-pochette.e2e.ts not supported.
Instead change the require of index.js in /Users/rafasiqueira/Work/etips/projects/etips-registration/test/creating-pochette.e2e.ts to a dynamic import() which is available in all CommonJS modules.

   at ../../node_modules/@playwright/test/lib/utilsBundleImpl.js:16

Here is a snippet of my code:
import { test, expect, Page, createEnvironment } from '@ngx-playwright/test';
import { MatIconHarness } from '@angular/material/icon/testing';

test('Should find icon', async ({ page }) => {
    const testEnvironment = createEnvironment(page);
    const icons = await MatIconHarness.with({name: 'create_new_folder'});
(...)

I have tried pre-compiling the test with tsc, but I get the same error during compilation. Any help will be appreciated!

The @angular/cdk/testing and @angular/material packages are ESM (with import/export syntax) only. The error you're getting indicates that your code is transpiled to CommonJS code (with require and module.exports). There's documentation in the playwright docs, but in short:
If you don't want to precompile the code, playwright comes with an (experimental) ESM loader that supports typescript. It requires that

the playwright config is written in TypeScript with ESM syntax,
the package.json has "type": "module", and
you're using node 16 or later.

If you set up playwright using ng add @ngx-playwright/test (or run ng generate @ngx-playwright/test:setup) all of this should be set up correctly for you.
If you want to keep precompiling your code, make sure the module in your tsconfig is set to es2017( or other esXXXX), esnext, node16, or nodenext; and have "type": "module" in your package.json to tell node to load the files as ESM.

Thank you!

I know this is not related to your library, but perhaps you have had experience with it?
I am getting this error now when importing MatIconHarness. Any help would be greatly appreciated!
Error: The injectable 'PlatformLocation' needs to be compiled using the JIT compiler, but '@angular/compiler' is not available.

The injectable is part of a library that has been partially compiled.
However, the Angular Linker has not processed the library such that JIT compilation is used as fallback.

Ideally, the library is processed using the Angular Linker to become fully AOT compiled.
Alternatively, the JIT compiler should be loaded by bootstrapping using '@angular/platform-browser-dynamic' or '@angular/platform-server',
or manually provide the compiler with 'import "@angular/compiler";' before bootstrapping.

    at getCompilerFacade (file:///Users/rafasiqueira/packages/core/src/compiler/compiler_facade.ts:52:11)
    at Module.ɵɵngDeclareFactory (file:///Users/rafasiqueira/packages/core/src/render3/jit/partial.ts:58:20)
    at file:///Users/rafasiqueira/Work/etips/node_modules/@angular/common/fesm2015/common.mjs:90:28
    at async Promise.all (index 0)

I'm afraid I've never used @angular/material or its harnesses, so I can't be of assistance there.

@bgotink , thank you. I will figure it out.
Trying to go forward without the angular material harness, my test execution gets stuck on:
if (status.onDetectChangesNow) {
      Promise.all(
        Array.from(pages, page => page.evaluate(waitUntilAngularStable)),
      ).then(
        () => status.onDetectChangesNow?.(),
        () => status.onDetectChangesNow?.(),
      );
    }

Yeah that's a feature I've been thinking of switching from opt-out to opt-in…
Every change performed via a harness waits for the angular app to stabilise before completing, but that requires the angular app to stabilise. That's not something you actually run into in apps itself, so it turns out it's common for apps to not be "stable" at all.
That's true even in our applications at work where we do require apps to be stable for certain functionality to work.
Set enableAutomaticStabilization to false to disable this behaviour. For example, in your playwright config:
const config = {
  use: {
    channel: 'chrome',
    headless: true,
    enableAutomaticStabilization: false
  },
  // ...
};

I had been trying to do that for a while. Setting it on the playwright config for some reason didn't work. I had to set it on the spec file using:
test.use({
  enableAutomaticStabilization: false
});

Setting it on the playwright config for some reason didn't work.

Oh, thanks for letting me know. That should be fixed in @ngx-playwright/test version 0.2.8.

Again, thank you so much for your project. I am now able to use all Angular Material component harnesses. I am still "waiting" for elements to be visible manually, since I couldn't get the stabilization thing figure out. See examples below, for your reference. Your project is really useful for anyone using Angular Material components and Playwright.
test.use({
  enableAutomaticStabilization: false
});

test.describe('Creating a pochette', () => {

  let loader: PlaywrightHarnessEnvironment;

  test.beforeEach((async ({harnessEnvironment}) => {
    TestBed.initTestEnvironment(
      BrowserDynamicTestingModule,
      platformBrowserDynamicTesting()
    );
    loader = harnessEnvironment;
  }));

  test('From an email with files attached individually', async ({ page }) => {

    await page.goto('/main');
    // await loader.waitForAngularReady();

    // 1. From the main page of the Registration Module (See How to access Registration Module)
    let icon: MatIconHarness;
    while (!icon) {
      icon = await loader.getHarnessOrNull(MatIconHarness.with({name: 'create_new_folder'}));
      await page.waitForTimeout(50);
    }

    // 2. Click on button "Create pochette"
    const button = await loader.getHarness(MatButtonHarness.with({text: new RegExp(/Create pochette/)}));
    await button.click();
    await expect(page).toHaveURL('/pochette');

    // 3. Select "Email" as the Type of submission
    const emailToggle = await loader.getHarness(MatButtonToggleHarness.with({text: new RegExp(/Email/)}));
    await emailToggle.toggle();

    // 4. Upload the email pdf by dropping it on top of the cloud icon or by clicking on that same icon
    const inputLocator = 'input[type="file"]';
    await page.locator(inputLocator).setInputFiles(`${testRoot}/files/1. Pochette from email with files attached individually.pdf`);

    // 5. Delete unnecessary attachments by clicking on the red garbage bin icon
    let discardBtns: MatButtonHarness[] = new Array<MatButtonHarness>();
    while (discardBtns.length === 0) {
      discardBtns = await loader.getAllHarnesses(MatButtonHarness.with({text: new RegExp(/delete/)}));
      await page.waitForTimeout(50);
    }
    await discardBtns[1].click();
    await discardBtns[2].click();
(...)


Example React build fails

Using the example at https://stackblitz.com/github/vitebook/vitebook/tree/main/examples/react?file=package.json the vitebook:build script fails with:
⠼ Rendering 3 pages...'useState' is not exported by .vitebook/.cache/deps_build-5c78b490/react.js, imported by src/Button/Button.story.tsx
file: /home/projects/qgniywqbg.github/src/Button/Button.story.tsx:8:0
 6:   EventsAddon,
 7: } from '@vitebook/preact/addons';
 8: import { useState } from 'react';
    ^
 9:
10: import Button from './Button';
🚨 Rendering 3 pages...

 Error: 'useState' is not exported by .vitebook/.cache/deps_build-5c78b490/react.js, imported by src/Button/Button.story.tsx
    at error (file:///home/projects/qgniywqbg.github/node_modules/rollup/dist/es/shared/rollup.js:1888:30)
    at Module.error (file:///home/projects/qgniywqbg.github/node_modules/rollup/dist/es/shared/rollup.js:12441:16)
    at Module.traceVariable (file:///home/projects/qgniywqbg.github/node_modules/rollup/dist/es/shared/rollup.js:12800:29)
    at ModuleScope.findVariable (file:///home/projects/qgniywqbg.github/node_modules/rollup/dist/es/shared/rollup.js:11452:39)
    at FunctionScope.findVariable (file:///home/projects/qgniywqbg.github/node_modules/rollup/dist/es/shared/rollup.js:6385:38)
    at ChildScope.findVariable (file:///home/projects/qgniywqbg.github/node_modules/rollup/dist/es/shared/rollup.js:6385:38)
    at Identifier.bind (file:///home/projects/qgniywqbg.github/node_modules/rollup/dist/es/shared/rollup.js:7452:40)
    at CallExpression.bind (file:///home/projects/qgniywqbg.github/node_modules/rollup/dist/es/shared/rollup.js:5282:23)
    at CallExpression.bind (file:///home/projects/qgniywqbg.github/node_modules/rollup/dist/es/shared/rollup.js:8948:15)
    at VariableDeclarator.bind (file:///home/projects/qgniywqbg.github/node_modules/rollup/dist/es/shared/rollup.js:5282:23) {
  code: 'MISSING_EXPORT',
  url: 'https://rollupjs.org/guide/en/#error-name-is-not-exported-by-module',
  id: '/home/projects/qgniywqbg.github/src/Button/Button.story.tsx',
  pos: 138,
  loc: {
    column: 0,
    file: '/home/projects/qgniywqbg.github/src/Button/Button.story.tsx',
    line: 8
  },
  frame: ' 6:   EventsAddon,\n' +
    " 7: } from '@vitebook/preact/addons';\n" +
    " 8: import { useState } from 'react';\n" +
    '    ^\n' +
    ' 9: \n' +
    "10: import Button from './Button';",
  watchFiles: [
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>    '/home/projects/qgniywqbg.github/.vitebook/.cache/deps_build-5c78b490/svelte.js',
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>    '/home/projects/qgniywqbg.github/.vitebook/.cache/deps_build-5c78b490/svelte_store.js',
    '/home/projects/qgniywqbg.github/.vitebook/.cache/deps_build-5c78b490/svelte_internal.js',
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>    '/:virtual/vitebook/pages',
    '/:virtual/vitebook/site',
    '/home/projects/qgniywqbg.github/.vitebook/theme/index.ts',
<EMAIL_ADDRESS>    '\x00vite/preload-helper',
    '/home/projects/qgniywqbg.github/src/Button/Button.story.tsx',
    '/home/projects/qgniywqbg.github/src/Button/docs.md',
<EMAIL_ADDRESS>    '/home/projects/qgniywqbg.github/node_modules/preact/package.json',
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>    '/home/projects/qgniywqbg.github/node_modules/preact/dist/preact.module.js',
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>    '/home/projects/qgniywqbg.github/.vitebook/App.tsx',
    '/home/projects/qgniywqbg.github/src/Button/Button.tsx',
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>    '/home/projects/qgniywqbg.github/node_modules/@vitebook/client/dist/client/components/OutboundLink.svelte?svelte&type=style&lang.css',
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>    '/home/projects/qgniywqbg.github/node_modules/preact/hooks/package.json',
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>    '/home/projects/qgniywqbg.github/node_modules/preact/hooks/dist/hooks.module.js',
<EMAIL_ADDRESS>    '/home/projects/qgniywqbg.github/node_modules/preact/compat/package.json',
    ... 148 more items
  ]
}

Even with a simple index.md
# Hello world

The build error we can get is:
$ vitebook build
<EMAIL_ADDRESS>
🚨 Rendering 1 pages...

 ReferenceError: require is not defined
    at eval (file:///home/projects/github-me4ejx/.vitebook/dist/server/entry-server.js:3192:27)
    at Object.$$render (file:///home/projects/github-me4ejx/.vitebook/dist/server/entry-server.js:343:22)
    at eval (file:///home/projects/github-me4ejx/.vitebook/dist/server/entry-server.js:3229:65)
    at Object.$$render (file:///home/projects/github-me4ejx/.vitebook/dist/server/entry-server.js:343:22)
    at eval (file:///home/projects/github-me4ejx/.vitebook/dist/server/entry-server.js:1059:83)
    at Object.$$render (file:///home/projects/github-me4ejx/.vitebook/dist/server/entry-server.js:343:22)
    at eval (file:///home/projects/github-me4ejx/.vitebook/dist/server/entry-server.js:1068:45)
    at Object.$$render (file:///home/projects/github-me4ejx/.vitebook/dist/server/entry-server.js:343:22)
    at eval (file:///home/projects/github-me4ejx/.vitebook/dist/server/entry-server.js:2587:50)
    at Object.$$render (file:///home/projects/github-me4ejx/.vitebook/dist/server/entry-server.js:343:22)

Should be fixed now, thanks for reporting :)

Thank you @mihar-22


Integrating deletion, search and auto references

Important changes include the addition of a search bar, the implementation of delete functionality for references, and the removal of the reference_id input field from the forms.

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 0% with 15 lines in your changes missing coverage. Please review.

Project coverage is 21.46%. Comparing base (0e72f1d) to head (de3e0fd).

Files with missing lines
Patch %
Lines

src/app.py
0.00%
15 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff             @@
##             main      #34       +/-   ##
===========================================
- Coverage   58.62%   21.46%   -37.16%
===========================================
  Files           1        4        +3
  Lines          58      177      +119
  Branches       12       22       +10
===========================================
+ Hits           34       38        +4
- Misses         17      132      +115
  Partials        7        7

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Fix postinst script to handle disabled server

This change makes the postinst script source the /etc/default/drserv file and
not try to start the server unless it has been properly enabled.
Before this change the postinst script used to exit with an unhelpful error
message (and an exit code) when the server was disabled.

Review please?
@Shaywei, @nresare

Tested locally and it works. +1

I would also like to see a fix that adds the same echo message if you do a "service drserv start" instead of "start: Job failed to start" (when the ENABLED != yes).

Remedied @Shaywei's suggestion as good as possible (log message, I wasn't able to print anything).


ActionModel needs to expose RouteValueDefaults

When generating action model, a key ingeredient to pass extra specific knowledge to the action as runtime is route value defaults (because they can be easily bound).
Also it would be nice to specify properties as key value pairs.

@rynowak @Eilon Close?

RouteValueDefaults was removed. Close please :+1:


Feature request: support site access on click

All of the extension I use support the following option, but ic-inspector doesn't provide it.

I like this feature because it allows me to opt-in to when an extension can access pages I'm browsing to. The alternative is to enable and disable the extension, which is less convenient.

Thanks for the report Paul (and for bringing it to my attention on the dev call today!) I'll have a look at how to add this feature.

So, extension permissions are... weird. This extension has never requested any explicit permissions, as you can see in the manifest file's empty permissions array.
I've identified that I can achieve the  interface in your screenshot by adding the <all_urls> host permission to the manifest, but it doesn't seem to affect the functioning of the extension in any way, even if it is apparently being restricted:

Notice that no access has been granted to the extension, but it's capturing messages in devtools.
My interpretation here is that ic inspector has no access to your data whatsoever, until you open up it's specific tab in devtools. Here's a screenshot using the current production permissions, where we can see the extension never gets access to the active tab:

I might be missing something here, since I don't have a lot of experience with extensions. As far as I can tell though, I have the most restrictive permissions possible (i.e. no permissions at all) and so it's frustrating that this is what people see when they check which permissions the extension requires:

Thanks for looking into it!


can't install in ubuntu 18

root@noob-U32U:/home/noob/Public/YouTube-Viewer# pip install -r requirements.txt
Collecting beautifulsoup4==4.9.3 (from -r requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/d1/41/e6495bd7d3781cee623ce23ea6ac73282a373088fcd0ddc809a047b18eae/beautifulsoup4-4.9.3-py3-none-any.whl
Collecting bs4==0.0.1 (from -r requirements.txt (line 2))
Using cached https://files.pythonhosted.org/packages/10/ed/7e8b97591f6f456174139ec089c769f89a94a1a4025fe967691de971f314/bs4-0.0.1.tar.gz
Exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 353, in run
wb.build(autobuilding=True)
File "/usr/lib/python3/dist-packages/pip/wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 376, in prepare_files
discovered_reqs.extend(self._prepare_file(
File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 634, in _prepare_file
abstract_dist.prep_for_dist()
File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 129, in prep_for_dist
self.req_to_install.run_egg_info()
File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 412, in run_egg_info
self.setup_py, self.name,
File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 387, in setup_py
import setuptools  # noqa
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 664, in _load_unlocked
File "", line 627, in _load_backward_compatible
File "", line 259, in load_module
File "/usr/share/python-wheels/setuptools-39.0.1-py2.py3-none-any.whl/setuptools/init.py", line 14, in
from setuptools.dist import Distribution, Feature
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 664, in _load_unlocked
File "", line 627, in _load_backward_compatible
File "", line 259, in load_module
File "/usr/share/python-wheels/setuptools-39.0.1-py2.py3-none-any.whl/setuptools/dist.py", line 24, in
from setuptools.depends import Require
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 664, in _load_unlocked
File "", line 627, in _load_backward_compatible
File "", line 259, in load_module
File "/usr/share/python-wheels/setuptools-39.0.1-py2.py3-none-any.whl/setuptools/depends.py", line 7, in
from .py33compat import Bytecode
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 664, in _load_unlocked
File "", line 627, in _load_backward_compatible
File "", line 259, in load_module
File "/usr/share/python-wheels/setuptools-39.0.1-py2.py3-none-any.whl/setuptools/py33compat.py", line 54, in
unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape)
AttributeError: 'HTMLParser' object has no attribute 'unescape'

try again after upgrading setuptools
pip3 install --upgrade setuptools

Traceback (most recent call last):
File "youtube_viewer.py", line 45, in
import undetected_chromedriver as uc
File "/home/noob/.local/lib/python3.6/site-packages/undetected_chromedriver/init.py", line 31, in
from . import v2
File "/home/noob/.local/lib/python3.6/site-packages/undetected_chromedriver/v2.py", line 4
from future import annotations
^
SyntaxError: future feature annotations is not defined

Install python 3.7 to 3.9

youtube_viewer.py:672: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead
input_keyword = driver.find_element_by_css_selector('input#search')
youtube_viewer.py:642: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead
f"//[@id='button' and @aria-label='{popup}']").click()
youtube_viewer.py:737: DeprecationWarning: find_element_by_ commands are deprecated. Please use find_element() instead
driver.find_element_by_css_selector('[title^="Pause (k)"]')
youtube_viewer.py:770: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead
"button.ytp-button.ytp-settings-button").click()
youtube_viewer.py:772: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead
"//div[contains(text(),'Quality')]").click()
youtube_viewer.py:785: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead
'//[@id="container"]/h1/yt-formatted-string').click()
/home/noob/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py:393: UserWarning: find_element_by_ commands are deprecated. Please use find_element() instead
warnings.warn("find_element_by_* commands are deprecated. Please use find_element() instead")
[26-Nov-2021 03:24:05] Tried 1 | Line : 1018 | invalid literal for int() with base 10: '800,'

need to modi all the find_element ?

which is the best 3.9.9 or 3.9.4?

after 3.9.4.
$python3 youtube_viewer.py
Traceback (most recent call last):
File "/home/noob/YouTube-Viewer/youtube_viewer.py", line 53, in
from undetected_chromedriver.patcher import Patcher
ModuleNotFoundError: No module named 'undetected_chromedriver.patcher'

pip install -r requirements.txt

ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python3 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-4nts_vwh/undetected-chromedriver_96c71e5a07b9439a9d62a76e5e21497f/setup.py'"'"'; file='"'"'/tmp/pip-install-4nts_vwh/undetected-chromedriver_96c71e5a07b9439a9d62a76e5e21497f/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-degyve03
cwd: /tmp/pip-install-4nts_vwh/undetected-chromedriver_96c71e5a07b9439a9d62a76e5e21497f/
Complete output (26 lines):
running egg_info
creating /tmp/pip-pip-egg-info-degyve03/undetected_chromedriver.egg-info
writing /tmp/pip-pip-egg-info-degyve03/undetected_chromedriver.egg-info/PKG-INFO
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-4nts_vwh/undetected-chromedriver_96c71e5a07b9439a9d62a76e5e21497f/setup.py", line 33, in
setup(
File "/home/noob/.local/lib/python3.9/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/usr/local/lib/python3.9/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/lib/python3.9/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/local/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/noob/.local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 292, in run
writer(self, ep.name, os.path.join(self.egg_info, ep.name))
File "/home/noob/.local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 656, in write_pkg_info
metadata.write_pkg_info(cmd.egg_info)
File "/usr/local/lib/python3.9/distutils/dist.py", line 1117, in write_pkg_info
self.write_pkg_file(pkg_info)
File "/home/noob/.local/lib/python3.9/site-packages/setuptools/dist.py", line 167, in write_pkg_file
write_field('Summary', single_line(self.get_description()))
File "/home/noob/.local/lib/python3.9/site-packages/setuptools/dist.py", line 151, in single_line
raise ValueError('Newlines are not allowed')
ValueError: Newlines are not allowed
WARNING: Discarding https://files.pythonhosted.org/packages/42/64/91559bf5a2202d9a0aa82f2c5c02f716055b36beb4c725992a48e1f934b9/undetected-chromedriver-3.0.2.tar.gz#sha256=febbc1ed7cabda2b94934f8f27c29add4f44c1290014e166aa0a562e7fc06af7 (from https://pypi.org/simple/undetected-chromedriver/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement undetected-chromedriver==3.0.2 (from versions: 1.3.5, 1.3.6, 1.3.7, 1.4.0, 1.4.2, 1.5.0, 1.5.1, 1.5.2, 2.0b0, 2.0.0, 2.0.1, 2.0.2, 2.1.0, 2.1.1, 2.1.2, 2.2.0, 2.2.1, 2.2.7, 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.0.4, 3.0.5)
ERROR: No matching distribution found for undetected-chromedriver==3.0.2

pip install setuptools<59
pip install -r requirements.txt

bash: 59 No such file or directory

pip3 install setuptools==58.5.3
Next time post screenshot of your error


Rewrite GoogleButton to tsx format and fix the sonar issues

Rewrite GoogleButton to tsx format and fix the sonar issues

You have pushed to this branch a bunch of unrelated to this task changes, try to fix that

I don't pass the tests unless I change something in these files.

wrong branch


Build errors in vs 2015 / 17 RC with latest Excel

Hi, when I try to export the data from excel to SQL server it gives me a build error, can you please update.
Check if table exists in Sql Server then update the data

Can you give more log. I’d like to know the method you’re calling also.


Can you provide left shadow-hand robot files please.

The left hand of shadow robot is provided in the latest commit: e40c2a41c8552bff2d91c5ebbf800fdf9edac2aa. You can find it here robots/shadow_hand/shadow_hand_left.urdf


Project name clears from time entry

💻 Environment

Platform: MacOS
OS Version: OS X 10.14.4
Toggl Version: Stable, 7.4.484
🐞 Actual behavior

After creating a new project for running time entry, the project field clears up. Video here:
https://www.dropbox.com/s/go0q6bt4avjuk4l/Screen Recording 2019-08-05 at 2.29.03 pm.mov?dl=0
💯 Expected behavior

Project name that is created and selected should stay selected.
🔨 Steps to reproduce
Video here: https://www.dropbox.com/s/go0q6bt4avjuk4l/Screen Recording 2019-08-05 at 2.29.03 pm.mov?dl=0
📦 Additional info
User Report

user case

Case


Support more SeaOrm datatypes in scaffold

Feature Request
SeaOrm supports things such as blob but generate scaffold doesn't support it.
Describe the solution you'd like
have blob and blob! types

Thanks!
we'll be happy to accept a PR if you'd like to become a contributor.
it is as easy as adding the types you want here: https://github.com/loco-rs/loco/blob/master/src/gen/mappings.json
you'll want to also read around in the gen module to see how they're used.
if you can't make it, please let me know and we'll add this type quite easily.

Currently I'm very busy so I don't think I'll have the time to do it until some time in the future.

No worries.
Fixed in master!


fix  open dependencies/python-requirements.txt: no such file or directory

2024/07/23 09:49:28 server.go:24: [ERROR]failed to setup runner dependencies: open dependencies/python-requirements.txt: no such file or directory.

Thanks a lots!


Quota Limits reached in Walidator

Pull Request
43
What happened?
EIP Walidator fails to run, as it hits quota limits;
Refer https://github.com/ethereum/ERCs/actions/runs/6664676695/job/18112762062?pr=43
2023-10-27T08:17:18.7430627Z ##[group]Run ethereum/eipw-action@6858f56d1d9e5cbe49a0afd85150669f6983ed20
2023-10-27T08:17:18.7431484Z with:
2023-10-27T08:17:18.7432800Z   token: ***
2023-10-27T08:17:18.7433175Z   unchecked: 1
2023-10-27T08:17:18.7433562Z   path: ERCS/
2023-10-27T08:17:18.7433976Z   options-file: config/eipw.toml
2023-10-27T08:17:18.7434476Z ##[endgroup]
2023-10-27T08:17:19.4213347Z Request quota exhausted for request GET https://api.github.com/repos/ethereum/ERCs/pulls/43/files
2023-10-27T08:17:19.4214730Z Retrying after 1751 seconds!
2023-10-27T08:46:30.6570178Z Request quota exhausted for request GET https://api.github.com/repos/ethereum/ERCs/pulls/43/files
2023-10-27T08:46:30.6571285Z Retrying after 0 seconds!

Relevant log output
2023-10-27T08:17:18.2926106Z Resolving deltas:  95% (92/96)
2023-10-27T08:17:18.2926598Z Resolving deltas:  96% (93/96)
2023-10-27T08:17:18.2927103Z Resolving deltas:  97% (94/96)
2023-10-27T08:17:18.2968699Z Resolving deltas:  98% (95/96)
2023-10-27T08:17:18.2969660Z Resolving deltas: 100% (96/96)
2023-10-27T08:17:18.2970193Z Resolving deltas: 100% (96/96), done.
2023-10-27T08:17:18.3648393Z From https://github.com/ethereum/ERCs
2023-10-27T08:17:18.3649969Z  * [new ref]         ed28201473f772b953998fe7e49f3a284a655866 -> pull/43/merge
2023-10-27T08:17:18.3680115Z ##[endgroup]
2023-10-27T08:17:18.3681065Z ##[group]Determining the checkout info
2023-10-27T08:17:18.3683078Z ##[endgroup]
2023-10-27T08:17:18.3683994Z ##[group]Checking out the ref
2023-10-27T08:17:18.3690140Z [command]/usr/bin/git checkout --progress --force refs/remotes/pull/43/merge
2023-10-27T08:17:18.6841119Z Note: switching to 'refs/remotes/pull/43/merge'.
2023-10-27T08:17:18.6841912Z
2023-10-27T08:17:18.6842930Z You are in 'detached HEAD' state. You can look around, make experimental
2023-10-27T08:17:18.6844180Z changes and commit them, and you can discard any commits you make in this
2023-10-27T08:17:18.6845373Z state without impacting any branches by switching back to a branch.
2023-10-27T08:17:18.6846151Z
2023-10-27T08:17:18.6852155Z If you want to create a new branch to retain commits you create, you may
2023-10-27T08:17:18.6853525Z do so (now or later) by using -c with the switch command. Example:
2023-10-27T08:17:18.6854492Z
2023-10-27T08:17:18.6854846Z   git switch -c <new-branch-name>
2023-10-27T08:17:18.6855315Z
2023-10-27T08:17:18.6855575Z Or undo this operation with:
2023-10-27T08:17:18.6856016Z
2023-10-27T08:17:18.6856242Z   git switch -
2023-10-27T08:17:18.6856573Z
2023-10-27T08:17:18.6857136Z Turn off this advice by setting config variable advice.detachedHead to false
2023-10-27T08:17:18.6857941Z
2023-10-27T08:17:18.6858811Z HEAD is now at ed28201 Merge ba664d607145f45456d605275c3841afe2b49d5f into a5f4f4632e6df7915343fdbfa57ce4d270cf2562
2023-10-27T08:17:18.6877596Z ##[endgroup]
2023-10-27T08:17:18.6936305Z [command]/usr/bin/git log -1 --format='%H'
2023-10-27T08:17:18.6976128Z 'ed28201473f772b953998fe7e49f3a284a655866'
2023-10-27T08:17:18.7430627Z ##[group]Run ethereum/eipw-action@6858f56d1d9e5cbe49a0afd85150669f6983ed20
2023-10-27T08:17:18.7431484Z with:
2023-10-27T08:17:18.7432800Z   token: ***
2023-10-27T08:17:18.7433175Z   unchecked: 1
2023-10-27T08:17:18.7433562Z   path: ERCS/
2023-10-27T08:17:18.7433976Z   options-file: config/eipw.toml
2023-10-27T08:17:18.7434476Z ##[endgroup]
2023-10-27T08:17:19.4213347Z Request quota exhausted for request GET https://api.github.com/repos/ethereum/ERCs/pulls/43/files
2023-10-27T08:17:19.4214730Z Retrying after 1751 seconds!
2023-10-27T08:46:30.6570178Z Request quota exhausted for request GET https://api.github.com/repos/ethereum/ERCs/pulls/43/files
2023-10-27T08:46:30.6571285Z Retrying after 0 seconds!
2023-10-27T08:46:30.8135944Z Request quota exhausted for request GET https://api.github.com/repos/ethereum/ERCs/pulls/43/files
2023-10-27T08:46:30.8137215Z Retrying after 0 seconds!
2023-10-27T08:46:30.9641981Z Request quota exhausted for request GET https://api.github.com/repos/ethereum/ERCs/pulls/43/files
2023-10-27T08:46:30.9700847Z RequestError [HttpError]: API rate limit exceeded for installation ID 2246695. If you reach out to GitHub Support for help, please include the request ID 7C80:4BF1:169136D:176DA4E:653B78E6.
2023-10-27T08:46:30.9703860Z     at /home/runner/work/_actions/ethereum/eipw-action/6858f56d1d9e5cbe49a0afd85150669f6983ed20/dist/index.js:6691:25
2023-10-27T08:46:30.9705643Z     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2023-10-27T08:46:30.9707818Z     at async Job.doExecute (/home/runner/work/_actions/ethereum/eipw-action/6858f56d1d9e5cbe49a0afd85150669f6983ed20/dist/index.js:8747:22) {
2023-10-27T08:46:30.9709374Z   status: 403,
2023-10-27T08:46:30.9710259Z   response: {
2023-10-27T08:46:30.9711902Z     url: 'https://api.github.com/repos/ethereum/ERCs/pulls/43/files',
2023-10-27T08:46:30.9712700Z     status: 403,
2023-10-27T08:46:30.9713144Z     headers: {
2023-10-27T08:46:30.9713650Z       'access-control-allow-origin': '*',
2023-10-27T08:46:30.9722362Z       'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset',
2023-10-27T08:46:30.9725524Z       'content-encoding': 'gzip',
2023-10-27T08:46:30.9726256Z       'content-security-policy': "default-src 'none'",
2023-10-27T08:46:30.9727125Z       'content-type': 'application/json; charset=utf-8',
2023-10-27T08:46:30.9727919Z       date: 'Fri, 27 Oct 2023 08:46:30 GMT',
2023-10-27T08:46:30.9728994Z       'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
2023-10-27T08:46:30.9729973Z       server: 'GitHub.com',
2023-10-27T08:46:30.9730895Z       'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
2023-10-27T08:46:30.9731876Z       'transfer-encoding': 'chunked',
2023-10-27T08:46:30.9732643Z       vary: 'Accept-Encoding, Accept, X-Requested-With',
2023-10-27T08:46:30.9733442Z       'x-content-type-options': 'nosniff',
2023-10-27T08:46:30.9736040Z       'x-frame-options': 'deny',
2023-10-27T08:46:30.9736770Z       'x-github-media-type': 'github.v3; format=json',
2023-10-27T08:46:30.9737712Z       'x-github-request-id': '7C80:4BF1:169136D:176DA4E:653B78E6',
2023-10-27T08:46:30.9738517Z       'x-ratelimit-limit': '1000',
2023-10-27T08:46:30.9739129Z       'x-ratelimit-remaining': '0',
2023-10-27T08:46:30.9739757Z       'x-ratelimit-reset': '1698396390',
2023-10-27T08:46:30.9740415Z       'x-ratelimit-resource': 'core',
2023-10-27T08:46:30.9741039Z       'x-ratelimit-used': '1003',
2023-10-27T08:46:30.9741607Z       'x-xss-protection': '0'
2023-10-27T08:46:30.9742074Z     },
2023-10-27T08:46:30.9742403Z     data: {
2023-10-27T08:46:30.9744379Z       message: 'API rate limit exceeded for installation ID 2246695. If you reach out to GitHub Support for help, please include the request ID 7C80:4BF1:169136D:176DA4E:653B78E6.',
2023-10-27T08:46:30.9746692Z       documentation_url: 'https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting'
2023-10-27T08:46:30.9747744Z     }
2023-10-27T08:46:30.9748045Z   },
2023-10-27T08:46:30.9748370Z   request: {
2023-10-27T08:46:30.9748783Z     method: 'GET',
2023-10-27T08:46:30.9749506Z     url: 'https://api.github.com/repos/ethereum/ERCs/pulls/43/files',
2023-10-27T08:46:30.9750280Z     headers: {
2023-10-27T08:46:30.9750818Z       accept: 'application/vnd.github.v3+json',
2023-10-27T08:46:30.9751759Z       'user-agent': 'octokit-core.js/3.6.0 Node.js/20.8.1 (linux; x64)',
2023-10-27T08:46:30.9752636Z       authorization: 'token [REDACTED]'
2023-10-27T08:46:30.9753182Z     },
2023-10-27T08:46:30.9753503Z     request: {
2023-10-27T08:46:30.9753928Z       agent: [Agent],
2023-10-27T08:46:30.9754424Z       hook: [Function: bound bound register],
2023-10-27T08:46:30.9755006Z       retryCount: 2
2023-10-27T08:46:30.9755390Z     }
2023-10-27T08:46:30.9755716Z   }
2023-10-27T08:46:30.9756008Z }
2023-10-27T08:46:30.9795088Z ##[error]API rate limit exceeded for installation ID 2246695. If you reach out to GitHub Support for help, please include the request ID 7C80:4BF1:169136D:176DA4E:653B78E6.
2023-10-27T08:46:30.9948397Z Post job cleanup.
2023-10-27T08:46:31.1356185Z [command]/usr/bin/git version
2023-10-27T08:46:31.1415055Z git version 2.42.0
2023-10-27T08:46:31.1469921Z Temporarily overriding HOME='/home/runner/work/_temp/356dc8ac-02e1-4d8d-a597-143f04bf7d0d' before making global git config changes
2023-10-27T08:46:31.1471950Z Adding repository directory to the temporary git global config as a safe directory
2023-10-27T08:46:31.1478690Z [command]/usr/bin/git config --global --add safe.directory /home/runner/work/ERCs/ERCs
2023-10-27T08:46:31.1532604Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2023-10-27T08:46:31.1580892Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2023-10-27T08:46:31.1912725Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2023-10-27T08:46:31.1957589Z http.https://github.com/.extraheader
2023-10-27T08:46:31.1973757Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
2023-10-27T08:46:31.2021639Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2023-10-27T08:46:31.2739924Z Cleaning up orphan processes

This is a limitation of the GitHub API. I dp believe that there could be a better error message though. Please open an issue on the Walidator repository.

Or CC @SamWilsn to transfer the issue


Adds tracks list in album. #82

This PR adds tracks into the album model and solves the issue #82.
It is a little hacky, since I can't access the tracks directly without going through the Paging model first. So I added a property called tracksNative that parses each item in the Page and converts it into a TrackSimple model. Any suggestion on how to improve this is welcome.

@roisnir @rinukkusu Can I merge it?


Mosaic plot (2x2) - statistics tab frontend updates

For the 2x2 mosaic plot viz tool, mockups for the statistics tab can be found here: https://www.figma.com/file/Su1eNm3Bby85CfrnX42L7q/2x2-table-UI?node-id=123%3A233

Related to https://github.com/VEuPathDB/CoreUI/issues/114 and #1383

@danicahelb May I ask a couple of questions regarding this ticket and relevant one (https://github.com/VEuPathDB/web-eda/issues/1383)?
First, I made a collapsible banner and screenshots can be found in https://github.com/VEuPathDB/CoreUI/pull/129#issue-1453839015, which is a mock up for #1383. But from my understanding, there are no Quadrant input variable yet, thus the banner won't be included till the Quadrant input variable is made, right? (Quadrant work is not part of #1383 nor #1384 (this ticket)).
Second, this is a question for this ticket, statistics tab updates: a) If I understand correctly, we will also have a separate banner for this tab only. Can you please confirm this? b) the scoop of this work is to have statistic table too? It is currently shown as "Coming soon".

Hi @moontrip thanks for raising this topic!
I apologize, this key ticket got lost in the shuffle! https://github.com/VEuPathDB/EdaDataService/issues/130 (the ticket title, major feature & release info are now up to date)
Yes, the stats tab will have a separate banner. See: https://www.figma.com/file/Su1eNm3Bby85CfrnX42L7q/2x2-table-UI?node-id=123%3A234&t=eO7GflQwHS5GJltr-0
We are still finalizing the stats tab and need to make decisions on:

what stats to include
table layout
verbiage for the banner

Please pause work on this ticket until the stats tab has completed review by the EDA UX group

styling ticket for stats tab: https://github.com/VEuPathDB/web-eda/issues/1253

backend tickets to calculate the various statistics: https://github.com/VEuPathDB/EdaNewIssues/issues/533 & https://github.com/VEuPathDB/EdaNewIssues/issues/534
We would like to roll out these stats for b62:

Chi-square
Fischer's exact
Prevalence
Odds Ratio
Risk Ratio

There are an additional 4 stats related to diagnostic test performance, backend for these may not be ready for b62. If backend is NOT ready, please leave placeholder rows in the table with "Coming soon" where the stats would be

Mockups have been finalized: https://www.figma.com/file/Su1eNm3Bby85CfrnX42L7q/2x2-table-UI?node-id=391%3A637&t=C7dsTpu6Y9mLsmoz-0
95% CI should each be given 2 decimal places
P values should be given to two significant figures, unless p<0.0001

I'm running into this bug when I try to construct a 2x2 table. I will QA this ticket once the bug has been fixed and the plot/stats can be made.

@nkittur-uga Thank you for your information. I have double checked with the backend response (statsTable) just in case that I misallocated values to be displayed but I think the backend did return the same one with the frontend display.
@d-callan Can you please take look at this? Thanks in advance 👍

I think this is due to a difference in how we calculate the odds ratio. From the site linked in a previous comment I see: "Where zeros cause problems with computation of the odds ratio or its standard error, 0.5 is added to all cells (a, b, c, d) (Pagano & Gauvreau, 2000; Deeks & Higgins, 2010)."
We aren't doing this, and since we instead have NaN I suspect this is the issue.

@d-callan there are no zeros here, unless I am missing something?

I used this calculator and got this: http://vassarstats.net/odds2x2.html

Using this site (which sets up their table differently than us): http://vassarstats.net/odds2x2.html

I've hand checked the calculation for Odds ratio and Risk ratio, using the formulas in our more information dropdown.
I get the same numbers as the calculator:
Odds ratio = 5.63
Risk ratio = 2.04
So the risk ratio point estimate is also wrong.
Please check & fix calculations for:

Odds ratio 95% CI
Risk ratio point estimate
Risk ration 95% CI

statistics calculations should be addressed in: https://github.com/VEuPathDB/plot.data/issues/202 (which also failed QA)

@moontrip, can you confirm that you are using the right values for Sensitivity, Specificity, PPV and NPV?
It seems like values for (i) Sensitivity and PPV are swapped; and (ii) values for Specificity and NPV are swapped. I don't know if this should be fixed on the frontend or backend
See more info here: https://github.com/VEuPathDB/plot.data/issues/203

@danicahelb I left my comment with the screenshot of the backend response at https://github.com/VEuPathDB/plot.data/issues/203#issuecomment-1450727474

@moontrip can you please capitalize the first letter in both words of Odds Ratio? thanks!

@danicahelb I have made the corresponding PR to address your request. Here is a screenshot. Please feel free to let me know if you need further change(s)

@moontrip can you also change Case control or Cross-sectional studies to Case control & Cross-sectional studies

@danicahelb new screenshot with the change:

looks good!


Resize handle unselectable when using Minimal theme

Hello! I really love this plugin :D. For some reason tho, the Minimal theme is preventing dragging the resize handle. Instead of the up and down arrows, I just see the regular text cursor. I'm not sure if there's any setting in Minimal to fix this. Thought I'd let you know

It seems to work for me, so I'm marking this issue as resolved since it took me a while to get to this. If it's still not working for you could you send me a recording of the problem, and the css file for your theme just incase we're using different versions. Thanks!
https://github.com/ocapraro/obsidian-math-plus/assets/25327166/280f28d5-c09d-40cd-9f13-9149644885d9


Replacing |. with -> gives compile error when using underscore

With bsb -version 4.0.3, this code works:
let doStuff = (a: int, b: int, c: int): int => {
  a + 2 * b + 3 * c;
};

let test1 = 5 |. doStuff(3, _, 7);
Js.log2("Test1:", test1);

but this gives a compile error:
let test2 = 5 -> doStuff(3, _, 7);
Js.log2("Test2:", test2);

The error is:
  We've found a bug for you!
  /home/david/testor/src/Demo.re 9:13-24

   7 │ Js.log2("Test1:", test1);
   8 │
   9 │ let test2 = 5 -> doStuff(3, _, 7);
  10 │ Js.log2("Test2:", test2);
  11 │

  This function has type (int, int) => int
  It only accepts 2 arguments; here, it's called with more.

(The 5 -> doStuff is highlighted)
Note: Even when the _ is in the first position, the same error occurs. Getting rid of the underscore fixes the problem.

@jdeisenberg thanks for the bug report, I'm investigating.
Hmm
let test1 = 5 |. doStuff(3, _, 7);
let test2 = 5->doStuff(3, _, 7);

is parsed as
let test1 = 5 |. (__x => doStuff(3, __x, 7));
let test2  = (__x) => (5 |. doStuff)(3, __x);

@anmonteiro this is a side-effect of 5->doStuff(3, _, 7); being parsed as (5->doStuff)(3, _, 7);.
What do you think? Do we modify process_underscore_application?


Add useExpanded in the pluginOrder

Selection get broken when rows are expanded if the plugin order is wrong.
#2578

https://github.com/tannerlinsley/react-table/issues/2578

:tada: This PR is included in version 7.4.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Swift 5 support, upgrade current support swift version from 3 - > 4.2…

Swift 5 support, upgrade current support swift version from 3 - > 4.2, which is the min support version by xcode 10.2

Xcode 10.2 is released, time to merge this? 🙃

Hi @rickyzh, thank you very much for the PR, JASON has been updated to the latest Swift version.
Thanks again :)


[Proposal] particle serial terminal

This is similar to particle serial monitor but user input will be passed to the device as well. It should be a single char passed each time a key is pressed.

:+1: this would be an extremely useful feature. It would be awesome if the command had an option that would open a serial connection to device, enter a character sequence, return the output to stdout, and exit.  Perhaps my feature request is a bit different than what Kenneth is describing, my use case is more like a generalized version of particle identify
To contextualize my use case, here is an example task I'd use this feature for: Write a test case somewhere in our CI pipeline that would assert a device reports expected versions of things ala particle serial command s > some.json then read some.json and assert the system versions are set correctly.

particle serial send xxx ?

Perfect.
On Tue, Aug 4, 2015 at 10:07 AM, Kenneth Lim<EMAIL_ADDRESS>wrote:

particle serial send xxx ?
—
Reply to this email directly or view it on GitHub
https://github.com/spark/particle-cli/issues/85#issuecomment-127643901.

For particle serial terminal, a useful feature would be to reopen the serial port automatically after a device reset. It's a pain right now to use another terminal program that needs to be reopened whenever the device is reflashed. I end up missing the first few lines of output unless I put a big delay at the beginning of the user firmware...

would it also be useful if we allow users to specify whether they want each char sent immediately, or if they want to queue them up and then send on newline? This would be helpful against typos when playing with basic protocols that are newline terminated.

For some reason I thought particle serial monitor already did this... but as I'm finding out it does not?  Is particle serial monitor read (monitor) only?  When I type characters they are shown on the screen which is very misleading, as if they are being sent and echoed.
I would love to see particle serial terminal supersede particle serial monitor and depreciate monitor completely.  I realize monitor might be considered the GET request of serial if it's read only, and more of a safe serial terminal... however I think it would feel too confusing to have terminal and monitor and not being able to send characters doesn't seem like a mode we need for serial interaction.  If we did a command called particle debug monitor or even particle debug for a dedicated system debug output that might be better suited for read-only.
For particle serial terminal all echoing should be handled by the device, and all line terminations be entered by the user by default.  We could get fancy with options here particle serial terminal <baud> <--echo> <--line-ending crlf|lfcr|cr|lf> but I would not let this or the other features requested above keep us from implementing the default operation first :)
@brycekahle this is a valuable & much needed built-in feature for the CLI and totally gets :+1: :+1: of support.  Please let me know if you want help bringing this to life.

Since we now got the auto-reconnect feature --follow this would be the logical next step 😉

Please don't close that. I'd still want this to happen.


Alter runtests for py2 + py3 compat

which I noticed when running make on an Arch system where python is python3.
:memo: Before this can pass, the v2 branch will need to be up to date with master, as the examples in the v1 README use v1-specific bits that have no corresponding bits in v2.

:sparkles: nothing changed anymore :rofl:


checksum-dependency-plugin: implement DNS-based load balancing

As of now the plugin relies on Java's HttpUrlConnection, and OpenJDK does not implement DNS load-balancing.

Implemented in checksum-dependency-plugin 1.24.0


Allow split by length and overridable spliter by options

what are you using this to parse?
If it's really a fixed length format, I'm guessing it would probably use a count of bytes, not utf8 characters (which may be a different bytelength)


Could not download symbols.  The request for path ... failed with code Forbidden. Reason: You do not have the following permissions on TableData Published Application: IndirectRead for company ...

1. Describe the bug
I got an on-prem (container) implementation upgraded from 18.0 to 19.0
I've checked previous similar issues and Google, but nothing worked for me.
When try to download symbols I'm getting a permission error.
I've double checked app.json and launch.json.
I've checked permission assigned to the user I'm using. I got SUPER and EXTEN. MGT. - ADMIN roles assigned without specifying company.
My dev/demo license for BC19 got this object with full rights, also is on the Effective permissions page.
2. To Reproduce
Steps to reproduce the behavior:
I got the same on a clean container using 19.0.29884.30693 W1 artefacts.
3. Expected behavior
To be able to download symbols in VS Code.
4. Actual behavior
[2022-02-22 22:19:18.94] Using reference symbols cache path: some_path
[2022-02-22 22:19:18.94] Sending request to some_url:7049/BC/dev/packages?publisher=Microsoft&appName=Application&versionText=<IP_ADDRESS>&appId=00000000-0000-0000-0000-000000000000&tenant=default
[2022-02-22 22:19:18.94] Sending request to some_url:7049/BC/dev/packages?publisher=Microsoft&appName=System&versionText=<IP_ADDRESS>&appId=8874ed3a-0643-4247-9ced-7a7002f7135d&tenant=default
[2022-02-22 22:19:19.51] The request for path /BC/dev/packages?publisher=Microsoft&appName=System&versionText=<IP_ADDRESS>&appId=8874ed3a-0643-4247-9ced-7a7002f7135d&tenant=default failed with code Forbidden. Reason: You do not have the following permissions on TableData Published Application: IndirectRead for company some_company.
To view details about your permissions, see the Effective Permissions page. To report a problem, refer to the following server session ID: '31'.
[2022-02-22 22:19:19.51] The request for path /BC/dev/packages?publisher=Microsoft&appName=Application&versionText=<IP_ADDRESS>&appId=00000000-0000-0000-0000-000000000000&tenant=default failed with code Forbidden. Reason: You do not have the following permissions on TableData Published Application: IndirectRead for company some_company.
To view details about your permissions, see the Effective Permissions page. To report a problem, refer to the following server session ID: '32'.
[2022-02-22 22:19:19.52] Could not download reference symbols. Please ensure that:

The correct server name and instance are specified in the launch.json file.
The correct application version is specified in the app.json file.
The dependencies are correctly specified in the app.json file.

5. Versions:

AL Language: 18.0, runtime 7.0 / 19.0, runtime 8.0
Visual Studio Code: 1.64.2
Business Central: W1 19.0 (Platform 19.0.29884.30666 + Application 19.0.29894.30693)

Thank you for help.

When you are deploying an app, then the app will added to all companies. That means you need to have insert permissions in all companies.
If you have a Production license then you are allowed to publish to both evaluation and non-evaluation companies.
but if you have a evaluation license then it is allow to publish to evaluation companies, but only to non-evaluation companies for 30 days.
For example, I have an environment with a evaluation license. It is 25 days since I got the license and created the environment.
My environment has both a non-evaluation and a evaluation company, and I deploy all the apps I like. Then another 10 days roll by and now I try to deploy an app again, but now it fails.
In order to keep testing my apps, I then go in and remove my non-evaluation companies, so my environment now only contains evaluation companies, and it is again possible for me to deploy apps.
To learn more see: Licensing in Business Central - Business Central | Microsoft Docs

how to resolve this

And any hope to get a clearer error message about it as current notification is very misleading and improper.

Same error message as mention in subject line.
I am upgrading from BC16.6 to BC19. after upgrade trying to download
symbols from VS
On Tue, Mar 22, 2022 at 6:26 AM NH2501 @.***> wrote:

And any hope to get a clearer error message about it as current
notification is very misleading and improper.
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/AL/issues/6946#issuecomment-1074784934, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AYKRTNKXUULW2VGZ3DEZO5TVBFRYJANCNFSM5PCU2WRQ
.
You are receiving this because you commented.Message ID:
@.***>

--
Regards;
Sarika Maheshwari
+91-9971484155

Sarikamaheshwari, you can try to remove evaluation company mark directly in SQL, in company table, as far as I remember. This should allow you to proceed.

although this is quite an old issue I would like to add an unexpected resolution that worked for me.
the solution is to use the .flf  file in stead of the .bclicense file.
Cost me some time to exclude all other suggestions. It is a license related issue so I tried the .flf file
BC Version 19.1 ? Please try this if everything else fails.


pkg/convert: reduce allocation

Why?
in my company, the jfr to pprof allocate too much times of objects, and those cost most of cpu.
this is the allocation memory profile of my parca-agent:

What?
this pr reduce 99% percent of memory allocation and reduce 80% of cpu cost in jfr to pprof.

How?
1, cache parseArgs and getFileName and getPid and dimensions.
2, reuse slice as more as we can.
3, use bytes.buffer to build function key, and use unsafe string in map search to avoid memory allocation.
4, use unsafe string sample key in map search, only create locations when new sample created.
5, use uint64 as location key to avoid int to string convert.

Nice work!!


Create iam_repository

Q
A

🐛 Bug fix?
no

🚀 New feature?
no

⚠ Deprecations?
no

❌ BC Break
no

🔗 Related issues
#165

❓ Documentation
no

Description
iam_* supplier have mutual hidden dependency, so i create iam_repository to avoid this.
I moved the error handling out of the suppliers and back into the resources.
I can divise this PR into multiples pr for each resources.

Can you check the test related to pkg/remote/aws/test/iam_user_policy_multiple/results.golden.json ? It should not change the results, try to keep the same id.


optOptimize SetOperator by removing unnecessary maintained elements.

What problem does this PR solve?
Previously, the Set operator needed to maintain a RowRefListWithFlag,
but in fact, only a "visited" flag is needed.
before
mysql [test]>select count(u64) from( select u64 from nums1 except DISTINCT select u64 from nums2 )t;
+------------+
| count(u64) |
+------------+
|          0 |
+------------+
1 row in set (3.97 sec)

now
mysql [test]>select count(u64) from( select u64 from nums1 except DISTINCT select u64 from nums2 )t;
+------------+
| count(u64) |
+------------+
|          0 |
+------------+
1 row in set (3.01 sec)

Release note
None
Check List (For Author)

Test

[ ] Regression test
[ ] Unit Test
[ ] Manual test (add detailed scripts or steps below)
[x] No need to test or manual test. Explain why:

[x] This is a refactor/code format and no logic has been changed.
[ ] Previous test can cover this change.
[ ] No code files have been changed.
[ ] Other reason

Behavior changed:

[x] No.
[ ] Yes.

Does this need documentation?

[x] No.
[ ] Yes.

Check List (For Reviewer who merge this PR)

[ ] Confirm the release note
[ ] Confirm test cases
[ ] Confirm document
[ ] Add branch pick label

run buildall

run buildall


While using DBContextManager in rxwebcore for using Transaction for relational save Error

Right now I solved problem by using AutoTransactionsEnabled false.
Following code:
Uow.Context.Database.AutoTransactionsEnabled = false;
var result = await DbContextManager.BeginTransactionAsync(this.Uow);
using (result)
{
try
{
await Uow.RegisterNewAsync(entity.Student);
Uow.Context.SaveChanges();
await Uow.RegisterNewAsync(entity);
Uow.Context.SaveChanges();
await result.CommitAsync();
}
catch(Exception e)
{
Console.WriteLine(e.Message);
await result.RollbackAsync();
}
}
Uow.Context.Database.AutoTransactionsEnabled = true;
If it can be included in DbContextManager it will be more good.

Thanks @varsharoberoi for this ticket. As internally we are using the UseTransaction and BeginTransaction Method so we don't need to accommodate AutoTransactionsEnabled in the lib. Instead of the result object, you can directly use the CommitAsync method of DbContextManager.

I am closing this ticket as no further action required on this. It's good you have shared your inputs. Keep continuing the same :).


fix: update node groups on node ready

Fixes #1072

@jmgiaever Could you give this a try?

Pull Request Test Coverage Report for Build 758161514

0 of 3 (0.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 17.927%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

lib/ZwaveClient.js
0
3
0.0%

Totals

Change from base Build 758154115:
0.0%

Covered Lines:
2030

Relevant Lines:
11614

💛  - Coveralls

Sure, thank you! I'm just compiling 3.4.0 first and push it to the latest snap-branch first.
(Is it Monday already? :wink: )

Seems to be working now. Great job! :)


Extension issue

Issue Type: Bug
Extension Name: vscode-pull-request-github
Extension Version: 0.34.3
OS Version: Windows_NT x64 10.0.19044
VS Code version: 1.63.2

:warning: We have written the needed data into your clipboard. Please paste! :warning:

Closing due to missing info.


[Documentation]: Entity Extraction should use CUDA if available or at least surface the option

Documentation Issue Description
During my exploration of metadata extractors, particularly the line:
EntityExtractor(prediction_threshold=0.5),

I initially discovered the necessity of installing span_marker. Post-installation, despite having a GPU, the extractor ran on CPU. This led me to the span_marker docs, where the recommendation to move the model to CUDA for faster inference is stated:
# It is recommended to explicitly move the model to CUDA for faster inference, if possible
model.cuda()

Somewhere, possibly in LlamaIndexBot, I found that the device parameter is where GPU/CPU preference should be set. CoPilot then provided a succinct check:
EntityExtractor(prediction_threshold=0.5, device="cuda" if cuda_available else "cpu"),

I propose an explicit section in both the setup procedure and documentation to ensure a straightforward setup of CUDA when available.
Documentation Link
https://docs.llamaindex.ai/en/stable/examples/metadata_extraction/MetadataExtractionSEC.html

what is the best way to interact with you? also, can you review my code if i send it to you?
Sent from Surface Duo

From: dosubot[bot] @.>
Sent: Saturday, November 4, 2023 5:50:08 PM
To: run-llama/llama_index @.>
Cc: Nick Katsivelos @.>; Mention @.>
Subject: Re: [run-llama/llama_index] [Documentation]: Entity Extraction should use CUDA if available or at least surface the option (Issue #8685)
🤖
Hello @nickknychttps://github.com/nickknyc,
I'm Dosu, a friendly bot here to assist you with LlamaIndex! I can help you solve bugs, answer questions, and guide you in becoming a contributor. While we wait for a human maintainer, feel free to ask me anything. I'm here to help!
It seems like you've got this under control, if you want help or have specific questions, let me know what I can do for you!
—
Reply to this email directly, view it on GitHubhttps://github.com/run-llama/llama_index/issues/8685#issuecomment-1793563187, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAMGI3G523EQTSU73X3W7H3YC22BBAVCNFSM6AAAAAA6527NM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJTGU3DGMJYG4.
You are receiving this because you were mentioned.Message ID: @.***>

Please let me know if you see any issues with the following ingestion pipeline
import logging
import os
import sys
import pinecone
import torch
from llama_index import (ServiceContext, SimpleDirectoryReader,
StorageContext, VectorStoreIndex, set_global_service_context)
from llama_index.embeddings import OpenAIEmbedding
from llama_index.indices.prompt_helper import PromptHelper
from llama_index.llms import OpenAI
from llama_index.node_parser import SimpleNodeParser
from llama_index.node_parser.extractors import (EntityExtractor, KeywordExtractor,
MetadataExtractor, QuestionsAnsweredExtractor,
SummaryExtractor, TitleExtractor)
from llama_index.text_splitter import TokenTextSplitter
from llama_index.vector_stores import PineconeVectorStore
Set up logging
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
setup system tests
test to see if cuda is available
cuda_available = torch.cuda.is_available()
These are user defined variables - they should be moved into .env or somehting
DOC_PATH = r"D:\ObsidianVault\test"  # be sure to keep the r in front of the string to make it a raw string
SPLITTER_CHUNK_SIZE = 1024
SPLITTER_CHUNK_OVERLAP = 20
NODE_CHUNK_SIZE = 100
NODE_CHUNK_OVERLAP = 20
LLM_MODEL_MODEL = "gpt-4"
LLM_MAX_TOKENS = 2000
EMBED_MODEL_MODEL = "text-embedding-ada-002"
CONTEXT_WINDOW = 512
Metadata variables
PRODUCT_NAME = "myBot"
DATA_VERSION = "v0.0.1"
USERNAME = "username"
RELEASE_STATUS = "test"  # test, dev, prod
set up OpenAI API
OPENAI_API_KEY = os.environ["OPENAI_API_KEY"]
Set up Pinecone - assumes you have an index already created
PINECONE_INDEX_NAME = "s13"
PINECONE_ENVIRONMENT = "gcp-starter"
PIENCONE_API_KEY = os.getenv("PINECONE_API_KEY")
pinecone.init(api_key=PIENCONE_API_KEY, environment=PINECONE_ENVIRONMENT)
pinecone_index = pinecone.Index(PINECONE_INDEX_NAME)
Set up the vector store
vector_store = PineconeVectorStore(pinecone_index=pinecone_index)
Set up the storage context
storage_context = StorageContext.from_defaults(vector_store=vector_store)
set the LLM
llm = OpenAI(
temperature=0.1,
model=LLM_MODEL_MODEL,
max_tokens=LLM_MAX_TOKENS,
api_key=OPENAI_API_KEY,
)
Define metadata
metadata = {
"product_name": PRODUCT_NAME,
"data_version": DATA_VERSION,
"username": USERNAME,
"release_status": RELEASE_STATUS,
}
Set up the text splitter
text_splitter = TokenTextSplitter(
separator=" ",
chunk_size=SPLITTER_CHUNK_SIZE,
chunk_overlap=SPLITTER_CHUNK_OVERLAP,
backup_separators=["\n"],
)
Set up the metadata extractor
The MetadataExtractor class is used to extract metadata from TextNodes.
metadata_extractor = MetadataExtractor(
extractors=[
TitleExtractor(nodes=5, llm=llm),
QuestionsAnsweredExtractor(questions=3, llm=llm),
EntityExtractor(
prediction_threshold=0.5,
#it will load the Entity Extractor on the gpu if it is available
device="cuda" if cuda_available else "cpu",
label_entities=True,
),
SummaryExtractor(summaries=["prev", "self"], llm=llm),
KeywordExtractor(keywords=10, llm=llm),
# CustomExtractor()
],
)
Set up the node parser
node_parser = SimpleNodeParser.from_defaults(
chunk_size=NODE_CHUNK_SIZE,
chunk_overlap=NODE_CHUNK_OVERLAP,
text_splitter=text_splitter,
include_metadata=True,
include_prev_next_rel=True,
metadata_extractor=metadata_extractor,
)
#Set up the embed model
embed_model=OpenAIEmbedding(
api_key=OPENAI_API_KEY,
temperature=0.1,
model=EMBED_MODEL_MODEL
)
Set up the prompt helper
prompt_helper=PromptHelper(
context_window=CONTEXT_WINDOW,
)
#set up ServiceContext
service_context = ServiceContext.from_defaults(
llm=llm,
embed_model=embed_model,
node_parser=node_parser,
prompt_helper=prompt_helper,
)
set ServiceContext as global
set_global_service_context(service_context)
Loading the data
The SimpleDirectoryReader class is used to load a directory of documents.
documents = SimpleDirectoryReader(DOC_PATH).load_data()
Update the metadata for each document
for document in documents:
document.metadata.update(metadata)
Construct nodes from the document text
my_nodes = node_parser.get_nodes_from_documents(documents)
Generate embeddings for each node
for node in my_nodes:
node_embedding = embed_model.get_text_embedding(
node.get_content(metadata_mode="all")
)
node.embedding = node_embedding
node.metadata.update(metadata)
storage_context.vector_store.add(my_nodes)
index = VectorStoreIndex.from_documents(
documents,
service_context=service_context,
show_progress=True,
storage_context=storage_context,
)
From: dosubot[bot] @.>
Sent: Sunday, November 5, 2023 10:32 AM
To: run-llama/llama_index @.>
Cc: Nick Katsivelos @.>; Mention @.>
Subject: Re: [run-llama/llama_index] [Documentation]: Entity Extraction should use CUDA if available or at least surface the option (Issue #8685)
🤖
Sure, you can definitely submit your code for review. The LlamaIndex project encourages contributions from the community. You can make a pull request (PR) on the LlamaIndex Github page. Here are the steps you can follow:

Fork the LlamaIndex repository.
Clone the forked repository to your local machine.
Create a new Python virtual environment using poetry.
Install all dependencies needed for most local development with poetry install --with dev,docs.
Make your changes in the code.
Format and lint your changes with make format; make lint.
If your changes are significant, create a unit test in the tests folder and run all unit tests with pytest tests or make test.
If your changes involve entirely new features, consider adding an example Jupyter notebook to showcase this feature in the examples folder.
Commit your changes and push them to your forked repository.
Open a pull request against the main LlamaIndex repository.

Please refer to the project's CONTRIBUTING.md file for the most accurate and up-to-date information.
Regarding your suggestion about an explicit section in both the setup procedure and documentation for setting up CUDA when available, it's a great idea. However, I couldn't find any specific information on setting up CUDA for faster inference in the LlamaIndex data framework in the context provided. You can refer to the official documentation of LlamaIndex for more detailed information and guidance.
As for the similar solved issue, you might find the discussion on the issue [Feature Request]: Tools for tuning retrieval hyperparametershttps://github.com/run-llama/llama_index/issues/6531 helpful. It was last updated on September 27, 2023 and was referenced by [Discussion/Tracking]: Declarative Global Confighttps://github.com/run-llama/llama_index/issues/6883 which was last updated on October 19, 2023.
Sources

[Feature Request]: Tools for tuning retrieval hyperparametershttps://github.com/run-llama/llama_index/issues/6531
CONTRIBUTING.mdhttps://github.com/run-llama/llama_index/blob/main/CONTRIBUTING.md
README.mdhttps://github.com/run-llama/llama_index/blob/main/README.md

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bothttps://github.com/dosu-bot.
—
Reply to this email directly, view it on GitHubhttps://github.com/run-llama/llama_index/issues/8685#issuecomment-1793769353, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAMGI3FLAR7JUZPKRY4S573YC6WOZAVCNFSM6AAAAAA6527NM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJTG43DSMZVGM.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

Thanks for the suggestion! We should definitely make the documentation clearer on this.
nickknyc Would you like to make PR to update this directly? Should be super straightforward!

on my todo list