envoy's least request load balancing not working perfectly when there are multiple envoy pods

Title: Envoy's load balancing not working perfectly when there are multiple envoy pods
Description:
We have a hard limit of 2 on the maximum number of active requests that can be served by a backend application pod(ms-dummy-suki-asr-v2 in this case). With this envoy config and only one envoy pod, least request load balancing works fine:
kind: ConfigMap
metadata:
  name: envoy-ms-dummy-suki-asr-v2-conf
data:
  envoy.yaml: |
    node:
      id: "nvidia"
      cluster: "suki-dev"

    static_resources:
      listeners:
      - address:
          socket_address:
            address: <IP_ADDRESS>
            port_value: 443
        filter_chains:
        - filters:
          - name: envoy.http_connection_manager
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
              access_log:
              - name: envoy.file_access_log
                typed_config:
                  "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
                  path: "/dev/stdout"
              codec_type: AUTO
              stat_prefix: ingress_https
              route_config:
                name: local_route
                virtual_hosts:
                - name: https
                  domains:
                  - "*"
                  retry_policy:
                    num_retries: 10
                  routes:
                  - match:
                      prefix: "/"
                    route:
                      cluster: ms-dummy-suki-asr-v2
                      timeout: 300s
                    response_headers_to_add:
                      - header:
                          key: "x-content-type-options"
                          value: "nosniff"
                        append: true
                      - header:
                          key: "x-frame-options"
                          value: "deny"
                        append: true
                      - header:
                          key: "x-xss-protection"
                          value: "1; mode=block"
                        append: true
                      - header:
                          key: "strict-transport-security"
                          value: "max-age=63072000; includeSubDomains; preload;"
                        append: true
                      - header:
                          key: "content-security-policy"
                          value: "default-src 'none'; frame-ancestors 'none';"
                        append: true
              http_filters:
              - name: envoy.health_check
                typed_config:
                  "@type": type.googleapis.com/envoy.extensions.filters.http.health_check.v3.HealthCheck
                  pass_through_mode: false
                  headers:
                  - name: ":path"
                    exact_match: "/healthz"
                  - name: "x-envoy-livenessprobe"
                    exact_match: "healthz"
              - name: envoy.filters.http.router
                typed_config:
                  "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
          transport_socket:
            name: envoy.transport_sockets.tls
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
              common_tls_context:
                tls_certificates:
                  - certificate_chain:
                      filename: "/etc/ssl/envoy/tls.crt"
                    private_key:
                      filename: "/etc/ssl/envoy/tls.key"
                alpn_protocols: ["h2", "http/1.1"]
      clusters:
      - name: ms-dummy-suki-asr-v2
        connect_timeout: 0.5s
        type: STRICT_DNS
        dns_lookup_family: V4_ONLY
        lb_policy: LEAST_REQUEST
        least_request_lb_config: {"choice_count": 10000}
        common_http_protocol_options: {"idle_timeout": 0.1s}
        http2_protocol_options: {}
        load_assignment:
          cluster_name: ms-dummy-suki-asr-v2
          endpoints:
          - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: ms-dummy-suki-asr-v2.microservices.svc.cluster.local
                    port_value: 10001
    admin:
      access_log_path: "/dev/stdout"
      address:
        socket_address:
          address: <IP_ADDRESS>
          port_value: 8090

Envoy's service and deployment yaml, for reference:
apiVersion: v1
kind: Service
metadata:
  name: envoy-ms-dummy-suki-asr-v2
  namespace: microservices
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  - name: admin-port
    port: 8090
    protocol: TCP
    targetPort: 8090
  selector:
    app: envoy-ms-dummy-suki-asr-v2
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    suki.ai/owner: '@DevOps'
  labels:
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/name: envoy-ms-dummy-suki-asr-v2
    app.kubernetes.io/part-of: suki-Network
  name: envoy-ms-dummy-suki-asr-v2
  namespace: microservices
spec:
  replicas: 1
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: envoy-ms-dummy-suki-asr-v2
  template:
    metadata:
      annotations:
        linkerd.io/inject: disabled
        prometheus.io/scrape-envoy: "true"
      labels:
        app: envoy-ms-dummy-suki-asr-v2
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - envoy-ms-dummy-suki-asr-v2
            topologyKey: kubernetes.io/hostname
      containers:
      - env:
        - name: ENVOY_UID
          value: "0"
        image: envoyproxy/envoy:tools-v1.29.7
        livenessProbe:
          httpGet:
            httpHeaders:
            - name: x-envoy-livenessprobe
              value: healthz
            path: /healthz
            port: 443
            scheme: HTTPS
          initialDelaySeconds: 10
        name: envoy-ms-dummy-suki-asr-v2
        ports:
        - containerPort: 443
          name: https
        readinessProbe:
          httpGet:
            httpHeaders:
            - name: x-envoy-livenessprobe
              value: healthz
            path: /healthz
            port: 443
            scheme: HTTPS
          initialDelaySeconds: 3
        resources:
          limits:
            cpu: 7000m
            memory: 4000Mi
          requests:
            cpu: 7000m
            memory: 4000Mi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - any
          readOnlyRootFilesystem: true
        volumeMounts:
        - mountPath: /etc/envoy
          name: config
        - mountPath: /etc/ssl/envoy
          name: certs
      volumes:
      - name: certs
        secret:
          secretName: letsencrypt-suki-dev-tls-digicert
      - configMap:
          name: envoy-ms-dummy-suki-asr-v2-conf
        name: config

The above config with 1 envoy replica works fine, and I am able to create 600 concurrent active requests from our test client. However when I increase the number of envoy pods to 2, the load balancing does not work properly anymore, and I get a few errors due to envoy not sending the new request to the pod with the least number of active requests. How can I fix this to make it work for multiple envoy pods as well for high availability?
[optional Relevant Links:]

Any extra documentation required to understand the issue.

cc @wbpcode @tonya11en @nezdolik

Can you frame this in the following way:

Describe exactly what you expected to happen. What does a system that is "working perfectly" look like in the different scenarios you mention (1 vs. 2 pods)? What do you expect the load balancer to do exactly when it is sending requests and which upstream endpoints would receive them? An example would be ideal for all these cases, so that we can make sure we don't talk past each other.
Describe what exact behavior you are observing and how this differs from what you expect to be the correct behavior.
Write out what you're trying to convey with the YAML you pasted in the description. What are the containers that are in the request path, where are the envoys, and which envoys are configured with LEAST_REQUEST load balancing?
What do you mean by "envoy pods" in your description? Are these sidecars attached to some downstream service or upstream service? Do both services have envoy sidecars? How do you know which upstream endpoints are being chosen by the load balancer?

This would be a good starting point for us to make sure we agree on expected behavior and to verify that something anomalous is actually happening.

I am creating 600 long-lived requests from a test client sequentially, the requests spaced 10ms apart. There are 300 backend pods with 1 GPU each. Each pod can serve a max of 2 concurrent requests. This restriction is a hard limit set in the application code. In this case, a system that is working perfectly would mean that for any new request that comes to envoy, the request is sent to a pod with the least number of active requests, and there are no failures observed for any of the requests. Success for a request means that it was able to find a pod which was not already serving 2 requests. Failure for a request means that it was not able to find a pod which was not already serving 2 requests.

When I test for the above scenario(300 backend pods, and 600 requests with a 10ms interval) with 1 envoy pod, I almost always get success for all the 600 requests. However, when I do the same test with 2 envoy pods, I consistently get 5 to 10 errors in the requests.

In the yamls shared above, envoy-ms-dummy-suki-asr-v2-conf is the configmap that is used for envoy. ms-dummy-suki-asr-v2 is the cluster name in the envoy config and this represents the backend GPU-service with 300 pods. This cluster is also configured to use the LEAST_REQUEST load balancing with a choice_count of 10000. envoy-ms-dummy-suki-asr-v2 is the name of both the service behind which the envoy pods are present as well as envoy's deployment.

The envoy pods are not sidecars. None of the services have sidecars. They have their own service and deployment - both called envoy-ms-dummy-suki-asr-v2. Using the configmap that resides on the envoy pods, the traffic is routed to the backend service - ms-dummy-suki-asr-v2. There is a downstream service called ms-asr-manager which calls the envoy's service endpoint to send traffic to the backend service ms-dummy-suki-asr-v2. To check which pods the requests are going to, I am checking envoy admin's /clusters endpoint.

Awesome, this is a great description of what's going on. Thanks.
The setup
Stating my understanding of the situation, so correct me if any of this is wrong. You've got:

1 test client
N envoys, with least request LB, with an enormous choice count.
300 backends (each with a GPU)

You ran this test with N=1 and N=2 envoys. In the N=1 case you've got no errors, as expected. In the N=2 case, you expected 0 errors, but you consistently get 5-10 requests that return error.

Given the understanding I have above, I'm wondering:
If you have multiple envoys, why do you expect 0 errors when they don't share the same load balancer state? For example, if 1 envoy sends 2 in-flight requests to a backend, why would we expect another envoy to avoid sending to that same backend and getting an error?
I would expect errors in the N=2 case you've presented and it's not really something you can avoid even if you configure retries. The N=1 case has global understanding of the in-flight requests, so you shouldn't get any errors in that case.
To avoid errors entirely, you would need a way to share that in-flight request state across multiple backends. That's hard, not very scalable, and would require a custom load balancer extension.
Would it be feasible to queue requests at the backends? If you do this, all the load balancers would keep the queue lengths balanced and you'd likely avoid these issues entirely.

Would it be feasible to queue requests at the backends? If you do this, all the load balancers would keep the queue lengths balanced and you'd likely avoid these issues entirely.

@tonya11en what exactly is queueing requests at backend?
Are you referring to circuit breaker configurations of max_connections and max_pending_requests configurations (as mentioned here)or is there some other dedicated configuration support queuing connections in envoy?

I was referring to the backend application that is using the GPUs. Instead of immediately failing a request if it is already serving 2 concurrent requests, would you be able to form a queue instead?
If you are able to queue requests, the load balancers will keep their lengths comparable without the need for synchronization. Otherwise, there's not really a way to make multiple envoys coordinate their load balancing decisions because they don't know how many in-flight requests there are on a backend.

I was referring to the backend application that is using the GPUs. Instead of immediately failing a request if it is already serving 2 concurrent requests, would you be able to form a queue instead?
If you are able to queue requests, the load balancers will keep their lengths comparable without the need for synchronization. Otherwise, there's not really a way to make multiple envoys coordinate their load balancing decisions because they don't know how many in-flight requests there are on a backend.

That's not possible as the processing needs to happen in real time. The connections have to be established ASAP.
Just curious to know why would a custom load balancer extension not scale?


Learning Rate stays at 0.0?

The training loss is consistently is 0.0 - I'm messing something up either in the parameters or because of the model I am trying to fine tune. I started with my own dataset and then thought perhaps there was a problem in the dataset format (there is one column, text with instruction/response).  --data-path tatsu-lab/alpaca  dataset and had the same challenge - no training loss.  What is missing? Thank you.

--text-column text \
--lr ${LEARNING_RATE} \
--batch-size ${BATCH_SIZE} \
--epochs ${NUM_EPOCHS} \
--block-size ${BLOCK_SIZE} \
--warmup-ratio ${WARMUP_RATIO} \
--lora-r ${LORA_R} \
--lora-alpha ${LORA_ALPHA} \
--lora-dropout ${LORA_DROPOUT} \
--weight-decay ${WEIGHT_DECAY} \
--gradient-accumulation ${GRADIENT_ACCUMULATION} \
$( [[ "$USE_FP16" == "True" ]] && echo "--fp16" ) \
$( [[ "$USE_PEFT" == "True" ]] && echo "--use-peft" ) \
$( [[ "$USE_INT4" == "True" ]] && echo "--use-int4" )
# $( [[ "$PUSH_TO_HUB" == "True" ]] && echo "--push-to-hub --token ${HF_TOKEN} --repo-id ${REPO_ID}" )

Here is my input (NOTE: the model was picked because of its small size):
project_name = 'test_tune' # @param {type:"string"}
model_name = 'EleutherAI/pythia-70m' # @param {type:"string"}

#@markdown ---
#@markdown #### Hyperparameters
learning_rate = 2e-4 # @param {type:"number"}
num_epochs = 2  #@param {type:"number"}
batch_size = 1 # @param {type:"slider", min:1, max:32, step:1}
block_size = 1024 # @param {type:"number"}
trainer = "sft" # @param ["default", "sft"] {type:"raw"}
warmup_ratio = 0.1 # @param {type:"number"}
weight_decay = 0.01 # @param {type:"number"}
gradient_accumulation = 4 # @param {type:"number"}
use_fp16 = True # @param ["False", "True"] {type:"raw"}
use_peft = True # @param ["False", "True"] {type:"raw"}
use_int4 = True # @param ["False", "True"] {type:"raw"}
lora_r = 16 #@param {type:"number"}
lora_alpha = 32 #@param {type:"number"}
lora_dropout = 0.05 #@param {type:"number"}

and the output:
> INFO    Running LLM
> INFO    Params: Namespace(version=False, train=True, deploy=False, inference=False, data_path='tatsu-lab/alpaca', train_split='train', valid_split=None, text_column='text', model='EleutherAI/pythia-70m', learning_rate=0.0002, num_train_epochs=2, train_batch_size=1, warmup_ratio=0.1, gradient_accumulation_steps=4, optimizer='adamw_torch', scheduler='linear', weight_decay=0.01, max_grad_norm=1.0, seed=42, add_eos_token=False, block_size=1024, use_peft=True, lora_r=16, lora_alpha=32, lora_dropout=0.05, logging_steps=-1, project_name='test_tune', evaluation_strategy='epoch', save_total_limit=1, save_strategy='epoch', auto_find_batch_size=False, fp16=True, push_to_hub=False, use_int8=False, model_max_length=1024, repo_id=None, use_int4=True, trainer='default', target_modules=None, merge_adapter=False, token=None, backend='default', username=None, func=<function run_llm_command_factory at 0x7d4fc9b775b0>)
...
Extracting data files: 100% 1/1 [00:00<00:00, 1167.35it/s]
Generating train split: 52002 examples [00:00, 313492.65 examples/s]
Using pad_token, but it is not set yet.
You are resizing the embedding layer without providing a `pad_to_multiple_of` parameter. This means that the new embedding dimension will be 50277. This might induce some performance reduction as *Tensor Cores* will not be available. For more details about this, or help on choosing the correct value for resizing, refer to this guide: https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html#requirements-tc
Running tokenizer on train dataset:  12% 6000/52002 [00:01<00:13, 3400.38 examples/s]Token indices sequence length is longer than the specified maximum sequence length for this model (1034 > 1024). Running this sequence through the model will result in indexing errors
Running tokenizer on train dataset: 100% 52002/52002 [00:19<00:00, 2604.23 examples/s]
Grouping texts in chunks of 1024 (num_proc=4): 100% 52002/52002 [00:10<00:00, 4756.07 examples/s]
> INFO    creating trainer
{'loss': 0.0, 'learning_rate': 0.0, 'epoch': 0.8}
{'loss': 0.0, 'learning_rate': 0.0, 'epoch': 1.6}
{'train_runtime': 1085.9721, 'train_samples_per_second': 10.315, 'train_steps_per_second': 2.578, 'train_loss': 0.0, 'epoch': 2.0}
100% 2800/2800 [18:05<00:00,  2.58it/s]
> INFO    Finished training, saving model...

you should probably disable peft with pythia 70m


HIVE-28219: Support drop partitions by names in IMetaStoreClient

What changes were proposed in this pull request?
In thrift api definition, HMS support to drop partitions by both partition names and expressions. But current api in IMetaStoreClient only support drop partitions by expressions, we should add new api to support drop partitions by names.
  // IMetaStoreClient.java
  List<Partition> dropPartitions(String catName, String dbName, String tblName,
                                 RequestPartsSpec partsSpec, PartitionDropOptions options)
      throws NoSuchObjectException, MetaException, TException;

Why are the changes needed?
To improve the dropPartitions function.
Does this PR introduce any user-facing change?
Yes, user can use the new api to drop partitions by names.
Is the change a dependency upgrade?
No.
How was this patch tested?
Add unit test:
mvn test -Dtest=org.apache.hadoop.hive.ql.metadata.TestHive -pl :hive-exec

This PR is followed https://github.com/apache/hive/pull/4905#discussion_r1464194112, @zhangbutao @dengzhhu653 @deniskuzZ pls take a look in your convenience.

Does drop partitions by names have any advantages over drop partitions by expressions?
My concern is that we already have lots of redundant api and other components(Spark/Trino/Impala) may not need the new added api.
But if the new added api has some advantages over old api, i think we can move on.
I also want to listen to other folks's opinions. :)

Does drop partitions by names have any advantages over drop partitions by expressions?

drop partitions by names is not aimed to replace drop by expressions, they each serve distinct purpose.
And it's more easy to use and do not need deserialization in server side compared with expressions.

Does drop partitions by names have any advantages over drop partitions by expressions?
My concern is that we already have lots of redundant api and other components(Spark/Trino/Impala) may not need the new added api.
But if the new added api has some advantages over old api, i think we can move on.
I also want to listen to other folks's opinions. :)

+1
This change looks like a variant of dropPartitions based on one current implementation, it's not a good choice to declare it as a first class API.
If we really need to add it in the future, I would prefer making it an util method instead in MetaStoreUtils

This change looks like a variant of dropPartitions based on one current implementation, it's not a good choice to declare it as a first class API.

It's more like a function complement, before this PR, hive client can only drop partitions by exprs, but actually the server side supports drop partitions by names.


Remove the usage of io/ioutil package

/kind bug
/king cleanup
What happened?
pkg/fibrechannel/fibrechannel.go:21:2: SA1019: "io/ioutil" has been deprecated since Go 1.16: As of Go 1.16, the same functionality is now provided by package io or package os, and those implementations should be preferred in new code. See the specific function documentation for details. (staticcheck)
	"io/ioutil"
	^
pkg/driver/mount_test.go:17:2: SA1019: "io/ioutil" has been deprecated since Go 1.16: As of Go 1.16, the same functionality is now provided by package io or package os, and those implementations should be preferred in new code. See the specific function documentation for details. (staticcheck)
	"io/ioutil"
	^
pkg/driver/sanity_test.go:6:2: SA1019: "io/ioutil" has been deprecated since Go 1.16: As of Go 1.16, the same functionality is now provided by package io or package os, and those implementations should be preferred in new code. See the specific function documentation for details. (staticcheck)
	"io/ioutil"
	^

Mentioned package is deprecated, let's replace with other packages like io, os
What you expected to happen?
How to reproduce it (as minimally and precisely as possible)?
Anything else we need to know?:
Environment

Kubernetes version (use kubectl version):
Driver version:

/assign


docs: adds Props import to cell-component example

Description
Adds Props import to cell component example in admin/components
Before:

After:

[x] I have read and understand the CONTRIBUTING.md document in this repository.

Type of change

[x] Chore (non-breaking change which does not add functionality)
[x] This change requires a documentation update

Checklist:

[x] Existing test suite passes locally with my changes
[x] I have made corresponding changes to the documentation

@JarrodMFlesch All set.


ARM64 Build

Hello,
is there any plan to create ARM64v8 images of this project?.
They are only amd64 as far as I've seen and it would be useful for all of us having Raspberry Pi 4 (or clusters of it).
Thank you!

Any news?

Any word on an arm64 build? @jlesage

Yes, work still in progress!

Need ARM32 V7 as well

Images version 4.0.0 are now multi-arch.


[TACHYON-1240] Improved logging message formats in tachyon-underfs-* modules

https://tachyon.atlassian.net/browse/TACHYON-1240

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Tachyon-Pull-Request-Builder/6687/
Test PASSed.

Thanks for the improvement! LGTM.

I just had a minor question about the new syntax and exception arguments. Thanks!

@gpang That was a hint from @apc999 in another PR with a link to the documentation of SLF4J. The documentation states "...supports parametrization in the presence of an exception, assuming the exception is the last parameter". This will work with version 1.6.0. Tachyon uses version 1.7.2.

LGTM


Make R work again

I added a script that fixes three problems:

The multiple inheritance issue: https://github.com/swig/swig/issues/1867
Garbage collection. Now if we force SGObject destructor to print to stdout and clear R's local variables we can see them  deleted. This did not work before. It was done by attaching a "finalizer" at the function that wraps pointers in SEXP (R objects). Originally this was only done for wrapped functions of constructors.

rm(list=ls())
gc()
[09/07/20 09:40:14 info] SGObject destroyed (0x1b1fe9e0)
...

Handling memory allocation when casting https://github.com/swig/swig/issues/1660. This is more or less copy pasted with small modifications from the generated python bindings. If we only remove the assertion (that previously caused the error) and run valgrind, it will find lots of "definitely lost" memory leaks. Now no "definitely lost" memory leaks exist (although valgrind finds an error when attaching the library, but as far as I can tell it is not related to shogun + finds "possibly lost" similar to cpp example version).

@theartful awesome work!
@gf712 @karlnapf any objections to merge? the failures are unrelated....

@theartful looks great!

@karlnapf @vigsterkr can I merge this so that for now we have a functional R interface?

@theartful thanks for this!

I think there was a Jupyter notebook failing that used "Statistics/Math" classes. I will make another PR to either fix this or readd them to swig,


Server platform is broken

Godot version:
master at 4b4ed9b72
OS/device including version:
Debian 9
Issue description:
Server platform was working ok in commit c90f18e3f but now it outputs this:
ERROR: initialize: AudioDriverManager: all drivers failed, falling back to dummy driver
   At: servers/audio_server.cpp:153.
handle_crash: Program crashed with signal 11
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[1] /lib/x86_64-linux-gnu/libc.so.6(+0x33030) [0x7f127b4e4030] (??:0)
[2] ./linux_server.64() [0x1623208] (??:?)
[3] ./linux_server.64() [0x1629c33] (??:?)
[4] ./linux_server.64() [0x423238] (??:?)
[5] ./linux_server.64() [0x408509] (??:?)
[6] ./linux_server.64() [0x4175e6] (??:?)
[7] ./linux_server.64() [0x41520f] (??:?)
[8] ./linux_server.64() [0x407fef] (??:?)
[9] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f127b4d12b1] (??:0)
[10] ./linux_server.64() [0x407eea] (??:?)
-- END OF BACKTRACE --
Abortado

I've just tested this on Ubuntu and it didn't crash for me, can you try compiling with target=debug? Is backtrace different with that?


Plugin promotions, deprecations and deletions

This does not move the slice plugin yet, since we have at least one serious fix to land before we move it.

@djcarlin , the healthcheck plugin will be gone in 10, replaced by statichit

@djcarlin , the healthcheck plugin will be gone in 10, replaced by statichit

Well, no, deprecated in ATS v10, removed in v11. We strongly recommend people moving over to statichit, and file issues and requests to add whatever features statichit may be missing.

@djcarlin , the healthcheck plugin will be gone in 10, replaced by statichit

Well, no, deprecated in ATS v10, removed in v11. We strongly recommend people moving over to statichit, and file issues and requests to add whatever features statichit may be missing.

My mistake. It is deprecated, not removed in 10.

In doc/admin-guide/plugins/index.en.rst, we need to move the promoted plugins to the right section.
Everything else looks fine.

We should add a doc for statichit later on.

@shukitchan got merge conflicts again of course, so need a new review please :).


Change missing secret name log level to V(3)

This is a proposal of changing a log entry from warning to V(3).Info. The motivation of this change is that a missing secret name on Spec.TLS is a valid configuration.

This change is

/lgtm

Coverage decreased (-0.03%) to 43.955% when pulling 4453a06caafb1d1cf72b0b1ec31d0da70c13e831 on jcmoraisjr:jm-fix-noisy-log into ce5e3999ab3186b134645fc7c6184c5a26696ced on kubernetes:master.


update

Problem solved
Short description of the bug fixed or feature added

PR-Codex overview
This PR focuses on enhancing the functionality and testing of media type resolution, wallet icons, and clipboard hooks in the thirdweb package. It introduces new utility functions, improves existing tests, and ensures better handling of media URIs and wallet icons.
Detailed summary

Added resolveMediaTypeFromUri function for media type resolution.
Created tests for resolveMediaTypeFromUri and hasSmartAccount.
Introduced getQueryKey function for generating query keys in NFT components.
Enhanced wallet icon management with new social icons and tests.
Improved clipboard hooks tests for functionality and timing.
Added structural sharing tests for data comparison.
Updated MIME type resolution tests for better coverage.

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#5825  👈 (View in Graphite)
#5605
main

How to use the Graphite Merge Queue
Add either label to this PR to merge it via the merge queue:

merge-queue - adds this PR to the back of the merge queue
hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.


Layout presets do not render in starting points

Exporting site content to be used in a starting point, if a layout preset has been used as defined in the page_theme getThemeAreaLayoutPresets() function then this layout will not carry through when importing from a package with $pkgAllowsFullContentSwap = true;. Instead a full width area is added.
Assuming there must be something missing from the import routine, thus not excepting the markup for layout presets.
For example, here is the markup from a layout created manually:
	<block type="core_area_layout" name="">
		<arealayout type="theme-grid" columns="12">
			<columns>
				<column span="6" offset="0">
					<block type="content" name="">
						<data table="btContentLocal">
							<record>
								<content><![CDATA[
some content
]]></content>
							</record>
						</data>
					</block>
				</column>
				<column span="6" offset="0">
					<block type="content" name="">
						<data table="btContentLocal">
							<record>
								<content><![CDATA[
some content
]]></content>
							</record>
						</data>
					</block>
				</column>
			</columns>
		</arealayout>
	</block>
</blocks>

Andy then one created with a layout preset:
	<block type="core_area_layout" name="">
		<arealayout type="custom" preset="theme_frame_two_column_collapse">
			<columns>
				<column>
				  <block type="content" name="">
				    <data table="btContentLocal">
				      <record>
				        <content><![CDATA[
some content
]]></content>
				      </record>
				    </data>
				  </block>
				</column>
				<column>
				  <block type="content" name="">
				    <data table="btContentLocal">
				      <record>
				        <content><![CDATA[
some content
]]></content>
				      </record>
				    </data>
				  </block>
				</column>
			</columns>
		</arealayout>
	</block>
</blocks>

Notice there appears to be no column span available in the preset version. Furthermore, I believe by even adding this span manually to the content.xml file does not resolve the issue.

Have been looking at this. It's a bit confusing to me to see the import routine use a block controller, but as far as I can tell the task is to ensure the preset gets saved as if it was saved from the add/edit block concrete5 UI?
The code I think is preventing this is the addFromPost()`` method of the 'core_area_layout' block's controller.php```. To be specific it seems we need to satisfy the requirement to pass an object which satisfies an interface here..
$arLayout = PresetLayout::add($arLayoutPreset);
``
It's not clear to me how to build that interface when I look at the ```presetInterface.php```. If someone can give me a steer here, whether I'm on the right track (or not) and some tips on how to proceed, I feel we could pick this issue up.

I think the issue is that import/export of the layout presets was never fully implemented. Haven't done this yet but I think it'd be as simple as

Adding support for this export in src/Area/Layout/Layout.php in the export() method (note: the if statement in here is the reason that the type="custom" in the export block above.

Adding support for this "type" in the getImportData() method. That method contains no code that will let it work with the PresetLayout layout type, just the theme grid and custom layout types.

That'd probably be enough to make it work.

@aembler thanks for the info. Adding the type attribute of "preset" in the export method was relatively straightforward however the save routine within the core_area_layout block controller not so.
Adding an additional case to the switch statement to target type="preset" in the block controller also pretty straight forward but the actual routine to save the layout preset is proving difficult. Not sure if it is because the presets are not available as the theme is not currently the active theme but there doesn't seem like a way to run this.


[Web ui] Switching models to openvino, if guidance is left at non 1, program breaks, memory leaks.

While using webui and switching from lora models to openvino models and not manually setting guidance to 1, program execution is blocked at inference process and machine swap memory starts getting filled and pc blocks completely after short time or process is killed eventually by some linux service. [Linux/ I7 12700f].

Recommend guidance scale is 1.0. If you want to use negative prompt increase guidance but it will take more memory .


change axioms #1023

solve #1023 partly
remove wrong axiom 'has input' some 'ocean current'

Due to inconsistencies with domains/ranges of has energy participant, I removed two axioms. This should be adressed in #1023.


add ip.mpg.de - Planck Institute for Innovation and Competition

The Max Planck Institute for Innovation and Competition (ip.mpg.de) is a member of the Max Planck Society.
Website: https://www.ip.mpg.de/en/

@SbstnErhrdt First stage of verification passed, official domain is ip.mpg.de as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@SbstnErhrdt The institute looks purely research one. Please notice that the educational license is exclusively intended for educational purposes. Does the institute have any students (such as PhD students, for example) who study in the institute?

There are at least 30+
Here u go:
https://www.ip.mpg.de/en/persons.html?tx_epxipmpg_person[sword]=&tx_epxipmpg_person[personFieldOfAction]=54&tx_epxipmpg_person[researchArea1]=&tx_epxipmpg_person[researchArea2]=#form
https://www.ip.mpg.de/en/persons.html?tx_epxipmpg_person[sword]=&tx_epxipmpg_person[personFieldOfAction]=2&tx_epxipmpg_person[researchArea1]=&tx_epxipmpg_person[researchArea2]=#form

@philipto

Thanks @philipto

@SbstnErhrdt Thank you for the clarification and your patience while waiting for a review and processing your request. Pull request merged.
Please notice that the educational license is exclusively intended for educational purposes. If any employees of the Institute who are not students or teachers need the license, please kindly instruct them to email<EMAIL_ADDRESS>to request an academic discount for a paid license.


ci: update build

Updates GH Actions versions
Updates maturin version
Adds musl linux wheels
Updates pyproject.toml where it diverged from spec (https://packaging.python.org/en/latest/guides/writing-pyproject-toml/)

All looks good to me. I have created a minor release branch to merge this into. Do you want to remove the section referred to above and then I will merge?


Logitech Additions and General Docs

Adding LGS to docs, adding general troubleshooting guide

Er... I buggered something up....


Use $exceptionHandler instead of logging directly to the console. Thi…

…s makes mocking easier, and also provides us with hooks for easily adding on additional exception handling, such as logging errors to some external location.

Implemented tokenized search

Ready for publish.


What about docs?

Let's work on documentation with suggestions:

[ ] #7
[ ] Create a docs folder

I wonder what we should do for this

[ ] Create a docs folder
[ ] Create a newindex.md document

Thanks. I created the issues from.the first item


Link with TS

enhance Class or add sub class to write the whitelist on the server because of the limit of the TS3 server when ip is not on whitelist


Add support for custom entries

:+1: Thank you for the contribution! I'll include this in the next release.


Wrong ruleset returned with Eval method

While QA'ing the v0.6 I encountered an annoying bug:
Let A a ruleset at path a, let AB a ruleset at path a/b:

Create ruleset A: version XYZ
Create ruleset AB: version JKL

In etcd:
/a/XYZ
/a/b/JKL

The store Latest method, which is used by the Eval endpoint, when called with the former path returns the latter.
rs, err := store.Latest(ctx, "/a") // returns /a/b

Maybe this one should also be applied on 0.7.
To verify.

The problem here came because of the use of / as a separator between the path and the version.
It shouldn't be a problem anymore normally thanks to your change of the separator to !


In repo check

This prevents 7 commands from blowing up because they're not run from inside of a repository. It also makes a couple slight tweaks to the integration test suite:

The verbose logs at the end of a failure (server logs and your local env) are ONLY shown from script/cibuild. We care about this output on CI servers, but probably not on our local machines.
The trace file is truncated after every individual test. This way a test failure will show just its own git trace log, not the trace log for every test that has run in the file.

My only comment concerns moving the whole "check and exit if not in repo" business into a func and reusing that code in several of the commands.  Looks like some tests are failing as well, which I'd like to look at.

I don't get the failures on my mac. I'll try in a docker image in a bit. Seems to be a difference in the exit code that's returned.

Okay, I figured the problem out.
   trace git-lfs: run_command: 'git' config -l
   trace: built-in: git 'config' '-l'
   trace git-lfs: run_command: 'git' config -l -f /home/travis/gopath/src/github.com/github/git-lfs/.git/config
   trace: built-in: git 'config' '-l' '-f' '/home/travis/gopath/src/github.com/github/git-lfs/.git/config'

It's running from inside the github/git-lfs directory, so it's not crashing. The tests pass for me because I set GIT_LFS_TEST_DIR=/Users/rick/p/git-lfs-temp for my tests.


Fix transformer execute action-request

Fix malformed action-request contract that is
executed on transformer match
Change-type: patch
Signed-off-by: Josh Bowling<EMAIL_ADDRESS>

@balena-ci rebase

@balena-ci I self certify!


TODO keyword moving needs checks at table edges

I really like this kanban implementation! It naturally extends the org environment. Thanks!
It works as it is meant for, but looks like the code does not check for edge conditions: If I try to move TODO keywords past first or last table column, unexpected things happen: The table line disappears or cursor moves to the previous line.

Thanks for the issue!
Yes ... I also saw this ... I think it comes from the fact, that if you toggle through the todo states of an org-todo item, you also loose the todo (thing). But I think I will fix this ... one can always jump to the todo and delete the TODO tag if thats really needed ... lets see if i find some free time!

This fix works perfectly for the left edge, but the point ends up in the wrong line at the other edge.
When the direction is right and the new state is in the last column, the point ends up one line up, except when it is on the first line.
When the direction is left and old state was in the last column, the point ends up one line down.
This leads to a curious game where you can repeatedly press the j and k keys and which ends when all the line states are at penultimate column and point ends up past the table which leads to an error: goto-char: Search failed: "[["
:)

Hi @heikkil, I tried to reproduce the issue now with a clean emacs config in an alpine docker container as described in the readme.org. unfortunately its not failing for me as in your description. could you elaborate more?

OK. It is something in my config, but I am having trouble pinpointing the cause. Sorry for the noise.

no problem ... perhaps its an old version of org-kanban


Drop github.com/Microsoft/hcsshim/test

This package drags in old versions of a bunch of things transitively.
Do we really need this client_windows_test.go? if we do then we should
have it in a separate directory and with its own go.mod/go.sum files to
avoid cluttering the root go.mod/sum
Signed-off-by: Davanum Srinivas<EMAIL_ADDRESS>

We still need client_windows_test.go, as it defines the base images used by the tests, and a few other constants needed by the integration test suite.
If you want to get rid of the use of github.com/Microsoft/hcsshim/test, then what you actually need to do is find another source for rsrc_amd64.syso, as that's the only thing we use from there.
The simplest thing would be to just copy https://github.com/microsoft/hcsshim/tree/master/test/functional/manifest/ into the containerd test suite, the reason I didn't do that previously (#4431) was simply that I didn't want to deal with copyright provenance of a binary blob, and hadn't had time to look into rsrc and //go:generate to make it more copyable.
Alternatively, for the "drags in old versions" concern, perhaps just update the version of github.com/Microsoft/hcsshim/test? I recently revendored containerd into github.com/Microsoft/hcsshim/test with 1.5.0-beta1 so it should be must less "old versions" than it was, and it would 100% make sense to me if containerd's go.mod had a replaces for any uses of github.com/containerd/containerd to be ., since it makes no sense to me for depended-upon modules when building containerd to themselves depend on different containerd.

You will need to change the integration test runner setup in the Makefile, which currently runs github.com/containerd/containerd, to run github.com/containerd/containerd/test/client, as with this PR currently, no tests run in the Integration stages.
This might be a simple change to the value of INTEGRATION_PACKAGE, plus changing the integration: rule to run ${INTEGRATION_PACKAGE} explicitly.
And then, because it appears go list ${GO_TAGS} ./... doesn't cross module boundaries, all the $(filter-out ${INTEGRATION_PACKAGE},XXX) behaviour could be just XXX.
That would certainly be a nice improvement, in my opinion, as I've often gotten lost trying to distinguish integration tests from other tests in the Makefile.

@TBBle thanks for the tips! trying it out now

@TBBle now i have just the windows CI job failing. all with similar pattern
2021-03-02T21:07:15.2282218Z     container_test.go:1139: : exec: "D:\\a\\containerd\\containerd\\src\\github.com\\containerd\\containerd\\integration\\client": file does not exist: unknown```

@TBBle any tips?

Would it be much simpler if rsrc_amd64.syso moves back to hcsshim main module? As it's just a syso, no extra dependency will be pulled.

@zhsj i think separating out these tests is definitely a worthy goal as make integration currently runs all tests AFAICT

make integration should only run the containerd integration tests, i.e. the ones in the github.com/containerd/containerd package that you're looking at moving, not the unit tests or CRI integration tests, which are all in subdirectories and hence match github.com/containerd/containerd/....
You can see the difference in the build logs for the steps in the master HEAD CI run.

@TBBle how about the failures on windows, is that related to test.root too?

@TBBle how about the failures on windows, is that related to test.root too?

No, that would lead to running no tests, not failing tests. I think the 'is root' check is skipped on Windows, from memory, I'm not sure why.

@TBBle the integration CI jobs on linux seem to be running the right things now. need to dig into the windows one now

I grabbed your branch, and locally, just running one of the integration tests directly seems to work:
> $ENV:PATH = "C:\Users\paulh\go\src\github.com\containerd\containerd\bin;${ENV:PATH}"
> which containerd
C:\Users\paulh\go\src\github.com\containerd\containerd\bin\containerd.exe
> go test -"test.root" -"test.run" TestContainerStart
Windows test image: mcr.microsoft.com/windows/nanoserver:20H2 , Windows build version: 19042
time="2021-03-04T02:29:47+11:00" level=info msg="running tests against containerd" revision=6f2b9dd35123930c9fa478d5123438e48f76f40e runtime= version=v1.5.0-beta.2-13-g6f2b9dd35.m
time="2021-03-04T02:29:47+11:00" level=info msg="start to pull seed image" image="mcr.microsoft.com/windows/nanoserver:20H2"
PASS
ok      github.com/containerd/containerd/integration/client     27.458s

If I run all of them, TestContainerHook fails due to lack of ps.exe on my local system, and with that commented out, the suite passes. The error at the end is because I had a shell open in the directory it was trying to clean up.
C:\Users\paulh\go\src\github.com\containerd\containerd\integration\client> go test -"test.root" .
Windows test image: mcr.microsoft.com/windows/nanoserver:20H2 , Windows build version: 19042
time="2021-03-04T02:42:05+11:00" level=info msg="running tests against containerd" revision=6f2b9dd35123930c9fa478d5123438e48f76f40e runtime= version=v1.5.0-beta.2-13-g6f2b9dd35.m
time="2021-03-04T02:42:05+11:00" level=info msg="start to pull seed image" image="mcr.microsoft.com/windows/nanoserver:20H2"

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms
The command line is too long.
Reply from ::1: time<1ms
Reply from ::1: Reply from ::1: time<1ms
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
PASS
failed to remove test root dir hcsshim::DestroyLayer - failed failed in Win32: The process cannot access the file because it is being used by another process. (0x20)
exit status 1
FAIL    github.com/containerd/containerd/integration/client     133.918s

And then to be sure, I tested with make integration, after closing the afore-mentioned shell.
C:\Users\paulh\go\src\github.com\containerd\containerd> C:\msys64\mingw64\bin\mingw32-make.exe integration
Access is denied.
""+" integration"
Windows test image: mcr.microsoft.com/windows/nanoserver:20H2 , Windows build version: 19042
time="2021-03-04T02:54:46+11:00" level=info msg="running tests against containerd" revision=6f2b9dd35123930c9fa478d5123438e48f76f40e runtime= version=v1.5.0-beta.2-13-g6f2b9dd35.m
time="2021-03-04T02:54:46+11:00" level=info msg="start to pull seed image" image="mcr.microsoft.com/windows/nanoserver:20H2"

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms
The command line is too long.
Reply from ::1: time<1ms
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms

Pinging minwinpc [::1] with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
PASS
ok      github.com/containerd/containerd/integration/client     134.414s

So whatever is wrong, it's not reproducing for me here on Windows 10 20H2. I don't know if the difference relates to OS version, or something else in my environment.
I also can't see where in the code the error is coming from, i.e. what is exec and why is it trying to access the current working directory, which is why I was hoping to reproduce the problem locally.

recheck

recheck

/test containerd-build-arm64

recheck

recheck

recheck


Movable Camera pivot point Fix #265

I'm putting this here to gather feedback, it is a good proof of concept but has not been tested at all for bugs and omissions.
In particular I know CameraController.updateController() is broken, making the pose tab unusable.
However it demonstrates the feature's design :

Adds a single "pivot" vector to rotate around
Default behavior should be 100% consistent with existing scenes (to be tested).
double-click on a model sets the pivot point there, resets the offset

The whole thing should ideally be animated but this will be the subject of another PR to keep things simple.
I'll show this around a bit and clean it up in a few days/weeks once I'm confident enough about the feature design.
@oM4Uo feel free to test it and comment if you had something else in mind.

@sdumetz I'm really interested in trying and test your modifications but I wasnt able to build from source by following the https://smithsonian.github.io/dpo-voyager/introduction/installation/. The Docker container gives me an error and doesn't start (my Docker and in general my "coding" experience is poor, I'm sorry)
if you have time and willingness, and send me a build dist, I will be more than happy to try it and give you my feedback

@oM4Uo if you want to start another issue (or a discussion) and provide some more details on the error you are getting, I can try and help with the build issues.

@gjcope Thank you, but I was able to fix the issue just now. Specifically the container stops and print the error "Exited (127)"
I had to:

run this command "git config --global core.autocrlf false" (I'm on windows)
update the node.js used by the docker image; after several tests the one that worked for me was the version 16.15.1

@sdumetz As you wrote, double-clicking on the model resets the offset, and it works as expected. I also tested it on existing scenes, and it works. I also noticed that the 'orthographic' view is broken,
However, I was considering a different option, and for sure I didn't explain myself well the first time. Double-clicking on the model, or even a single click, should 'move' the point of rotation (around which the model rotates) to the spot that has been clicked. If I'm correct, right now the model rotates always around the center of the scene.
@gjcope, @sdumetz and all the people behind the prejoct I really thank you for your hard work

Hi @sdumetz , today I learned a little bit more about how tree.js (are you using that right?) scenes, cameras, and so on work, and now I totally understand why you are 'confused'... again, it was my lack of knowledge and I expressed myself badly, sorry.
So, if I understood correctly, in voyager the camera is orbiting around the center of the scene at (0,0,0) as shown in the next figure

What I meant is that when you double-click on a point on the model, the camera should 'target' that point (eventually zoom on it) and orbit around it, as shown in the next figure

As examples:

ATON and you could play with this model Nuraghe La Prisgiona  - in this case after double-click camera zoom on the point and orbit around it, double-click again and camera zoom a little bit more; to go back to the initial position ATON use the "Home" button that has a saved viewpoint attached (like the "center" button here in voyager)
View3D, look under the "Option" tab to find a model to play with - in this other one after double-click camera zoom on the point and orbit around it, double-click again bring the camera back to the initial position
Potree, even though it is primarily designed to handle point clouds after double-click camera zoom on the point and orbit around it

Where I'm using the word "zoom", I don't know if the camera simply changes the FOV, moves closer to the point, or a combination of both, or something else. However, I hope I don't mislead you again.
Thank you again, and I'm more than happy to help, test, and provide feedback. I'm sorry I can't assist with coding.
Cheers

Unless I'm mistaken, you describe the default Voyager behavior and what you need is in this PR. Are you sure you checked-out the right branch?
The main difference between my implementation and your drawings is that I set the pivot point on the model's center while I set it on its surface (where the user clicked in 2D space, projected into the scene).
I'm pretty confident pivoting on the model's  center not a good idea because it makes no sense for the user as soon as the center is far from where he clicked.

After my modification it's working (kind of) like the others. Before, it was always pivoting around the scene's (0,0,0) point, which might not be the same as the model's center: Models have a center point (mesh's origin point, defined in the GLB) and a "transform" point, initially always set to (0, 0, 0) but that can be moved around.

The issue for me was that I was using the scene.svx.json file created from the Voyager Story Standalone. To have the double-click work as intended (i.e., similar to the other viewer that I mentioned), I had to manually create a new scene file using this simple template. Afterward, I also tried using this manually-created scene file in a local Voyager Story, made some modifications, saved it again, and finally everything was working fine.
I don't know which parameter is missing or set differently when generated from Voyager Story Standalone, but I will look into it in the next few days.
Regarding the issue with the "broke orthographic view," I don't think it's properly broken. However, when you click on it or, for instance, on every button under the "View" toolbox, the model most likely goes off-camera.
Finally, the camera behaves oddly if you are in "Tour mode" and double-click on the model.

@sdumetz So, when the model's pose is modified (e.g., position or rotation is different from 0,0,0 - in the JSON scene file, they are defined under model translation and rotation), the double-click doesn't work as expected. Ultimately, it was this setting that caused the unexpected double-click behavior for me, as I always used it to center the model in the scene.
I apologize for the confusion and any inconvenience caused by me. I'm patiently waiting for the final implementation of this feature. Again, sorry for any inconvenience

if you want to start another issue (or a discussion) and provide some more details on the error you are getting, I can try and help with the build issues.

@gjcope a small update, I saw that on the rc-40 branch, among other things, you updated the ubuntu and node version and with this branch I had no issue on windows
Thanks

I pushed cb74df4 with some fixes.
@gjcope I don't think I agree with ff-three's zoomExtents. If I understand it correctly (and I might not, my vector math is bad...) it uses the offset to fit the model's bounding box. Which feels wrong to me? Don't we want the view to stay centered? I'd like to have your opinion on this.
Apart from this, I still have to properly take into account the model's "pose" and I think I'll be done.

Everything looks OK now, except maybe the behavior of zoomExtents (which may as well be left as it is IMO).
Resetting the pivot point in an existing tour requires disabling the "navigation" and enabling it again. AFAIK it's not a bug in this code but a shortcoming of the current tour system that should be fixed separately. If a user does move the pivot point before/during/after a tour that was created without a pivot point, the view gets totally broken and I don't think I can do anything about it.
This whole feature could easily be toggled with a flag but my preference would be to have it "always on".

I pushed cb74df4 with some fixes.
@gjcope I don't think I agree with ff-three's zoomExtents. If I understand it correctly (and I might not, my vector math is bad...) it uses the offset to fit the model's bounding box. Which feels wrong to me? Don't we want the view to stay centered? I'd like to have your opinion on this.
Apart from this, I still have to properly take into account the model's "pose" and I think I'll be done.

Do you have an example of behavior that feels wrong? zoomExtents definitely needs to be improved, but I'm not sure centering is the issue; it's using the center of the bounding box.

The "pose" viewports that are centered around the model instead of the origin feels very wrong to me. However that could be tackled differently by adding a visual "origin" helper in the pose task.
As it is now and if we want to keep its current behaviour, zoomExtents doesn't zoom properly when autoZoom is enabled and the pivot point is not (0, 0, 0) because composeOrbitMatrix doesn't account for the pivot point.
thinking about it, I might just add a boolean toggle to zoom with offset (desired in autoZoom) or without ( desired(?) in PoseTask's Viewports).

Based on the thread, it seems like there is still an unresolved  issue with the pose task zoom working correctly? Other than that I think it just needs testing on our end.

Hi,
there is a debate over how the zoom should behave, but I don't think I have a good solution for it yet so I've left it no better or worse than it was, unless I introduced a bug somewhere.
I intend to work on better visual indicators to help position models in a scene in a few months because we have some users creating scenes with lots of heterogeneous data and the current system is only helpful for one to a few models. I'll look at zoomExtents again in this context then and see if I can improve it on the way.

Hi! I noticed that the saved annotation viewpoints take as reference the last camera scene position (the one set after double-clicking), but they are not updated after successive changes, in other words, when double-clicking after the annotation viewpoint is set. This causes the desired camera position not to be viewed correctly when moving to the saved annotation viewpoint. When switching to orthometric view, the camera moves far from the model, but as soon as you right or left-click on the screen, the camera goes back to the correct position. Lastly, the perspective/orthometric options are not synced between the options in the left panel and the toolbar.

Hi, thanks for taking time to test!
Annotation viewpoints were not saving the "Pivot" property. Fixed in 482cf14.

When switching to orthometric view, the camera moves far from the model, but as soon as you right or left-click on the screen, the camera goes back to the correct position

I couldn't reproduce this, when I switch from Ortho to perspective the view stays roughly the same. Maybe it's a scale thing or something similar, i'll admit I tinkered on the Ortho/Persp conversion algorithm and am not sure at all of its correctness. Could you check if it still happens after 4043f2 (pushed today) and if it does, share a scene where you see this happening?

the perspective/orthometric options are not synced between the options in the left panel and the toolbar

It's unrelated to those changes but I was able to track it down to both fields referencing a different property with only a one-way binding. Fixed in 4043f21. @gjcope I did the same thing in as in #280 and removed the duplicate as it served no meaningful purpose that I could find. The binding is a little bit convoluted in CVViewTool, going through document.setup.navigation.scene.activeCameraComponent to find the actual property. Maybe there is a shortcut that I couldn't find?

@sdumetz I haven't had a chance to test your change yet but taking out the projection property binding means the component update won't get triggered when projection updates.

Yes but it doesn't need to, does it?
The actual camera is updated from CVCamera->CCamera when the projection updates. and that was the only thing this updates did. Or did I miss something?
I didn't notice anything failing to update, but yet again those changes are hard to really test beside clicking-around a bit and see if something is off.

7ef3a3c8 fixes a bug that happens when you nest models as children of another model. That's slightly unconventional but we use this as a workaround for https://github.com/Smithsonian/dpo-voyager/issues/255

Force-push to rebase on v0.43.0 and include changes from https://github.com/Smithsonian/ff-three/pull/3

Sorry for the delay, I was finally able to take a closer look at this.

I see the same Ortho view issue noted by @oM4Uo in this thread. The projection changes with a zoom and needs a re-render (click, etc.) to update correctly. This seems to have been introduced by 47c5c8be9e0383ce64f35d5121071b44886794b8 - it goes away if I revert that commit.
The perspective view in the pose task doesn't zoom extents correctly with a modified pivot.
It's also noted in this thread, but I think the snapshot transformation issue is a barrier. Having tours or annotation viewpoints that don't provide the intended view seems like a blocker. If the state machine always animates on stored parameters, it should be workable no? Except for legacy scenes with no stored pivot, in which case it seems like we can assume the center of the original bound. Deserves some more though but I think needs to be solved...
Is this actually meant to be in the PR? source/client/ui/story/AddNodeMenu.ts

I Fixed 4.. Still looking into 2. and 3..
Concerning 1. I'd really like to keep https://github.com/Smithsonian/dpo-voyager/commit/47c5c8be9e0383ce64f35d5121071b44886794b8 because ~I really hate duplicated properties~ that would make the problem reappear if we somehow changed the camera's projection from another place.
The underlying issue is that CCamera has no clue of the render process so it won't force CameraController to update. 342ab8fe is a workaround that adds back a stored value for camera projection in CVOrbitNavigation but without the property ambiguity.
It works, but I don't like it. Having camera state flow back to the orbit navigation component seems especially fragile if we are looking into implementing alternative navigation modes in the near future.

Thanks for the updates. Agree with you on the fragility. I anticipate the future work will involve a generalized navigation component from which the different modes inherit so slotting this check there could make it a little more robust.

Pivot coordinates were not properly rotated before application in zoomExtents. That's done and the results now looks OK.
zoomExtents still does not feel natural (to me) when applied to the autoZoom feature but I can't figure out what to make of it.


Add .editorconfig file to specify coding style

Add a .editorconfig file to specify the Boogie coding style.
This file is supported by C# editors such as Visual Studio, Rider and Resharper.

@zvonimir :  Please review this PR.


Image appears during scroll

Hi!
I want an image that appears on top of the panel during the scroll with a parallax effect. It's possible?
Thanks in advance

Place an ImageView in the main content, and move it accordingly in the onPanelSlide


When will the codes be released?

Thanks for your great contribution! When will the codes be released? This can be very helpful.

Hi,
Thank you for your interest.
Tentatively in February is the plan currently.
Best,
Ayush

It may be too late T^T

Hello, is there any progress about the code releasing?

+1

we are working on it, should be out soon!


Deus Ex Mankind Divided from gog.com exits after launcher without any error-message in wine.

The gog.com version from Deus Ex Mankind Divided exits immediately after klicking the play-button in the launcher.
There are no error-messages in STDOUT/STDERR.
It doesn't seem to be a crash either since no infos in dmesg and coredumpctl.
Without DXVK the game can be launched from wine resulting in alot of graphical errors.
Software information
Deus Ex Mankind Divided Digital Deluxe
System information

GPU: Navi 14 [Radeon RX 5500]
Driver: radeonsi/mesa-20.1.9
Wine version: wine-5.18 (Staging)
DXVK version: 1.7.2

Apitrace file(s)
apitrace.exe could not be found
Log files

d3d11.log: DXMD_d3d11.log
dxgi.log: DXMD_dxgi.log

If more Informations are needed let me know!

Looks like the dedcompiler_47 has to be installed.
Now i can run the game but it crashes during the intro when starting a new game with:
err:   D3D11RasterizerState: Antialiased lines not supported
Is there a way to bypass that?
DXMD_d3d11.log
DXMD_dxgi.log

Now it seems to be something else, it is crashing with a memory violation (while compiling shaders?).
Sorry for the inconvenience by creating this bugreport.

The steam version works fine here; unless there is a difference to the GoG build there's going to be something wrong on your end.

stdouterr.txt containing the console output. If i should enable some WINEDEBUG let me know.
Interessting might be
Deus Ex -  Mankind Divided.log
containing:
14:48:40:047 (00000416) > [Crash] Unhandled exception at address 0x0000000041725c9b (code: 0xc0000005): "Access violation"
Which seem to be a common problem on Windows too?
Thank you for investigating!

@roidal : d3dcompiler_47 is for DX12 while d3dcompiler_43 is for DX11. You might want to double check that out.
I tried this on my system with very similar setup, GOG version too. Game is running fine for me, benchmark and first mission loaded without issues. Details below.
System information
OS: Debian 9.12
CPU: Core i5-4590
GPU: Navi 14 [Radeon RX 5500]  - 8GB Model
Driver: radeonsi/mesa-20.1.9
Wine version: wine-5.9 (Staging)
DXVK version: 1.7.2
First Mission:

Graphics settings:

Wine overrides used:

Thanks for your reply!
I'am not able to get the game running, even when trying to copy your setup.
The only difference i see is that i have a newer LLVM and wine-version.
Meanwhile i'am out of ideas.
@jrugia how do you launch the game, and in which directory is it installed?

update:
after setting RADV_DEBUG=errors
i have following lines in stderr:
../src/amd/vulkan/radv_descriptor_set.c:494: VK_ERROR_OUT_OF_POOL_MEMORY

@roidal I use my own scripts for running Wine games (No lutris, steamplay....etc). I kept the game installed, will give it another try tomorrow with Wine 5.18 to match your setup even closer and report back.
Afraid I can't help much about the error in radv debug though. It's beyond my understanding.

Update:
Seem to be a problem with wine itselfe.
After changing to wine from lutris (version 5.7) the benchmark works.


Android Guidelines: Fit content to new Guidelines structure

Description
Per the MQ Guidelines initiative to fill gaps in the guidelines across languages, we are moving to a new common structure for the guidelines and identifying areas where there are content gaps -- i.e. one or more of the other Tier 1 language guidelines have guidance that is not language-specific that is not represented in this language's guidelines.
Note
For Implementation, the below structure is based on .NET content.  It is intended to provide implementation details for things that aren't visible in the public API, with a TOC that largely maps to the API Design section in the Design TOC, with sections filled for areas where you already have content written.  Feel free to reach out with any questions.
New Structure
(Numbers for ease of reading here, but won't be shown in the guidelines page TOC)
For further details, please see the restructured .NET guidelines here.
Design

Introduction

Design Principles
General Guidelines
Non-HTTP Libraries

Azure SDK API Design

Service Client

Service Client Constructors

Client Configuration
Setting the Service Version
Sync and Async
Client Immutability

Service Methods

Naming
Return Types
Cancellation

Service Method Parameters

Options Parameters
Parameter Validation

Methods Returning Collections (Paging)
Methods Invoking Long Running Operations
Conditional Request Methods
Hierarchical Clients

Supporting Types

Model Types
Enumerations
Using Azure Core Types
Using Primitive Types

Exceptions (Errors)
Authentication
Namespaces
Support for Mocking

Azure SDK Libraries (Packages)

Packaging

Service-Specific Common Libraries

Versioning
Dependencies
Native Code
Doc Comments

Repository Guidelines

Documentation Style
README
Samples

Implementation

API Implementation

Service Client

Service Methods

HttpPipeline
HttpPipelinePolicy/Custom Policies

Service Method Parameters

Parameter Validation

Supporting Types
1. Serialization
2. Enumeration-like Structs

SDK Feature Implementation

Configuration
Logging
Distributed Tracing
Telemetry

Testing
Language-specific other

@salameer FYI


[CT-153] [Feature] Expose the meta field as a query parameter for Metadata API calls

Is there an existing feature request for this?

[X] I have searched the existing issues

Describe the Feature
Enable the meta data field to be queryable for the following API calls -
https://docs.getdbt.com/docs/dbt-cloud/dbt-cloud-api/metadata/schema/metadata-schema-sources
https://docs.getdbt.com/docs/dbt-cloud/dbt-cloud-api/metadata/schema/metadata-schema-models
Describe alternatives you've considered
No response
Who will this benefit?
This'll help dbt user who leverage the Metadata API.
Requested by Loft.
Are you interested in contributing this feature?
No response
Anything else?
No response

Thanks @BrJan!
This one is for the Metadata team at dbt Labs (cc @barryaron). My understanding is that meta is available as a field for metrics, but not yet for other node types.
I will note that there's a related Core issue we need to resolve, to ensure that all meta fields are consistently represented in the node's meta dictionary in manifest.json: https://github.com/dbt-labs/dbt-core/issues/4459. This is an outcome of meta being configurable in many more places (since v0.21), and wanting to consolidate all meta values into a single dictionary, for ease of downstream parsing. Otherwise, downstream metadata consumers (both dbt-docs + Metadata API) will need to check the values in both node.meta and node.config.meta. We're planning that fix for a future v1.0 patch release, targeting v1.0.3 in March.


Replace mem::uninitialized calls

Part of gtk-rs/gir#806 (not a regen but same thing).
cc @EPashkin @sdroege
This one should be the last one.

Thanks, 👍 after rust becomes 1.36.0

I forgot to update travis once again... Good catch @EPashkin !

On 13 July 2019 19:33:11 EEST, Evgenii Pashkin<EMAIL_ADDRESS>wrote:

EPashkin commented on this pull request.

@@ -50,11 +50,11 @@ impl PdfSurface {

pub fn get_versions() -> impl Iterator<Item = PdfVersion> {
    let vers_slice = unsafe {

       let mut vers_ptr: *mut ffi::cairo_pdf_version_t =

mem::uninitialized();

       let mut vers_ptr = mem::MaybeUninit::<*mut

ffi::cairo_pdf_version_t>::uninit();

I'd recommend ptr::null_mut() here like we do for all pointers everywhere.
I'll review everything else tomorrow.
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

@sdroege, record filled by cairo_pdf_get_versions, so IMHO current version is right.

Updated.

:+1:

@GuillaumeGomez all green


Failure on yarn marketlaunch:setup

Looks like I'm not able to compile on the yarn marketlaunch:setup step as it throws lots of TS errors. Adding "node" to the "types" in tsconfig.json solved this however.
"types": ["reflect-metadata", "node"]

Removing the field seems to work fine as-well, while reducing complexity and undocumented options we have enabled. Making a PR now.


Add support for branch.name in sonar action

Feature Request
🔑
Motivation Behind Feature
Fastlane action sonar doesn't support branch analysiswhich is important for SonarQube usage
Feature Description
Add branch_name parameter in sonar action to be passed to sonar scanner.
the action will look like this sonar(branch_name: "feature/f1")
There's no downside to this parameter as this one will be optional.
Alternatives or Workarounds
Alternate is to ditch fastlane and use command line
None of the existing plugins or actions provides alternate solution

The fix has been released.


Ability to rewrite the host part of the URL in M3U file

Is your feature request related to a problem? Please describe.
I would like to be able to use m3u-filte to be able to rewrite the host names of the endpoints in the m3u file or the response from the api so that i can proxy the connection though a local proxy. I can achieve this now using m3u-filter to rewrite a file and iptv-proxy to expose the file but i would like to use xteam and your multiple playlist feature as well and i cannot see how this would be possible.
Describe the solution you'd like
I would like m3u-filter to offer a feature to rewrite the host part of the endpoints in the m3u filer / api response

I would like to add that for each m3u xtream playlist id like to have different host names so i can route traffic to the proxies differently. This would allow me to create a playlist that can user the local address of my proxy but also route it externally via a different playlist.

Hi, can you please give me an example what you want to do ?
As i understood for m3u , you have an entry like
#EXTINF:-1 tvg-id="" tvg-name="One HD" group-title="GB | One",Public HD
http://my.iptv.provider.com:8080/myuser/token/streamid

and you want to change it to
http://one.two.three.tv/myuser/token/streamid

in xtream it would be the direct_source  property ?
Is this what you want to do ?

This is currently only possible if you want to change the host parts of all entries from a target.
Each target  has a rename block.
``ỳml
rename:
- { field: url,  pattern: '(https?://.?)/(.)',  new_name: http://my.bubble-gum.tv/$2 }

When you write this,  each entry of  a target will be replaced.

I will also add this to the mapping part, to more fine grained replacements,  for example each group could be have different hosts.

And what are you missing in `m3u-filter` for serving xtream and m3u-files  with password and username ?

Ok, i looked into the code and i saw that the url part wasnt implemented :(  I will fix this.

@adamsdavis572 could you test it ? Can we close the issue ?
My question about iptv-proxy, what functionality are you missing in m3u-filter ?

@adamsdavis572 this isnt a build issue, it is a config issue. The config format has changed.
look at line 32 in your config.yml
input configuration has changed. input is now a list.
sources:
  inputs:
    - enabled: true
      type: xtream
      headers:
        User-Agent: "Mozilla/5.0 (Linux; Tizen 2.3) AppleWebKit/538.1 (KHTML, like Gecko)Version/2.3 TV Safari/538.1"
        Accept: application/json
        Accept-Encoding: gzip
      url: 'http://localhost:8080'
      username: test
      password: test
  targets:
    - name: pl1
      publish: true
      filename: playlist_1.m3u
      type: m3u
      filter: '!ALL_CHAN!'

So this is the config yml i have been using and im still getting the error
api: { host: <IP_ADDRESS>, port: 8901, web_root: /home/m3u-filter/web }
threads: 0
working_dir: /home/m3u-filter/data
# sec  min   hour   day of month   month   day of week   year
schedule: "0  0  8,12,16,20,22,1  *  *  *  *"
sources:
  inputs:
    - enabled: true
      type: xtream
      headers:
        User-Agent: "Mozilla/5.0 (Linux; Tizen 2.3) AppleWebKit/538.1 (KHTML, like Gecko)Version/2.3 TV Safari/538.1"
        Accept: application/json
        Accept-Encoding: gzip
      url: 'http://localhost:8080'
      username: test
      password: test
  targets:
    - name: pl1
      publish: true
      filename: playlist_1.m3u
      type: m3u
      filter: '!ALL_CHAN!'

Oh sorry,  it is my failure,    sources is a list too,

sources:
  - inputs:   <-------------  he list dash
    targets:

Thanks - that worked - ill test properly night :-)

Thanks - that worked - ill test properly night :-)

Ok, i am waiting for your response

so i have tested the rename rule and it works exactly as needed - many thanks.
however i cannot get the api-proxy to work.
i have configure api-proxy.yml
server:
  protocol: http
  ip: <IP_ADDRESS>
  http_port: 8901
  #https_port: 443
  #rtmp_port:
  timezone: Europe/London
  message: Welcome to m3u-filter
user:
  - target: pl1
    credentials:
      - {username: home, password: xxxxxxx}

and config.yml
api: { host: <IP_ADDRESS>, port: 8901, web_root: "./web" }
threads: 0
......

but when i hit http://<ip-address>:8901/player_api.php?username=home&password=xxxxxxxxx i get a 400
[2023-10-24T18:47:39Z INFO  actix_web::middleware::logger] <IP_ADDRESS> "GET /player_api.php?username=home&password=xxxxxxx HTTP/1.1" 400 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36" 0.000069

but if i hit http://<ip-address>:8901/get.php?username=home&password=xxxxxxxxx if returns 200 and the list correctly
any suggestions? thx

you have defined a user for the target pl1.
Do you have  a target pl1 in the config. yml?
You can only define a user for a target.
And did it run and created the files?

Output doc
In the README is output described

yes apologies this is user error i forgot to include the xtream output. now all endpoints are working  :-)
however i have one more issue that im not sure how to resolve.
when i call http://x.x.x.x:8901/player_api.php?username=home&password=<password>
i get a successful response
{"user_info":{"active_cons":0,"allowed_output_formats":["ts"],"auth":1,"created_at":1666639623,"exp_date":1729711623,"is_trial":0,"max_connections":1,"message":"Welcome to m3u-filter","password":"dfdfdf","username":"home","status":"Active"},"server_info":{"url":"x.x.x.x","port":8901,"https_port":443,"server_protocol":"http","rtmp_port":1935,"timezone":"Europe/London","timestamp_now":1698175623,"time_now":"2023-10-24 20:27:03"}}

however the returned password field seems to be hardcoded
https://github.com/euzu/m3u-filter/blob/76e96093d3dc00fe9cef4984007c2450e78628d8/src/api/xtream_player_api.rs#L25
i assume this is not correct?
thx

Additional info
This user name and password is used in subsequent calls from the iptv client (smarters iptv) and then fails due to mismatch.

Ah ok,
i implemented the xtream  api  based on data i have gathered on github. I dont have an api xtream account and couldnt test it well.
It could be that i forgot to write the right password in the response. İ will fix this tomorrow. Now its a little bit too late.

@adamsdavis572 i have pushed a new version, please test it.

@adamsdavis572 when you give me a go, i will create a new release with the new web-ui changes.

so /player_api.php?username=home&password=password now works correctly and returns the correct info - thank you 👍
{"user_info":{"active_cons":0,"allowed_output_formats":["ts"],"auth":1,"created_at":1666721205,"exp_date":1729793205,"is_trial":0,"max_connections":1,"message":"Welcome to m3u-filter","password":"password","username":"home","status":"Active"},"server_info":{"url":"x.x.x.x","port":8901,"https_port":443,"server_protocol":"http","rtmp_port":1935,"timezone":"Europe/London","timestamp_now":1698257205,"time_now":"2023-10-25 19:06:45"}}

however the other, subsequent xtream api calls all fail with a 400
/player_api.php?username=home&password=password&action=get_live_categories
/player_api.php?username=home&password=password&action=get_vod_categories
/player_api.php?username=home&password=password&action=get_series_categories
error from /player_api.php?username=home&password=password&action=get_live_categories
[2023-10-25T18:06:39Z INFO  actix_web::middleware::logger] <IP_ADDRESS> "GET /player_api.php?username=home&password=password&action=get_live_categories HTTP/1.1" 400 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36" 0.000144

@adamsdavis572 i will try to figure out what the problem is. It happens when you make the same api  call in short cycles ?

thanks - it happens when you supply any action as a query param. how do you provide the mapping from categories to m3u groups? is it inherited from the original xstream api call?

if your surce is xtream, then the categories and groups are the same.
if you source is m3u the categories are guessed based on the video.extensions  configuration.
Everyhting that matches the  extension is categorized as Video, anything alse as Live.
I have tested the api and called it with curl, i got no 400 responses.
I try to write more debug logs to figure out what is happening at your side.

i have added some debug information. You need to set loglevel via command line argument
Following log levels are supported:

debug
info default
warn
error

Use the -l or -log-level cli-argument to specify the log-level.

ok so im still getting the same 400 and i cant see any more debug info that might give me a clue
$./m3u-filter -s  -a api-proxy.yml -c config.yml -l debug
[2023-10-25T19:58:14Z DEBUG m3u_filter::filter] template ALL_CHAN  depends on [US_CHAN, UK_CHAN]
[2023-10-25T19:58:14Z DEBUG m3u_filter::filter] [
        PatternTemplate {
            name: "UK_CHAN",
            value: "Group ~ \"^.UK..*\"",
        },
        PatternTemplate {
            name: "US_CHAN",
            value: "Group ~ \"^.US..*\"",
        },
        PatternTemplate {
            name: "ALL_CHAN",
            value: "Group ~ \"^.US..*\" OR Group ~ \"^.UK..*\"",
        },
    ]
[2023-10-25T19:58:14Z DEBUG m3u_filter::filter] Created regex: ^.US..* with captures: []
[2023-10-25T19:58:14Z DEBUG m3u_filter::filter] Created regex: ^.UK..* with captures: []
[2023-10-25T19:58:14Z DEBUG m3u_filter::model::config] Filter: Group ~ "^.US..*" OR Group ~ "^.UK..*"
[2023-10-25T19:58:14Z DEBUG m3u_filter::filter] Created regex: ^.US..* with captures: []
[2023-10-25T19:58:14Z DEBUG m3u_filter::filter] Created regex: ^.UK..* with captures: []
[2023-10-25T19:58:14Z DEBUG m3u_filter::model::config] Filter: Group ~ "^.US..*" OR Group ~ "^.UK..*"
[2023-10-25T19:58:14Z INFO  m3u_filter] working dir: "/home/xxxx/git/m3u-filter/scratch/data"
[2023-10-25T19:58:14Z WARN  m3u_filter::config_reader] cant read mapping file: ./mapping.yml
[2023-10-25T19:58:14Z DEBUG m3u_filter::config_reader] no mapping loaded
[2023-10-25T19:58:14Z DEBUG m3u_filter] web_root: /home/xxxx/git/m3u-filter/scratch/web
[2023-10-25T19:58:14Z INFO  m3u_filter] server running: http://<IP_ADDRESS>:8901
[2023-10-25T19:58:14Z INFO  actix_server::builder] starting 2 workers
[2023-10-25T19:58:14Z INFO  actix_server::server] Actix runtime found; starting in Actix runtime
[2023-10-25T19:58:45Z INFO  actix_web::middleware::logger] 192.168.1.xx "GET /player_api.php?username=home&password=password&action=get_live_categories HTTP/1.1" 400 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36" 0.000114
[2023-10-25T19:58:46Z INFO  actix_web::middleware::logger] 192.168.1.xx "GET /player_api.php?username=home&password=password&action=get_live_categories HTTP/1.1" 400 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36" 0.000067

here is the config file im using
api: { host: <IP_ADDRESS>, port: 8901, web_root: ./web }
threads: 0
working_dir: ./data
templates:
  - name: UK_CHAN
    value: Group ~ "^.UK..*"
  - name: US_CHAN
    value: Group ~ "^.US..*"
  - name: ALL_CHAN
    value: >-
      !US_CHAN! OR
      !UK_CHAN!
sources:
  - inputs:
    - enabled: true
      type: xtream
      headers:
        User-Agent: "Mozilla/5.0 (Linux; Tizen 2.3) AppleWebKit/538.1 (KHTML, like Gecko)Version/2.3 TV Safari/538.1"
        Accept: application/json
        Accept-Encoding: gzip
      url: 'http://myportal.live:8080'
      username: Qf9GFSBeu8Up
      password: Qngq4aNjtaVt
    targets:
      - name: pl1
        output:
          - type: xtream
        rename:
          - { field: url,  pattern: '(https?:\/\/.*?)\/(.*)',  new_name: "http://192.168.1.xxxx:8901/$2" }
        sort:
          groups:
            order: asc
        filter: "!ALL_CHAN!"
      - name: pl2
        output:
          - type: xtream
        rename:
          - { field: url,  pattern: '(https?:\/\/.*?)\/(.*)',  new_name: "https://iptvproxy.xxxxx.net/$2" }
        sort:
          groups:
            order: asc
        filter: "!ALL_CHAN!"

Can you check your source code, ı have added  debug output here
If you dont see this output, the endpoint isnt called.

thanks for you patients - user error again
2023-10-25T20:38:46Z DEBUG m3u_filter::api::xtream_player_api] Incoming request UserApiRequest { username: "home", password: "password", token: "", action: "get_live_categories" }
[2023-10-25T20:38:46Z DEBUG m3u_filter::api::xtream_player_api] Could not open file for xtream target: pl1 Cant find collection: pl1/cat_live
[2023-10-25T20:38:46Z INFO  actix_web::middleware::logger] 192.168.1.xx "GET /player_api.php?username=home&password=password&action=get_live_categories HTTP/1.1" 400 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36" 0.000173
[

is there a file under . data/pl1/cat_live?

thanks for the pointer. i had nothing in the data directory as a result of a permissions issue. all APIs working now :-)
i now need to test the renaming, proxying etc.
many thanks for your help here. ill let you know the results of the testing tomorrow.

Q: if no schedule is defined when will the the call to the source happen to download the playlists? should it happen at startup in server mode? if so this is not happening. when i start in server mode nothing is downloaded to data. i have to start the service in non-server mode to force it to download the playlists. Is this the expected behaviour? thx

In server mode, the updates only happen on scheduled times. If not defined in config,
it will never happen in server mode.

I could pıut a manual update button into the web ui, to force updates manually in server mode

The Web-UI has now a manual update button. You can select the targets to update or  if you do not select any target, all targets will be updated.
@adamsdavis572 can i close this issue ?

@adamsdavis572 about your question, now i understand the difference to 'iptv-proxy',  'm3u-filter' only serves the metadata not the streams. No  stream request is handled. '/live/home/password/269970.ts'  wont be answered. What do you expect to return for this query ?

@euzu i actually wanted to have the request to my iptv provider proxied as i run m3u-filter (currently iptv-proxy) in a container that routes all traffic via a vpn. This then gives me a local endpoint that i can use in my home network or externally that is exposed via a normal HTTPS endpoint that then routes all traffic via a VPN to my IPTV provider.
i use pierro777/iptv-proxy
Now i understand how m3u-filter works i think i should be able to add it as an xtream provider to iptv-proxy and the let the proxy tdo the routing. ill try this shortly.
BTW im happy for you to close this issue as the original issue is definitely resolved - thank you 👍
if i have other questions could i ping you? or raise an issue? what would you prefer?

I have activated discussions. If you have questions you can open a discussion.


[stable/grafana] Typo in ingress definition for tls secretName

Is this a request for help?: no

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

Version of Helm and Kubernetes:
v2.9.1 / v.1.10.4-gke.0
Which chart:
Grafana
What happened:
TLS doesn't work with the actual ingress. There is a typo in the ingress definition
What you expected to happen:
TLS working
How to reproduce it (as minimally and precisely as possible):
Deploy chart with ingress enabled and TLS secretName
Anything else we need to know:

Are you sure about this?
It works for me:
  ingress:
      enabled: true
      annotations:
        kubernetes.io/ingress.class: nginx
        certmanager.k8s.io/cluster-issuer: letsencrypt-issuer
      hosts:
        - grafana.k8s.davidkarlsen.com
      tls:
      - hosts:
        - grafana.k8s.davidkarlsen.com
        secretName: grafana-secret

Ay,
I've tried with and without the "-". Without my Ingress never got his SSL.

Ay,
I've tried again and I stil got the problem.
Without fix
HTTP
krustyhack@Unknown:~/tmp/grafana/grafana$ curl http://a.b.c.d
Found.

HTTPS
krustyhack@Unknown:~/tmp/grafana/grafana$ curl https://a.b.c.d
default backend - 404

Ingress
apiVersion: v1
items:
- apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    annotations:
      kubernetes.io/ingress.class: nginx
    creationTimestamp: 2018-06-25T12:04:18Z
    generation: 1
    labels:
      app: grafana
      chart: grafana-1.11.5
      heritage: Tiller
      release: testgrafana
    name: testgrafana
    namespace: qa
    resourceVersion: "32848583"
    selfLink: /apis/extensions/v1beta1/namespaces/qa/ingresses/testgrafana
    uid: e37e7a04-786f-11e8-8c52-42010a8401dd
  spec:
    rules:
    - host: a.b.c.d
      http:
        paths:
        - backend:
            serviceName: testgrafana
            servicePort: 80
    tls:
    - hosts:
      - a.b.c.d
      secretName: ingress-nginx-certs
  status:
    loadBalancer:
      ingress:
      - ip: w.x.y.z
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

With fix
HTTP
krustyhack@Unknown:~/tmp/grafana/grafana$ curl http://a.b.c.d

<title>308 Permanent Redirect</title>

308 Permanent Redirect

nginx

HTTPS
krustyhack@Unknown:~/tmp/grafana/grafana$ curl https://a.b.c.d
Found.

Ingress
apiVersion: v1
items:
- apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    annotations:
      kubernetes.io/ingress.class: nginx
    creationTimestamp: 2018-06-25T12:04:18Z
    generation: 2
    labels:
      app: grafana
      chart: grafana-1.11.5
      heritage: Tiller
      release: testgrafana
    name: testgrafana
    namespace: qa
    resourceVersion: "32849624"
    selfLink: /apis/extensions/v1beta1/namespaces/qa/ingresses/testgrafana
    uid: e37e7a04-786f-11e8-8c52-42010a8401dd
  spec:
    rules:
    - host: a.b.c.d
      http:
        paths:
        - backend:
            serviceName: testgrafana
            servicePort: 80
    tls:
    - hosts:
      - a.b.c.d
    - secretName: ingress-nginx-certs
  status:
    loadBalancer:
      ingress:
      - ip: w.x.y.z
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""


BUG: Date seems to be incorrect?

July or January?

Assigning to @grdw

@grdw was is the status of this?

It's in a pull request.