Running snakemake in a SLURM job context

In our HPC environment I am not alowed to run anything on the login-nodes (kicked out, banned...). So I have to run snakemake ALWAYS with sbatch or using an interactive session via srun.
Therefore I always get the warning
You are running snakemake in a SLURM job context. This is not recommended, as it may lead to unexpected behavior.Please run Snakemake directly on the login node.

And the warning is correct. Because especially the number of threads are not correctly set.
Any idea how to run snakmake within a SLURM job properly? Slurm interplayed perfectly with snakemake 7 using --cluster --clsuter-cancel and so on. But now with snakemake 8 I am forced to run it with this plugin which does not work in our environment...

Well that's how it is and I totally understand the behavior. Just imagine 200 users run small local rules on the login node...
Also not the complete file system is visible. So the only way I can run Snakemake is after hopping to a node using an interactive session.

Maybe it is possible to overwrite environment vars using the resources for the rule?

Just imagine 200 users run small local rules on the login node...

I do. And? ;-) If they don't "really" start to calculate, the system will cope. That what Linux is designed for.

Also not the complete file system is visible.

That is most inconvenient for any kind of work. I am sure that there are alternative solutions.

Maybe it is possible to overwrite environment vars using the resources for the rule?

I will look into it - and keep the issue open.

I am not permitted to perform any operations on the login node, and every time I attempt to run Snakemake there, it gets killed immediately. Our system administrator has advised that I should first allocate a compute node using salloc and then run Snakemake from the allocated node. I’ve tried two methods, both of which have failed.

First, I allocate a compute node with salloc and then run snakemake --configfile config/config.yaml --cores 10. This approach runs normally but is very slow.
First, I allocate a compute node with salloc, and then I attempt to run snakemake --configfile config/config.yaml --cores 10 --profile profile. This command tries to submit jobs from the compute node using sbatch, but I consistently receive a WorkflowError:

SLURM job submission failed. The error message was sbatch: error: invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Unspecified error

Does anyone have suggestions on how to resolve this issue or alternative methods to run Snakemake efficiently on our cluster?
@visze @cmeesters

The main process may also need to create containers and/or conda environments. Depending on the complexity, it may require a lot of CPU time and a large chunk of memory. In addition, the main process needs to build a DAG and potentially rebuild it later. That task might be quite resource-intensive too. While the environments can be created beforehand with --conda-create-envs-only (if the users remember), there is no easy fix for DAG building. I managed to create a pipeline requiring more than 4 GB of memory in the past. That was years ago, so perhaps algorithms got more efficient since then.
People tend to forget or to confuse the environments. We had to introduce restrictions (1 GB of memory, 10 minutes CPU time) after login nodes went down in flames a couple of times. So, we do not kick/ban users as was the case with OP but the users can still kick themselves. With those restrictions, some pipelines fail to run on login nodes. So, we have always recommended to run them as jobs. It does seem wasteful because, as you said, the main process is dormant most of the time. But the only other alternative I see is to try it on the login node and submit it as a job if it exceeds the limits.
I do not quite understand the parametrisation and inheritance comment. The environment is inherited either way and the resource requirements are typically overridden. When does it become a problem?

The executor has to export the environment - how else can snakemake find itself or other software defined in the environment? However, SLURM variables get exported, too. Please test PR #137 . Does it allow correct submission? It removes SLURM* variables within the snakemake process. Further suggestions on how to solve this issue are welcome.

I had the same issue as described: the scripts that launch my pipeline are designed to run in a SLURM job, and the threads were always set to 1. I've just tested this PR, and I can confirm it solved the problem.

I will run a few more tests tomorrow (there is no chance for in-job submission in our CI-pipeline) and will ready it for review, then.

I tried the PR on a fresh snakemake installation:
mamba create -n snakemake_slurm_test snakemake pip
pip install git+https://github.com/snakemake/snakemake-executor-plugin-slurm.git@feat/in_job_stability

Then I run snakemake using the slurm plugin within an interactive session (srun -i bash) with one CPU:
    JOBID PARTITION NAME                           ST         TIME NODES CPUS MIN_MEMORY NODELIST(REASON)
   8969734 medium    bash                            R     18:34:51     1    1        30G      hpc-cpu-109

I get the following error when just running snakemake (version 8.18.1). When running snakemake before installing the plugin it works fine!
Traceback (most recent call last):
  File "/data/cephfs-1/work/groups/kircher/users/schubacm_c/miniforge3/envs/snakemake_slurm/bin/snakemake", line 6, in <module>
    from snakemake.cli import main
  File "/data/cephfs-1/work/groups/kircher/users/schubacm_c/miniforge3/envs/snakemake_slurm/lib/python3.12/site-packages/snakemake/cli.py", line 22, in <module>
    from snakemake.api import (
  File "/data/cephfs-1/work/groups/kircher/users/schubacm_c/miniforge3/envs/snakemake_slurm/lib/python3.12/site-packages/snakemake/api.py", line 50, in <module>
    from snakemake.workflow import Workflow
  File "/data/cephfs-1/work/groups/kircher/users/schubacm_c/miniforge3/envs/snakemake_slurm/lib/python3.12/site-packages/snakemake/workflow.py", line 65, in <module>
    from snakemake.scheduler import JobScheduler
  File "/data/cephfs-1/work/groups/kircher/users/schubacm_c/miniforge3/envs/snakemake_slurm/lib/python3.12/site-packages/snakemake/scheduler.py", line 27, in <module>
    registry = ExecutorPluginRegistry()
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/cephfs-1/work/groups/kircher/users/schubacm_c/miniforge3/envs/snakemake_slurm/lib/python3.12/site-packages/snakemake_interface_common/plugin_registry/__init__.py", line 31, in __init__
    self.collect_plugins()
  File "/data/cephfs-1/work/groups/kircher/users/schubacm_c/miniforge3/envs/snakemake_slurm/lib/python3.12/site-packages/snakemake_interface_common/plugin_registry/__init__.py", line 77, in collect_plugins
    module = importlib.import_module(moduleinfo.name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/cephfs-1/work/groups/kircher/users/schubacm_c/miniforge3/envs/snakemake_slurm/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/cephfs-1/work/groups/kircher/users/schubacm_c/miniforge3/envs/snakemake_slurm/lib/python3.12/site-packages/snakemake_executor_plugin_slurm/__init__.py", line 29, in <module>
    from utils import delete_slurm_environment
ModuleNotFoundError: No module named 'utils'

I changed this line in init.py :
from utils import delete_slurm_environment
to
from .utils import delete_slurm_environment
to use the relative utils

Ah, sorry guys, I am a little bit stressed and forgot to commit my fixes. The CI is now all green and I hope, I can test myself within the hour.

Not a problem and honestly don't stress it, let us know if you want us to test anything or take some of the strain ?
Just to add +1 on running from compute nodes rather than head/login node, we have a similar setup.
We also run nextflow at times and found that caused chaos on the head node.
Being able to run a minimal sbatch script (1cpu/100mb) to launch snakemake would help keep things simpler and cleaner for us when helping users. Less chance of them accidentality running applications on headnode or building apptainer images and keeps it in line with nextflow setup for running workflows.  Although a slight waste of resource, having a job running with minimal specs to control snakemake still seems preferable than having users attached to login node while running (either via tmux or sometimes by leaving devices connected), it also keeps the processes and attached users on login node lower.
IF users know what they are doing then running from headnode is fine, but for testing and new users running on compute nodes (whether sbatch or srun) seems a lot safer and less headaches for us.

That's it exactly, if configured correctly it's not an issue but sadly things tend to go astray.
The new update works a lot better than previously in srun and sbatch, it's allocating and reporting correctly everywhere apart from in the slurm logs (think this is the same as dry-run issue with threads).
At the moment I am setting threads and cpus_per_task in the profile (different in this example for testing reasons), is there a way of combining to only set this once in the profile/config ?
If I set threads it doesn't allocate enough resources in slurm, if I just set cpus_per_task it doesn't fill in threads?
Not sure if there was a method I am missing?
Snakefile
rule all:
        input:
                "output/123.csv",
                "output/987.csv"

rule createtxt:
        output:
                "output/{num}.txt"
        shell: '''
                echo "Threads = {threads}"
                echo {wildcards.num}  > {output}
        '''

rule createcsv:
        input:
                "output/{num}.txt"
        output:
                "output/{num}.csv"
        shell: '''
                echo "Threads = {threads}"
                echo {wildcards.num}  > {output}
        '''

slurm_profile/config.v8+.yaml
jobs: 100
default-resources:
    slurm_partition: defq
    slurm_extra: "'-o smk.%j.out -e smk.%j.err'"
printshellcmds: True
set-threads:
        createtxt: 4
set-resources:
        createtxt:
            mem_mb_per_cpu: 500
            cpus_per_task: 6

running with:
snakemake --profile slurm_profile

I was now able to test the PR.
I started an interactive session usining srun -i batch and allocating only 1 CPU:
     JOBID PARTITION NAME                           ST         TIME NODES CPUS MIN_MEMORY NODELIST(REASON)
   8969734 medium    bash                            R   1-21:31:07     1    1        30G      hpc-cpu-109

Then I run snakemake with the slurm excecutor plugin and my rule gets 30 threads (part of the profile):
set-threads:
  assignment_mapping_bwa: 30

Interestingly the job just gets 2 CPUs instead of 30! That was the same behaviour before. So nothing improved on my side.
 JOBID PARTITION NAME                           ST         TIME NODES CPUS MIN_MEMORY NODELIST(REASON) USER                                  COMMENT
8986188 medium    8302f233-44d1-4ce9-84a6-ff80ef  R         0:00     1    2      9537M      hpc-cpu-198 schubacm_c rule_assignment_mapping_bwa_wildcards_as

I will try now running snakemake from an sbatch commit to see if something changes. But I am not so optimistic

If you try setting resources for the rule as well (also keep the threads), does it then use 30 cpu ?
This is what I am currently getting, have to set resources and threads in the profile/config file:
set-resources:         assignment_mapping_bwa:             cpus_per_task: 30
Or change the rule to use resources instead of threads and just set resources in config.
resources:                 cpus_per_task=30         shell: '''                 echo "cpus = {resources.cpus_per_task}"                 '''
Not sure of the best way to tie them together without breaking local workflows or requiring a rewrite.

I will try it and let you know. But in general this will be a drawback when threads are defined dynamically. Is there an option to set cpus_per_task to threads

We import a function from the jobstep-Executor:
def get_cpus_per_task(job: JobExecutorInterface):
    cpus_per_task = job.threads
    if job.resources.get("cpus_per_task"):
        if not isinstance(cpus_per_task, int):
            raise WorkflowError(
                f"cpus_per_task must be an integer, but is {cpus_per_task}"
            )
        cpus_per_task = job.resources.cpus_per_task
    # ensure that at least 1 cpu is requested
    # because 0 is not allowed by slurm
    return max(1, cpus_per_task)

Read: If you define threads or take a 3rd party workflow where threads is defined (or preferably read from the configuration), cpus_per_task is translated accordingly from threads. If, you choose to just read cpus_per_task from your parametrization file, this is also fine. You can use cpus_per_task to superseed threads, because threads is often hard coded and intended for local execution, whereas on a cluster you might achieve better scalability.

Think this is the part that's currently not working.
If you don't specify cpus_per_task >= threads it limits you to the maximum number of threads defined in the launch sbatch/srun command.
Works fine from head node (correct number of threads/cpus), but if you use an sbatch script with:
#SBATCH --cpus-per-task=4
snakemake --profile slurm_profile

The maximum number of threads it allows in any snakemake job will be 4, even though they can launch on different nodes and threads total is defined as higher in profile.
set-threads:
        createtxt: 8

We will probably just stick with duplicating threads and cpus_per_task in the config file for the moment rather than rewriting the script to using cpus_per_task as a resource, otherwise it will break it for local (non slurm) runs.

I totally agree with you @CarstenBaker. It works with cpus_per_task  but not when you only define threads

If you manually set SLURM_CPUS_PER_TASK on srun to be higher and then launch snakemake it will pick up the threads correctly (obviously still assuming you set it higher than threads requested).
export SLURM_CPUS_PER_TASK=10
But wouldn't recommend it as seems a bit of a hack, more just for a test.

What do you mean by "currently"? Even using the new PR?

Sadly yes, installed the new PR fresh this afternoon after your message.
Tried in conda and also with a standard venv (below):
# using python/3.12.1
python -m venv snakemake_8_slurmfix
source snakemake_12_slurmfix/bin/activate
pip install --upgrade pip
pip install snakemake
pip install git+https://github.com/snakemake/snakemake-executor-plugin-slurm.git@feat/in_job_stability

Still seems to be using SLURM variables?
Is there another method you used to install, although looking in the venv the updated files are there so assume they are working correctly.

I think I understand what the delete_slurm_environment() is trying to do, it's basically just removing the SLURM env commands.
I think the unset command is failing for me, if I try it manually in Python 3.12.1 it doesn't unset the env.
For example, if I srun with 6 CPUS_PER_TASK and try unset I get this, if I use pop instead it removes it (not sure if unset has changed, my python skills are greatly lacking!)
>>> import os
>>> print(os.environ.get('SLURM_CPUS_PER_TASK'))
6
>>> os.unsetenv('SLURM_CPUS_PER_TASK')
>>> print(os.environ.get('SLURM_CPUS_PER_TASK'))
6
>>> print(os.environ.pop('SLURM_CPUS_PER_TASK'))
6
>>> print(os.environ.get('SLURM_CPUS_PER_TASK'))
None

However, even if I unset SLURM_CPUS_PER_TASK manually and remove any reference to 6 cpus in env the snakemake job still won't go greater than the defined number of cpu's from sbatch/srun (not sure where else it picks up this number from?)


Add selected minor color token

:tada: This PR is included in version 4.20.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


In-depth PCDM - what it is? what it does? how it works? how to implement? new de...

In-depth PCDM - what it is? what it does? how it works? how to implement? new developments?
Suggested by: Sheila Rabun per request
Presenter: Maybe someone who is presenting on PCDM at Open Repository?
Format: ?
Audience: Managers, metadata people, developers, all

Esmé and Karen have agreed to do this.  (2016-07-14)
Full day workshop.

Correction: half-day workshop, please.

Here's the info for our In-Depth PCDM workshop:
Synopsis: The Portland Common Data Model (PCDM) is a flexible shared, linked data-based domain model for representing complex digital objects.  This workshop will review PCDM, its history, technical overview, recent developments, and Hydra-specific implementation considerations.  The workshop will also include an interactive modeling session where users will employ use cases from their repositories (or provided samples) to model in PCDM. The goals of the workshop include: increasing familiarity with PCDM, contributing back to PCDM from the activities of the participants, and increasing participants’ familiarly and comfort with data models more broadly.
Audience: all, but prior experience with data modeling (such as the "Basic introduction to linked data and related concepts in Hydra" session) will help greatly.
Two other things:

We'd like a standard projector setup, and also a separate space for diagraming.  A big whiteboard would be ideal, but easels or giant post-it paper we could stick to the wall would be OK too.

Attendees who want to walk through CurationConcerns to see how the data is modeled in Hydra and stored in Fedora and Solr should setup a Vagrant virtual environment following the instructions at https://github.com/projecthydra-labs/hydra-vagrant.


build: update style lint

PR Checklist
Please check if your PR fulfills the following requirements:

[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)
[ ] If applicable, have a visual design approval

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[x] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] clarity.design website / infrastructure changes
[ ] Other... Please describe:

What is the current behavior?
The style lint currently only understands CSS and errors a lot of Sass syntax incorrectly. This makes the build output noisy with warning log statements.

Issue Number: N/A
What is the new behavior?
The build logs are now cleaned up and don't have all the sass warnings. The style linter now errors instead of warns and understands Sass syntax. Two of the default errors are disabled as they cause a lot of noise and will require more refactoring of component styles at a latter time.
Deletions in styles for this PR are primarily duplicate properties or imports.
Does this PR introduce a breaking change?

[ ] Yes
[x] No

Other information

@Jinnie The scss deletions are duplicate properties or imports that already exist. The block deleted from the home page is from duplicate unnecessary statements. Duplicate statements (background) for things like gradients are already automatically done by the Angular CLI via a post css plugin. We technically don't event need that since our browser support does not even go back that far.


[Hugo 0.18] Error: Unable to find theme Directory

I am trying to start the server with hugo server inside the theme root directory.
I got the following error
Error: Unable to find theme Directory: C:\homepage\hugo-goa\themes\hugo-goa

Like:
- main-project-name
   -- data
   -- layout
   -- themes
      --- hugo-goa```


slot_leader table error  (erikd/shelley-32byte-address-hash)

I mean there is error with slot leader table (bad slot leader hashes, there are block hashes instead leader id); this is reason why
1 slot_leader hash = 1 row blocks table instead of 1 slot_leader = N rows blocks table
the current solution doesn't make sense to me and is somewhat different from actual byron mainnet solution

Can't really fix and validate this until the lower level libraries are fixed.

https://github.com/input-output-hk/cardano-ledger-specs/issues/1529

works great, good job @erikd thanks

This should now be fixed on master.


Cannot disable see through

Once the see through button has been pressed, a transparent radius surround the cursor as expected but I cannot disable by clicking again on the button.

Done


[SPARK-27448][SQL] File source V2 table provider should be compatible with V1 provider

What changes were proposed in this pull request?
In the rule PreprocessTableCreation, if an existing table is appended with a different provider, the action will fail.
Currently, there are two implementations for file sources and creating a table with file source V2 will always fall back to V1 FileFormat. We should consider the following cases as valid:

Appending a table with file source V2 provider using the v1 file format
Appending a table with v1 file format provider using file source V2 format

How was this patch tested?
Unit test

Test build #104543 has started for PR 24356 at commit 6c09fd7.

The issue is found in MetastoreDataSourcesSuite while migrating Parquet to file data source V2. It blocks the migration. We should

allow such scenarios and consider the v1/v2 providers as compatible
or
2.change the existing tests.

Option 1 seems more reasonable.

@cloud-fan

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9868/
Test PASSed.

Test build #104543 has finished for PR 24356 at commit 6c09fd7.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104543/
Test FAILed.

retest this please.

Test build #104548 has started for PR 24356 at commit 6c09fd7.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9872/
Test PASSed.

Test build #104548 has finished for PR 24356 at commit 6c09fd7.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104548/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9908/
Test PASSed.

Test build #104585 has started for PR 24356 at commit c145255.

Test build #104585 has finished for PR 24356 at commit c145255.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104585/
Test PASSed.

thanks, merging to master!


0770 chmod status

Hello, why are the permissions set to 0770 in the code? We build a package that gets installed on server with root:root. 0776 would be a more logical settings and following the other binaries composer makes.

Hello Tom.
Ok, let's discuss this...

why are the permissions set to 0770 in the code?

The permissions are set to 0770 to allow read+write+exec for user and group, but not for others.
0776 means that others get read+write, but not exec permissions.
My 0770 is more secure, than having an others-readable, others-writable binary file.
770 should cover the use-case that users install the Phantom binary as normal user on a production box. This is already a worst case, because Composer is a develop tool, which isn't supposed to run on a production box. It's also a reminder, that users needing to exec the file should be members of a group having access to it.
When your other binaries have 0776, then the current umask must be 0001,
because Composer subtracts permissions using 0777 & ~umask().
776 is not a good setting for binaries on a production box.
It's not clear to me why 0776 would be a more logical setting.

We build a package that gets installed on server with root:root.

Also the issue isn't clear to me,
because when you are root during install, you can set/alter the perms like you want, right?
I guess the underlying issue is, when you install/deploy as root you need to adjust the permissions of your app, because installing as root means that users are not allowed to alter or exec the files.
Why not simply change the ownership of the binary to the specific user/group for that folder or use a custom user to execute phantom stuff on the production machine?
Regards, Jens

Hello jakoch,
Sorry, I ment 0775 off course! Not 0776...
Composer doesnt run on the production machines. It runs on package-build machines and the debs get installed on the staging and production machines.
On staging we run the smoke tests first automated against phantomjs and afterwards we run the whole test suite against a selenium grid and multiple devices/browsers/... Packages get installed as certain system users but all developers/testers should be able to run the scripts in which phantomjs gets started in webdriver mode. That's why I dont see why this package should have it's own permissions set.

I do not see the need to change this.
It seems to work quite well for the majority of users.
I've described two possible workarounds.
Closing.

Strange, we could have made it work for even more than just the majority of users while not breaking it for the previous majority... i.e. have made it better.

Delete this line and everybody wins.
chmod($targetName, static::PHANTOMJS_CHMODE);

This change was applied (https://github.com/jakoch/phantomjs-installer/commit/bca1f360dffd56dae37b5eb01316eeef75ec3776#diff-05763998e682ea40296deedf66294cf4R382).
It ships starting with version 2.1.1-p09.


Update licensing-service-plan-reference.md

Added MCOPSTN5 - Domestic Calling Plan (120 min calling plan)

@msbemba : Thanks for your contribution! The author(s) have been notified to review your proposed change.

You can only sign off in the main PR conversation, not in review comments. In general, the creator of the PR should sign off, not the tech reviewer.


False-Positive result when checking for Unused Parameters

Current Behavior
False-Positive Violation when having two parameters with different parameter id (definition.parameters.${parameter_id}), but the same parameter name (definition.parameters.${parameter_id}.name.${parameter_name})
The behavior is incorrect since the usage of the parameter can be verified only by the usage of the link to the parameter. And this is done via parameter_id.
How to Reproduce
Part of the API definition to reproduce the behavior:
parameters:
  identifier_type:
    name: identifier_type
    description: Specifies the type of the customer identifier.
    in: path
    type: string
    x-extensible-enum:
      - customer_hash
      - cookie_id
      - fullvisitorid
      - client_id
      - idfa
    required: true

  target_identifier_type:
    name: identifier_type
    description: Specifies what type of customer identifiers to be returned
    in: query
    type: string
    x-extensible-enum:
      - customer_hash
      - cookie_id
      - fullvisitorid
      - client_id
      - idfa
    required: false

paths:
  /identifier-types/{identifier_type}/source-ids/{source_identifier}:
    get:
      tags:
        - id-mappings
      description: List of identifiers associated with the source id.
      parameters:
        - $ref: '#/parameters/identifier_type'
        - $ref: '#/parameters/source_identifier'
        - $ref: '#/parameters/target_identifier_type'
      responses:
        200:
          description: The identifiers associated with the source id.
          schema:
            $ref: '#/definitions/IdMappingResults'
        401:
          description: User is not authenticated
        403:
          description: User is not authorized
        404:
          description: Identifier is not found
          schema:
            $ref: '#definitions/Problem'
      security:
        - oauth2:
          - 'cross-device-graph-service.read'

Expected Behavior
Identifying of the unused parameters should be done based on the parameter_id instead of parameter_name.

I don't think the specified case is a valid swagger doc. parameter.name is the essential field that defines which parameter is which.
Proof №1: check the docs
Open this link and scroll a tiny bit up, because the header covers the title: https://swagger.io/specification/#fixed-fields-45
That's the spec for the parameters, and it says the name is required and only it identifies the parameter's name. Not the tagname above.

Proof №2: try the presented specification
Go here: http://editor.swagger.io and paste the api definition, presented in this issue above (simplified):
swagger: '2.0'
info:
  title: Product Service
  version: 1.0.0

paths:
  '/identifier-types/{identifier_type}/source-ids':
    get:
      parameters:
        - $ref: '#/parameters/identifier_type'
        - $ref: '#/parameters/target_identifier_type'
      responses:
        '200':
          description: The identifiers associated with the source id.
parameters:
  identifier_type:
    name: identifier_type
    description: Specifies the type of the customer identifier.
    in: path
    type: string
    required: true
  target_identifier_type:
    name: identifier_type
    description: Specifies what type of customer identifiers to be returned
    in: query
    type: string
    required: false

In the panel to the right, that shows the generated UI you can see that there is only one parameter: target_identifier_type is omitted. But if you change it to name: target_identifier_type - it appears in the UI as it should.
Proof №3: Java ignores it
In tests we do SwaggerParser().read("api_definition.json")
When I insert the presented example definition above into file, I can see in debug mode that the get operation node of the parsed swagger doc contains only 2 parameters, not 3. I think it matches them by names and therefore overrides the conflicting one.
Conclusion
Any thoughts? I think it's not a valid spec and the issue should be closed.

@innokenty Thank you for the research! Closing the issue.


Error in file(con, 'r") : cannot open the connection when user input incorrect filename into metadata parameter in `qza_to_phyloseq``

Desired behavior would be to output a more useful error message.
Currently, the following code
library(qiime2R)
qza_to_phyloseq(features = "features.qza",
                tree = "tree.qza",
                taxonomy = "taxonomy.qza",
                metadata = "metadata.tsv")

produces the following error if file metadata.tsv doesn't exist:
Error in file(con, "r") : cannot open the connection

which is not very informative.
This is due to the fact that there is no check for existence of a file in the body of is_q2metadata function.
is_q2metadata <- function(file){
  suppressWarnings(
  if(grepl("^#q2:types", readLines(file)[2])){return(TRUE)}else{return(FALSE)}
  )
}

I propose the following, simple change to provide users with a more informative error message in such cases.
is_q2metadata <- function(file){

  if (!file.exists(file)){stop("Input metadata file (",file,") not found. Please check path and/or use list.files() to see files in current working directory.")}

  suppressWarnings(
  if(grepl("^#q2:types", readLines(file)[2])){return(TRUE)}else{return(FALSE)}
  )
}

I opened a pull request with proposed change #63


it does not work

I have tried for many times, without success. Could you help to investiage?

It doesn't work at times. I tried turning it on and make a prompt and usually it will generate the results and prompt ChatGPT with a preset, right? But it doesn't and ChatGPT just answers my original question.

I have the same problem. The extension doesnt work for me. I enabled it but doesnt help

@qunash it seems your code broke and is no longer working on the site as of today. Likely they made some changes.

Hi all, this is really weird because it still works for me...
Can you provide console logs if thre are any errors?

Here there is the screen of mi console:

Please, check if it works now

No, still not working. Do you want me to upload a new screenshot?

I keep getting the error, ChatGPT cant browse the internet...? Something broke?

Same problem. It worked one time, but never after. No idea why

@Carnagal please provide more details/screenshot

Hello, well your question surprised me so I tested with a more simple question and it worked. I think my request is just too hard for it, It needs to go deeper in web. Sorry for my mistake and may be later with further improvements, it will work.

I'm curious to know which prompts it can't handle. Could you list them here or on the Discord server?

I'm curious to know which prompts it can't handle. Could you list them here or on the Discord server?

I have written it in french to make it the most precise I could. But it is a big fail :)
Je vais écrire une liste des #tag de joueurs clash royal ainsi que leurs pseudonymes. Il est possible de savoir dans quel clan chacuns se trouvent avec royal.api  Comme Carnagal https://royaleapi.com/player/Q8LC888Y qui se trouve dans le clan Band of adultes.
Donne moi la liste des pseudos suivi du clan dans lesquels ils se trouvent.
#9G9VVGCR	Magma
#QYP9VGRJR	Maxybestof
#Q9VQLY8	Dofla
#8URYRGPJ	Bembie

I'm curious to know which prompts it can't handle. Could you list them here or on the Discord server?

I have same problem, it doesn't work.

And my console logs


[FR] Allow managing of multi-factor users

Is your feature request related to a problem? Please describe.
It would be great if we could manage multi-factor users like described in https://cloud.google.com/identity-platform/docs/admin/manage-mfa-users
Describe the solution you'd like

Add multi_factor property to UserRecord
Add multi_factor arguments to create_user and update_user

Describe alternatives you've considered
None
Additional context
None

/assign @pragatimodi

Hi, is there any update about this? thanks!

We also need the mfa info exposed in the python admin sdk, is there any updates on this?

Do you have any workaround for managing users with multi-factor authentication? Thank you!


Update io.js version

I realize io.js is a fast moving target, but a few things have become more stable since the 1.0.0-pre version currently included. In particular, the nan module has been updated a couple of times to improve the compatibility of native modules with io.js.
For example, building leveldown for leveldb fails against 1.0.0-pre, but works fine on 1.1.0.

Compiling native modules can be a b*tch on 0.21.x, so :+1:
David Manning<EMAIL_ADDRESS>schrieb am Mo., 9. Feb. 2015 08:41:

I realize io.js is a fast moving target, but a few things have become more
stable since the 1.0.0-pre version currently included. In particular, the
nan module has been updated a couple of times to improve the compatibility
of native modules with io.js.
For example, building leveldown for leveldb fails against 1.0.0-pre, but
works fine on 1.1.0.
—
Reply to this email directly or view it on GitHub
https://github.com/atom/atom-shell/issues/1113.

Since posting this I've come to understand that the io.js / node version is tied to the version of v8 used in the packaged version of chrome. io.js 1.1.0 uses v8 <IP_ADDRESS>, which roughly matches v8 <IP_ADDRESS> in Chrome 41, which is still in beta. Seeing as how the author is undoubtedly aware of all this and it's part of the normal release cycle, I'm closing my issue.


An exception is reported when a context file path is longer than 100 chars

The following error is reported when the building context includes long file paths
java.lang.IllegalArgumentException: file name '.docusaurus/docusaurus-plugin-content-docs/default/category-docs-tutorialsidebar-category-tutorial-basics-918.json' is too long ( > 100 bytes)
        at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.handleLongName(TarArchiveOutputStream.java:702)
        at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(TarArchiveOutputStream.java:366)
        at io.seqera.wave.util.Packer.makeTar(Packer.java:82)
        at io.seqera.wave.util.Packer.layer(Packer.java:136)
        at io.seqera.wave.util.Packer.layer(Packer.java:132)
        at io.seqera.wave.util.Packer.layer(Packer.java:120)
        at io.seqera.wave.cli.App.prepareContext(App.java:459)
        at io.seqera.wave.cli.App.run(App.java:398)
        at io.seqera.wave.cli.App.main(App.java:217)

A quick search shows that it should be enabled the support for long file names in the tar using the following:
TarArchiveOutputStream stream = new TarArchiveOutputStream(...)
stream.setLongFileMode(TarArchiveOutputStream.LONGFILE_POSIX)

Read more here.

@pditommaso fix has been made in libseqera
https://github.com/seqeralabs/libseqera/pull/8
Once this is merged and a new version of wave-util is available
I will create PR here too


Instrument with shimmer and hook

Pull Request Test Coverage Report for Build 945

0 of 0   changed or added relevant lines in 0 files are covered.
193 unchanged lines in 9 files lost coverage.
Overall coverage decreased (-0.5%) to 66.679%

Files with Coverage Reduction
New Missed Lines
%

dist/plugins/integrations/MySQLIntegration.js
9
63.86%

dist/plugins/integrations/MySQL2Integration.js
11
66.27%

dist/plugins/integrations/MongoDBIntegration.js
12
68.89%

dist/plugins/integrations/RedisIntegration.js
12
69.77%

dist/plugins/integrations/IORedisIntegration.js
13
67.02%

dist/plugins/integrations/ESIntegration.js
16
67.67%

dist/plugins/integrations/PostgreIntegration.js
16
62.73%

dist/plugins/integrations/HttpIntegration.js
21
68.15%

dist/plugins/integrations/AWSIntegration.js
83
69.48%

Totals

Change from base Build 931:
-0.5%

Covered Lines:
2400

Relevant Lines:
3354

💛  - Coveralls


feature request: batched get_optimistic_exploration_action

Would it be straighforward to implement a batched version of get_optimistic_exploration_action?

Hi Samuel,
It is doable (a few hours effort), but not straightforward (< 20 minutes).
This is because:

the function get_optimistic_exploration_action requires computing one backward pass and per-sample gradient (and not sum of gradients over a batch of input).
AFAIK, pytorch doesn't support computing per-sample gradient.


Use references to the current skin in storyboard sprites

Storyboard sprites can sometimes refer to the current user's skin sprites. This happens only if beatmap's general section contains a property named UseSkinSprites. And again, as an example, you can look at this beatmap. Let's skip the fact that there's a whole bunch of broken spinners (that's for later). Then something even more terrible begins. This is called the "Go backwards" part, and it draws sliders and circles using the storyboard.
A piece of the storyboard that is responsible for this is here.
Source on the osu! wiki is here.

This one has the same thing:
https://osu.ppy.sh/beatmapsets/997827#osu/2087153

osu!lazer ignores UseSkinSprites when using a non-legacy skin, so the current behavior is the same as osu!lazer with the default skin:

AFAIK, there are a few possible solutions to this issue:

Ignore the UseSkinSprites option, making skin sprites invisible.
Display the sprites using the default osu skin (/src/play/resources/osu/osu.Game.Resources/Skins/Legacy/*.png), leading to inconsistency between the skin sprites and the actual skin.
Create a "legacy" renderer, like osu!lazer's, for maps like this. This also opens up the possibility of supporting map skins and custom skins, but adds a lot of complexity.

osu!lazer ignores UseSkinSprites when using a non-legacy skin, so the current behavior is the same as osu!lazer with the default skin

Not really? If you check this line in source code you will see that drawables are actually consider this option. I've debugged through the loading process of the "hitcircle.png" texture and i found out that osu! lazer just can't find file path if you are using a non-legacy skin (maybe this is only relevant for triangles skin, idk). If you are using triangles skin then at this moment file to storage path mapping will be empty for some reason, while a legacy one has all mappings.

The triangles skin is rendered almost entirely with primitives instead of sprites, so (almost) all UseSkinSprites references are not applicable when it is being used.


"Best" imagery not automatically selected

e.g. this location should show the Cape Town imagery instead of Bing:
https://www.openstreetmap.org/edit#map=19/-33.94348/18.49801
Thanks for the report @Firefishy!
It's a lat/lng - lng/lat issue caused by #3554

Any chance this release can be sped up a bit ?   It might not sound/look like a big issue but in .BE we've noticed this by spotting a bad edit based on some very obsolete background layer.  It would really help to prevent bad OSM data being entered.   aka: is it possible to hot-fix this today ?  Thanks a lot

Yes, I'll make a patch release this morning..

Much Appreciated!


⚠ Bofrost.es has degraded performance

In f5867cc, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1814 ms

Resolved: Bofrost.es performance has improved in 3872d35 after 6 minutes.


page does not refresh when press browser back button, if hash history is being used

what happed
1, create a default project by using preact-cli.
2, then following the preact-router document to add createHashHistory
3, then build the project, but a error will occur, just like mentioned below:
#377
4, after adding those build parameters --no-sw --no-ssr --no-esm --no-inline-css --no-prerender, the build can be done.
5, then run npm run serve to start dev server
6, open localhost:8080, then click Me, then click John
7, click the back button of browser several times.
8, the wrong page will be seen.
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

I had the same issue. I changed https://github.com/preactjs/preact-router/blob/master/src/index.js#L188 from:
this$1.routeTo(("" + (location.pathname || '') + (location.search || '')));
to:
this$1.routeTo(("" + (location.location.pathname || '') + (location.location.search || '')));
This might break other things but works for me.

Same issue here...
Browser back+forth doesn't work with hash history using<EMAIL_ADDRESS>with<EMAIL_ADDRESS>and<EMAIL_ADDRESS>It works fine with<EMAIL_ADDRESS>🤷♂

can confirm that using<EMAIL_ADDRESS>fixes it for me


buildsystem: deprecate perkeepd goui

buildsystem: deprecate perkeepd goui

- GopherJS is no longer required to build the perkeepd ui.

- Rename the `-buildWebUI` flag to `-buildPublisherUI`. Only the
  publisher needs gopherjs.

- Adapt `make.go` so that it no longer tries to generate the perkeepd
  web interface with GopherJS.

cc @bradfitz for review!

cc @bradfitz for review!

I haven't timed it but I am noticing faster load times. It makes sense because the goui was embeding a ton of stuff from the go stdlib.

I haven't timed it but I am noticing faster load times. It makes sense because the goui was embeding a ton of stuff from the go stdlib.


Changing the minimum log level at runtime

Is there a way to change the minimum log level at runtime? I'd like to be able to set the current log level from an action in a Asp.net core controller, so that I can diagnose issues without redeploying the application (it is running under Azure Service Fabric).
Thank you

You can implement it on multiple levels.

Wrap every TrackTrace call into your custom wrapper that can be controlled from outside. This wrapper will check severity and filter out unwanted traces.
Use log adaptors for Log4Net, SeriLog or whatever you like. Control how much trace they emit using their native APIs. In this case Application Insights will only receive traces you configured
Create a telemetry filter that will filter out traces based on severity. Control this processor in any way you'd like.

First approach is simplest and most performant, but has very limited use.
Second approach allows to trace verbose log into file while Error logs are sent to Application Insights. You pay for conversion from traces to application Insights events for events that will be send to AI.
Third approach is heaviest as you always convert strings to AI events. The benefit is that you can place Live Stream processor BEFORE this filter and get ALL traces while looking at telemetry "live". So you have more power troubleshooting ongoing issues.

Please re-open if you need more details. Create a user voice suggestion if you have more sophisticated scenario on mind: http://aka.ms/aiuservoice

Would be nice if this could be configured via an app insights config file.

You can do it via config, It will look like this:
<TelemetryProcessors>
  <Add Type="ApmTips.SeverityLevelFilter, ApmTips.dll">
    <SeverityLevel>Warning</SeverityLevel>
  </Add>
  <!--OTHER PROCESSORS-->
  <Add Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryProcessor, Microsoft.AI.ServerTelemetryChannel">
  <MaxTelemetryItemsPerSecond>5</MaxTelemetryItemsPerSecond>
  </Add>
<!--OTHER PROCESSORS-->
</TelemetryProcessors>

More on Application Insights config extensibility you can read here

Thank you for suggestions, @SergeyKanzhelev
I think that both Serilog and telemetry filter are valid options. Especially the second one, I didn’t know about it until now.
I still think that especially in asp.net core this should be kind of “out of the box”, but I like and accept your detailed  answer.

Thanks for the suggestions. We're going to further investigate Serilog. It looks very promising.

Just an update: Serilog + ASP.NET Core + Application Insights works great.

@leftside glad to hear that!


DownloadError: ERROR: ffprobe or avprobe not found. Please install one.

I keep getting this error DownloadError: ERROR: ffprobe or avprobe not found. Please install one.
I am on ubuntu 14.04 and I already did this
sudo apt-get install -y libav-tools

and ffmpeg is showing as installed
root@www:/home/www/yout# ffmpeg -version
ffmpeg version 2.6.3 Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1)

Here is the input I'm putting into the youtube-dl

temp_filepath = 'temp/%s-%s.%s' % (uuid.uuid4(), output_file, '%(ext)s')

    ydl_opts = {
    'verbose': True,
    'format': 'bestaudio/best', # choice of quality
    'outtmpl': temp_filepath,  # name the location
    'noplaylist' : True,        # only download single song, not playlist
    'postprocessors': [{
      'key': 'FFmpegExtractAudio',
      'preferredcodec': 'mp3',
      }],
     }

And here is the error
Traceback (most recent call last):
  File "/home/www/env/local/lib/python2.7/site-packages/rq/worker.py", line 557, in perform_job
    rv = job.perform()
  File "/home/www/env/local/lib/python2.7/site-packages/rq/job.py", line 492, in perform
    self._result = self.func(*self.args, **self.kwargs)
  File "/home/www/yout/worker.py", line 100, in extract_audio
    extraction_result = start_extraction(url, audio_filename)
  File "/home/www/yout/worker.py", line 132, in start_extraction
    result = ydl.download([url])
  File "/home/www/env/local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1446, in download
    res = self.extract_info(url)
  File "/home/www/env/local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 662, in extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/home/www/env/local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 708, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/home/www/env/local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1150, in process_video_result
    self.process_info(new_info)
  File "/home/www/env/local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1429, in process_info
    self.post_process(filename, info_dict)
  File "/home/www/env/local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1495, in post_process
    self.report_error(e.msg)
  File "/home/www/env/local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 529, in report_error
    self.trouble(error_message, tb)
  File "/home/www/env/local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 499, in trouble
    raise DownloadError(message, exc_info)
DownloadError: ERROR: ffprobe or avprobe not found. Please install one.

Any help would be appreciated

Can you run ffprobe or avprobe? (that's why it's failing)
Side note: don't run youtube-dl as the root user, we try to make it as safe as possible but you should avoid running programs as root if possible.

I will change the user after I get it working.
I believe I can run ffprobe and avprobe
root@www:/home/www/yout# ffprobe
ffprobe version 2.6.3 Copyright (c) 2007-2015 the FFmpeg developers
  built with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1)

root@www:/home/www/yout# avprobe
avprobe version 9.18-6:9.18-0ubuntu<IP_ADDRESS>+fdkaac, Copyright (c) 2007-2014 the Libav developers
  built on Apr 10 2015 23:18:58 with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1)
Simple multimedia streams analyzer

Post the full output you get when running youtube-dl --verbose --extract-audio SOME_URL using the same user as your python code.

this is bizzar when I run that it gives me this..
(env)root@www:/home/www/yout# youtube-dl --verbose --extract-audio 'https://www.youtube.com/watch?v=PEVUFKfpp5Y'
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'--extract-audio', u'https://www.youtube.com/watch?v=PEVUFKfpp5Y']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.04.09
[debug] Python version 2.7.6 - Linux-3.13.0-52-generic-x86_64-with-Ubuntu-14.04-trusty
[debug] exe versions: avconv 9.18-6, avprobe 9.18-6, ffmpeg 2.6.3, ffprobe 2.6.3
[debug] Proxy map: {}
[youtube] PEVUFKfpp5Y: Downloading webpage
[youtube] PEVUFKfpp5Y: Extracting video information
[youtube] PEVUFKfpp5Y: Downloading DASH manifest
[debug] Invoking downloader on 'https://r13---sn-aiglln76.googlevideo.com/videoplayback?id=3c455414a7e9a796&itag=141&source=youtube&requiressl=yes&mm=31&mn=sn-aiglln76&mv=m&ms=au&pl=48&nh=IgpwcjAzLmxocjE0KgkxMjcuMC4wLjE&ratebypass=yes&mime=audio/mp4&gir=yes&clen=3650358&lmt=1384815776052266&dur=114.474&fexp=901803,9406617,9407992,9408092,9408142,9408420,9408697,9408710,9409203,9413140,9413503,9414936,9415160,9415304,9415663,9415752,962731&mt=1433412898&signature=266ECED52115968C4E295BFFB03A30A4C64B1498.08626664DE30B666A5F8BC0121424F85C70C0929&sver=3&upn=rkYBKqv-OyQ&key=dg_yt0&ip=2a03:b0c0:1:d0::3ea:5001&ipbits=0&expire=1433434627&sparams=ip,ipbits,expire,id,itag,source,requiressl,mm,mn,mv,ms,pl,nh,ratebypass,mime,gir,clen,lmt,dur'
[download] Destination: 9 months old twins dancing and trying to get the music back-PEVUFKfpp5Y.m4a
[download] 100% of 3.48MiB in 00:00
[ffmpeg] Correcting container in "9 months old twins dancing and trying to get the music back-PEVUFKfpp5Y.m4a"
WARNING: Your copy of avconv is outdated, update avconv to version 10-0 or newer if you encounter any errors.
[debug] ffmpeg command line: avconv -y -i '9 months old twins dancing and trying to get the music back-PEVUFKfpp5Y.m4a' -c copy -f mp4 '9 months old twins dancing and trying to get the music back-PEVUFKfpp5Y.temp.m4a'
[debug] avconv command line: avprobe -show_streams '9 months old twins dancing and trying to get the music back-PEVUFKfpp5Y.m4a'
[youtube] Post-process file 9 months old twins dancing and trying to get the music back-PEVUFKfpp5Y.m4a exists, skipping

Post the output you get when running this python script (which I guess is similar to what you are using):
from youtube_dl import YoutubeDL
ydl_opts = {
    'verbose': True,
    'format': 'bestaudio/best', # choice of quality
    'outtmpl': 'example.%(ext)s',  # name the location
    'noplaylist' : True,        # only download single song, not playlist
    'postprocessors': [{
        'key': 'FFmpegExtractAudio',
        'preferredcodec': 'mp3',
    }],
}
ydl = YoutubeDL(ydl_opts)

ydl.download(['https://www.youtube.com/watch?v=PEVUFKfpp5Y'])

If that works but you script still fails, post the code you are running and its output so that we can try to reproduce it.

Ok, figured it out.  It was working fine when it was not in supervisor.  Once I put the app under supervisor the issue started.  Didn't click until just now.
In the supervisor config file I had to add the location of ffprobe in ontop of the virtual env directory as seen below.
environment=PATH="/home/www/env/bin:/usr/bin"

Thank you so much for your help!


Triage issue link not valid (should validate links?)

Hi there,
Just received an email with some issues to triage but one of the issue links is no longer valid (results in 500 error). Not sure if the site code validates the issue links before sending them to us? If not, might be handy. Here's the link that failed :
https://github.com/gugod/App-perlbrew/issues/305
Thanks.

Hello Erick,
The link you shared just worked for me. Did you receive any other broken links ?
If you had a 500 error, it might be on GitHub's end, and we should consider closing this issue.
Links aren't validated before sending (part of the code where the link is injected in the email).
Reguards.


fix: event hero padding right removed

Description
In addition to venue hero, same style fixes applied to the event hero
Issues
Closes
DEV-XXX:
Related
Testing
Automated tests
Manual testing
Screenshots
Additional notes

Hobbies-Helsinki branch is deployed to platta: https://harrastukset-pr407.dev.hel.ninja :rocket::rocket::rocket:

Sports-Helsinki branch is deployed to platta: https://liikunta-pr407.dev.hel.ninja :rocket::rocket::rocket:

Events-Helsinki branch is deployed to platta: https://tapahtumat-pr407.dev.hel.ninja :rocket::rocket::rocket:

TestCafe result is success for https://liikunta-pr407.dev.hel.ninja :laughing::tada::tada::tada:

TestCafe result is success for https://tapahtumat-pr407.dev.hel.ninja :laughing::tada::tada::tada:

EVENTS GRAPHQL PROXY is deployed to platta: https://events-graphql-proxy-pr407.dev.hel.ninja :rocket::rocket::rocket:

VENUE GRAPHQL PROXY is deployed to platta: https://venue-graphql-proxy-pr407.dev.hel.ninja :rocket::rocket::rocket:


The scipy online documentation returns 404 constantly

I can go into the main page of the scipy project. However when i try to check the documentation for the SciPy package through this link, it returns 404 page not found constantly.

Thanks for the fix!

Thanks a lot for the fix. Now it works totally.

[https://www.google.com/search?q=scipy.interpolate.poly.from+spline&rlz=1C1CHBF_enIN803IN803&oq=sc&aqs=chrome.2.69i60j69i59l3j69i57j69i60l3.1807j0j7&sourceid=chrome&ie=UTF-8](Search Results) not opening, returns error 404. Is it down again?

I have the same issue with intersphinx (can't reach 'https://docs.scipy.org/doc/numpy/objects.inv):
WARNING: failed to reach any of the inventories with the following issues: intersphinx inventory 'https://docs.scipy.org/doc/numpy/objects.inv' not readable due to ValueError: unknown or unsupported inventory version: ValueError('invalid inventory header: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">') WARNING: failed to reach any of the inventories with the following issues: intersphinx inventory 'https://docs.scipy.org/doc/scipy/reference/objects.inv' not readable due to ValueError: unknown or unsupported inventory version: ValueError('invalid inventory header: ')

@msmdev if you are looking to use the NumPy intersphinx URL this has moved to https://numpy.org/doc/stable/objects.inv
For SciPy see #15545

@melissawm: this and #15545 solved my problem - thank you very much!


SigNoz

Project: https://signoz.io/
License: Proprietary, MIT
Open-source claim: front page, description, README, ...
Discussion: TODO

Thanks @illiliti.
When it comes to mixed-license repos (Or "open core" projects in general), I generally wouldn't consider them misleading within the scope of this project unless:

The open offering cannot be built from open source code alone.
The open offering doesn't provide the core functionality/benefit being advertised mentioned.
Their marketing combines open source with the mention of the non-open components, presenting a potentially misleading scenario to users.

Not sure if Signoz currently meets any of those.
I'm not a fan of open core set-ups, and I really wish that if they took this approach they'd at least keep the non-open parts completely separate, but it gets more nuanced in if they're specifically being misleading and somewhat becomes opinion on their project setup, monetization methods or business approach.

If they were clearly distinguished, there would no issue. But when I see something is open-source, I expect that the whole bundle of source code is open-source. What I see instead? Some code is MIT, some code is proprietary. How is that not misleading?

By the same logic Skiff isn't guilty because it has some open-source code and some proprietary. Does it mean it has right to advertise itself as open-source? No because it is either fully open-source with no exceptions or not at all. Same here, either open-source or not at all. There is no middleground.

How is that not misleading?

From my view, it comes down to what they're advertising as open source vs what's actually open source. If I can't use their project from open source code, and gain the main features/functionality/purpose that's being advertised as open source, then that's an issue. If they have some optional components that they provide under a different license, that they choose to have within the same repo, then I don't think that's misleading as long as they're fairly clear and up-front about it. Again, I'd prefer they didn't mix repos like this, but it can somewhat be a preference/opinion of organization.
I respect that some may see the open core approach as misleading, but it gets closer to a gray area based upon expectations, hence my general rules above. To cover all open core approaches significantly changes the scope of this repo in my view.

By the same logic Skiff isn't guilty because it has some open-source code and some proprietary. Does it mean it has right to advertise itself as open-source?

I haven't looked deeply into Skiff, but it appears mostly licensed as non-open-source (CC BY-NC-SA) so I can't gain the advertised features/use-case/purpose/functionality from open source code alone, which seems quite different to SigNoz (again, haven't looked deeply to validate).

Aha so we have a different points of view. You say that if project advertises itself as open-source, then its claimed functionality must be available from the open-source code. I say that if project advertises itself as open-source, then all its source code must be open-source. From a certain point of view we are both right. Your viewpoint sound more relaxed than mine and therefore I don't like it. But I won't argue and simply concede. You can close this issue and other my issues that you believe don't align with your goals.


Running latest compiled zpool_influxdb command and receiving errors from Telegraf.

systemctl status telegraf
May 29 22:36:20 servert4 telegraf[51265]: 2020-05-30T03:36:20Z E! [inputs.exec] Error in plugin: exec: exit status 1 for command '/usr/local/bin/zpool_influxdb':
May 29 22:36:30 servert4 telegraf[51265]: 2020-05-30T03:36:30Z E! [inputs.exec] Error in plugin: exec: exit status 1 for command '/usr/local/bin/zpool_influxdb':
May 29 22:36:40 servert4 telegraf[51265]: 2020-05-30T03:36:40Z E! [inputs.exec] Error in plugin: exec: exit status 1 for command '/usr/local/bin/zpool_influxdb':

Output of comand:
zpool_stats,name=rpool,state=ONLINE alloc=14807851008u,free=162359549952u,size=177167400960u,read_bytes=593289216u,read_errors=0u,read_ops=14939u,write_bytes=242403319808u,write_errors=0u,write_ops=9853708u,checksum_errors=0u,fragmentation=0u<PHONE_NUMBER>045037386

Commit: bfba7d128b9bbc4bcb89d67df2c0d382fe126640

thanks for the report. can you share the output of modinfo zfs ?  This will show the zfs driver version and kernel version.

I suspect that the version of ZFS was upgraded, but zpool_influxdb was not compiled against the new ZFS source. I this case, gathering the scan/scrub stats could fail because it detects a state that is out of its known range. Recompiling against the same ZFS headers should correct it.

I am running CentOS 7 with ZFS 0.8.4 from ZFS on Linux repo.
[user1@servert4 ~]$ modinfo zfs
filename:       /lib/modules/3.10.0-1062.12.1.el7.x86_64/extra/zfs.ko.xz
version:        0.8.4-1
license:        CDDL
author:         OpenZFS on Linux
description:    ZFS
alias:          devname:zfs
alias:          char-major-10-249
retpoline:      Y
rhelversion:    7.7
srcversion:     E9712003D310D2B54A51C97

I also keep getting an issue with CMakeLists.txt about the sys/fs/zfs.h.
/usr/src/zpool_influxdb/zpool_influxdb.c:56:24: fatal error: sys/fs/zfs.h: No such file or directory

Error seems to persist even on newer kernel and headers should be correct for ZFS 0.8.4.

So there is a check for the scan/scrub stats that basically says "if I see something I don't recognize, bail out with error code = 1." I'm not particularly happy with that, but there isn't a very good way to communicate that through to telegraf.

If it makes sense, you can print a message to stderr and Telegraf should display it.

@tonymmm1 please verify the fix in commit 41a109b. Re-open if this doesn't fix.
@danielnelson I've added a log message to stderr in the case where we're out of sync with
the source codes and compiled versions of various bits. To prevent spamming, this message
is only printed once every 1000 iterations. Otherwise, best effort is made to get the other stats.


Top sites (pinned site shortcuts) are removed for a clean new tab page.

Is this possible to clear topsites (which is recently searched)? Some sites are getting injected into rows. I have activated this .js but I just visited codeberg and it got appeared into rows.

I don't use these. I believe you can remove them by right-clicking. If not, then no.
Alternatives

Bookmark keywords provide a better experience (YouTube example).
Learn how to narrow down suggestions in the URL bar by a specific type of result (* for bookmarks, ^ for browser history, etc.).

you can remove them by right-clicking.

Doesn't seem to work for me.
I can hover over them, click the three dots, and then click "Dismiss"

Thanks

@iam-py-test OP didn't show three dots in their post, but I do recall that being a thing. I haven't been able to test today.
Thanks. Cheers


[BUG] _do_handshake: TLS handshake error: -9984 and then Invalid operands 'String' and 'int' in operator '!='

Describe the bug
I am behind a school WiFi network what signs their own SSL certificates. When my code I see this:

And then this:

And it doesn't look like GodotFirebase does anything meaningful.
To Reproduce
Connect to a WiFi network that signs their own SSL certificates.
Run this after installing the GodotFirebase extension:
Firebase.Auth.login_anonymous()
var collection : FirestoreCollection = Firebase.Firestore.collection("leaderboards")
collection.get("TestDocument")
var document : FirestoreDocument = yield(collection, "get_document")
print(document)
Expected behavior
It should sign in anonymously and then fetch the /leaderboards/TestDocument document.
Screenshots
Added above
Environment:

OS: Windows 10
Browser: N/A running project in GoDot Editor

Additional context
I think this is all I need but am happy to provide more error info.
If this is intended behaviour, is there a way to trust certificates or just turn off SSL verification (this is a project, I don't really care about person-in-the-middle attacks).
Thank you for making this amazing repo and for reading through my issue and have a great day,
Kotlin_for_win

Hi! @BearDooks I've seen you solve so Firestore issues before. I was just wondering if you could please help out with this. Thank you for making such a great repo, it allows me to use my favourite cloud service on my favourite game engine and is fantastic!

Hey @Kotin-for-win, I was just reading up on this. I need to see if I can find that error in the documents somewhere, but best guess is the school WIFI is brokering the SSL connection for you and for some reason they are not trusting the connection, or Google does not like the man in the middle.
If you test this outside of this network does it work fine?

Yes, it does. Thank you so much for your help.

Do you know if there is a workaround, or will I have to only work on non-school WiFi? Thanks again, Kotlin-for-win

@BearDooks , I just discovered that I get _do_handshake: TLS handshake error: -9984 when I try and download the export templates within the Godot Editor. I think the error must have something to do with Godot's internal web request handler, and it not liking the self-signed SSL certificates.
Do you know if it is possible to turn off this SSL certification or trust my IT Department's certificate within Godot Engine?
Or is it possible for this to be disabled by calling a GDScript web request differently in GodotFirebase?
Thank you so much for helping out!

It's all resolved!
Sorry for wasting your time @BearDooks.
I did some more research and found that this was indeed an issue with Godot's internal web request handler.
I followed this tutorial to import my IT Department's certificate into GoDot's truststore, and everything works great now.
In case anyone encountering this issue finds this, I resolved this issue by following these steps (WINDOWS):

Go to a website like google.com in chrome white connected to your company/school WiFi.
Press the padlock -> Connection is secure -> Certificate is valid
In the window that appears, navigate to Certification Path
You will see three certificates, press the top one and then 'view certificate'
Go to details, then Copy to File
Continue with default settings and save as DER blah blah (.CER)
Once you see 'the export was successful', go to the folder where you saved the file.
Change it from blah.cer to blah.crt
Copy it to your project's res:// directory
In Godot editor go to Project -> Project Settings
Search for SSL
On certificates press the little file icon
The select blah.crt
And everything should work!

As a side note: since it's related to HTTP, one thing that could be tested is running the project while the certificate is untrusted, and setting the HTTPRequest Node policies to NOT verify SSL.
For example, here https://github.com/GodotNuts/GodotFirebase/blob/601d974ed268e96556e1ea0e8210e2a060cfc895/addons/godot-firebase/auth/auth.gd#L215
the third parameter, which is true, should be set to false, and then trying to execute such function

@Kotin-for-win you didn't waste time. Glad it worked out. Sorry it was late for me last night and I ended up falling asleep.
Is it ok if I take you fix and add it to the wiki?

*I think step 8 should be 'rename the file from blah.cer to blah.crt' to emphasize that it is just renaming, and no file converter is needed.

Please don't do this weird stuff with certs.
Latest version of Godot does connect just fine if you set up your ciphers right.
It is unforgivingly secure and the only real major downside is that it's not trivial to debug when your server isn't set up properly.


Is amcrest MQTT broken?

This has been working fine for me until this week. Im no longer getting any topic updates at amcrest2mqtt/serialno/doorbell and amcrest2mqtt/serialno/human etc... the event updates are working and I can see motion / doorbell presses but the sensor topics aren't updating at all.
Really confused by this. Have amcrest changed something with a firmware update?

i had to reset manual mqtt sensors to get this to work again. so my issue but may be of use if someone else comes across this.


FancyZones unexpected behavior for WIN+UP/DOWN

ℹ Computer information

PowerToys version: 0.27.1
PowerToy Utility: FancyZones
Running PowerToys as Admin: yes
Windows build number: [run "winver"] 1909 B 18363.1256

📝 Provide detailed reproduction steps (if any)

Given the settings as seen in the screenshot No.1, I set up my horizontal main monitor and secondary vertically pivoted monitor like you can see in screenshot No.2 and No.3.

I use the WIN + UP and WIN + DOWN hotkeys to maximize and minimize a window.

✔ Expected result
I should be able to minimize and maximize the window as usual on both screens.
❌ Actual result
It works flawless for the main (horizontal) screen, but does not work for as expected on the secondary (vertical) screen. I can minimize the window when it is in the second (i.e. final) row of the grid but not when it is on the top. I cannot maximize the window no matter where the window is (first or second/final row).
What is also unintuitive in this case is that I can only move the window from first and second row back and forth by using WIN+LEFT and WIN+RIGHT hotkeys. More intuitive and standard to windows is WIN+UP and WIN+DOWN.
📷 Screenshots
No.1 Settings:

No.2 Setup screen 1

No.3 Setup screen 2

@MichlF
this is the default behavior (bug) in Windows Snap, when a window occupies half of the screen horizontally, Windows Snap up/down keys are not maximizing/minimizing the window.
You can easily replicate this without FZ running.
I suggest you use the Windows Feedback Hub application to submit the bug against Windows Snap.

@enricogior: Maybe I dont understand you but it isn't default behavior in Windows Snap!?
With normal snap I cannot really occupy half of the screen horizontally in a pivoted screen. I can only occupy half of the screen vertically (which is why I looked up FZ in the first place).
Irrespective, as I explained, the real issue here is that maximize (WIN+UP or with any other hotkey combination) does not seem to work at all on a pivoted screen in FZ, while it does work flawlessly and in a predictable fashion without running FZ  (for any window, whether it is manually sized and placed or vertically placed with WIN+LEFT/RIGHT).

@MichlF

With normal snap I cannot really occupy half of the screen horizontally in a pivoted screen

If a window is already there, regardless how it has been put there (FZ or manually), Windows Snap has that behavior.
PowerToys doesn't have the ability to change how Windows Snap works.

I still think we talk past one another.
Do you agree that this behavior:

Irrespective, as I explained, the real issue here is that maximize (WIN+UP or with any other hotkey combination) does not seem to work at all on a pivoted screen in FZ, while it does work flawlessly and in a predictable fashion without running FZ (for any window, whether it is manually sized and placed or vertically placed with WIN+LEFT/RIGHT).

should be fixable? After all, it is something that works in Windows Snap without FZ but not with FZ enabled.
PS: I'd a screen recording to better illustrate what I mean but it seems that the game bar utility doesn't allow recording on a secondary screen.

@MichlF

the real issue here is that maximize (WIN+UP or with any other hotkey combination) does not seem to work at all on a pivoted screen in FZ, while it does work flawlessly and in a predictable fashion without running FZ

That is not correct.
If a window occupies the top half of the screen or the bottom half, Windows Snap will not maximize/minimize it.
To repro, simple zone the window on the monitor in portrait mode and then quit FZ. Windows Snap will still not maximize/minimize it.
FZ has nothing to do with that behavior.

Maybe the confusion here is coming from a different definition of "top". If I talk about a 90° pivoted screen, then the top is not the top of the screen for an unpivoted screen but the left (assuming you pivot the screen clockwise which is the default/standard).

To repro, simple zone the window on the monitor in portrait mode and then quit FZ. Windows Snap will still not maximize/minimize it.

Even though WIN+UP doesn't do anything here - you are correct in that - I can simply maximize the window by first doing WIN+RIGHT or +LEFT and then WIN+TOP. So again, the functionality clearly exists without FZ.

I can confirm issues here. If I disable fancy zones, Win+DN will minimize my window. When I enable fancy zones, it moves between zones. Same behavior with the override windows snap shortcut option.

@MichlF

Maybe the confusion here is coming from a different definition of "top". If I talk about a 90° pivoted screen, then the top is not the top of the screen for an unpivoted screen but the left (assuming you pivot the screen clockwise which is the default/standard).

Top-half of the screen is the same thing regardless of the orientation.
You can easily repro this on landscape as well (without FZ running).

Even though WIN+UP doesn't do anything here - you are correct in that - I can simply maximize the window by first doing WIN+RIGHT or +LEFT and then WIN+TOP. So again, the functionality clearly exists without FZ.

@MichlF
if you move the window somewhere else, that "fixes" the Windows Snap issue, buy again, this is not a FZ bug, this is Windows Snap that doesn't allow to maximize/minimize a windows if it occupies the top/bottom half of the screen.
So what you are asking is that FZ adds a workaround for this Windows Snap limitation or to be able to use both FZ and Windows Snap at the same time. Since Windows Snap doesn't allow to change the Win + arrow shortcut, FZ could allow it.
That feature request is tracked by https://github.com/microsoft/PowerToys/issues/2382 via https://github.com/microsoft/PowerToys/issues/6611


Fix exported resource references

Fixes #207 again :p
the value in \(varName) is always coming in as nil, and so reference() is never being called. I think this was supposed to be adding a reference to res anyways.
looks like incorrect use of \(varName) was introduced here: https://github.com/migueldeicaza/SwiftGodot/commit/3f3bf924a533039454af43d3d25b284e0c096858

After seeing that build error, I did some more investigating and I think the reason it failed is because oldRef is always nil, and so it never gets unreferenced. This just didnt show as a problem before because \(varName) was never being referenced in the first place and so there was no issue.
I will look in to get it to unreference properly

Thank you for tracking this down and submitting a fix.


Generate backdrop-filter.dist.yml

This will be a simple case after https://github.com/mdn/browser-compat-data/pull/22788 lands and we update BCD.

I think we should have a CI job that fails if we have listed compat_features that exactly match BCD. That would then fail when we roll a new version of BCD.
Anything that remains would be differences to BCD still in need of upstreaming.
If we have a better way to ignore specific BCD keys, we might be able to get rid of even more.

For this PR, we can just wait until the next BCD release I think.

We have a bunch of things in flight in BCD now, and can just make this another one of those.


Clean up fields 2

Flora has added an autofield primary key to Article. Formerly the dataset was implicitly expected to provide a primary key for articles.
User specified non-unique, non-primary key fields were renamed from _id to _number (e.g., article_number, question_number, answer_number), so that _id is reserved for fields that are primary keys. This change was propagated to serializers and views as required.

I manually reversed this within 4 minutes because the commit was attributed to me instead of Flora.

I squashed and merged these commits manually to master to create commit 201f252.


Update outdated TFRecords format detail reference

The link doesn't work anymore.

Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.
Post-Commit Tests Status (on master branch)

Lang
SDK
Apex
Dataflow
Flink
Gearpump
Samza
Spark

Go

---
---
---
---
---
---

Java

Python

---

---
---
---

On the other hand looks like TF website is broken itself, so I'll wait before merging it if it gets fixed

@akedin It looks good now. Thanks for checking!

If I search for tfrecords format details the first link is broken: https://www.tensorflow.org/s/results/?q=TFRecords Format Details
Same thing when you search for it in Google: https://www.google.com/search?q=tfrecords+format+details+python+io
It's unclear what's the canonical link should be

That's true. I'll keep an eye on this. Thanks!


macos/linux codecov cli, 0.3.8 throws an error

We download the latest codecov-cli to upload codecov results
CODECOV_CLI_VERSION=latest
CODECOV_OS_NAME=linux  # or macos
    curl https://keybase.io/codecovsecurity/pgp_keys.asc | gpg --no-default-keyring --keyring codecov-trustedkeys.gpg --import # One-time step
    curl -Os https://cli.codecov.io/${CODECOV_CLI_VERSION}/${CODECOV_OS_NAME}/codecov
    curl -Os https://cli.codecov.io/${CODECOV_CLI_VERSION}/${CODECOV_OS_NAME}/codecov.SHA256SUM
    curl -Os https://cli.codecov.io/${CODECOV_CLI_VERSION}/${CODECOV_OS_NAME}/codecov.SHA256SUM.sig
    gpgv --keyring codecov-trustedkeys.gpg codecov.SHA256SUM.sig codecov.SHA256SUM
    shasum -a 256 -c codecov.SHA256SUM

We then run the script
./codecov create-commit
./codecov create-report
./codecov --verbose --codecov-yml-path codecov.yml do-upload --dir coverage --flag unit --fail-on-error

When running in the context of our CI/CD (which is linux), we are seeing the following
Traceback (most recent call last):
  File "importlib/metadata/__init__.py", line 563, in from_name
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "codecov_cli/main.py", line 7, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 391, in exec_module
  File "codecov_cli/__init__.py", line 3, in <module>
  File "importlib/metadata/__init__.py", line 1008, in version
  File "importlib/metadata/__init__.py", line 981, in distribution
  File "importlib/metadata/__init__.py", line 565, in from_name
importlib.metadata.PackageNotFoundError: No package metadata was found for codecov-cli
[31] Failed to execute script 'main' due to unhandled exception!

When running locally on macos, we are seeing the following
Traceback (most recent call last):
  File "codecov_cli/main.py", line 5, in <module>
ModuleNotFoundError: No module named 'click'
[2961] Failed to execute script 'main' due to unhandled exception!

Same error on 0.3.8 on Windows, just by running the executable alone:
codecovcli.exe
Traceback (most recent call last):
  File "importlib\metadata\__init__.py", line 563, in from_name
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "codecov_cli\main.py", line 7, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 391, in exec_module
  File "codecov_cli\__init__.py", line 3, in <module>
  File "importlib\metadata\__init__.py", line 1008, in version
  File "importlib\metadata\__init__.py", line 981, in distribution
  File "importlib\metadata\__init__.py", line 565, in from_name
importlib.metadata.PackageNotFoundError: No package metadata was found for codecov-cli
[5776] Failed to execute script 'main' due to unhandled exception!

hey all, we are aware of the issue, it seems like something is wrong with the way we packaged the binary, although pip install does not have this issue. We have made a fix here and expect to deploy a new version shortly.

this should be resolved now on 0.3.8


Error: "3 INVALID_ARGUMENT: The Replay ID validation failed"

Component Bug Report
Description
SMG experienced the error: 3 INVALID_ARGUMENT: The Replay ID validation failed. Ensure that the Replay ID is valid. rpcId: f27efab0-ee6b-4f8c-949f-b6728e3b5412 in the Subscribe to PubSub trigger. This error occurs every 4 days in their RE-SF-User (pub/sub) flow.
Logs:
[2024-06-16 17:00:42.422]: Fetching credentials by secretId
[2024-06-16 17:00:42.422]: Starting "Subscribe to PubSub" trigger
[2024-06-16 17:00:42.515]: Preparing SalesForce connection...
[2024-06-16 17:00:42.942]: Connected to Pub/Sub API endpoint
[2024-06-16 17:00:43.534]: Schema loaded
[2024-06-16 17:00:43.534]: SalesForce connected
[2024-06-16 17:00:43.538]: Subscription will start from replayId: 341148058
[2024-06-16 17:00:43.567]: Requesting first 10 records
[2024-06-16 17:00:43.567]: Streaming client created
[2024-06-16 17:00:43.567]: Processing existing events if present and waiting for keepalive message to finish
[2024-06-16 17:00:43.846]: gRPC stream error: {"code":3,"details":"The Replay ID validation failed. Ensure that the Replay ID is valid. rpcId: f27efab0-ee6b-4f8c-949f-b6728e3b5412","metadata":{"content-type":["application/grpc"],"error-code":["sfdc.platform.eventbus.grpc.subscription.fetch.replayid.corrupted"],"rpc-id":["f27efab0-ee6b-4f8c-949f-b6728e3b5412"],"type":["Subscribe"],"date":["Sun, 16 Jun 2024 14:00:43 GMT"]}}
[2024-06-16 17:00:43.846]: gRPC stream ended
[2024-06-16 17:00:43.846]: Failed to get keepalive message to finish: 3 INVALID_ARGUMENT: The Replay ID validation failed. Ensure that the Replay ID is valid. rpcId: f27efab0-ee6b-4f8c-949f-b6728e3b5412
[2024-06-16 17:00:43.846]: gRPC stream status: {"code":3,"details":"The Replay ID validation failed. Ensure that the Replay ID is valid. rpcId: f27efab0-ee6b-4f8c-949f-b6728e3b5412","metadata":{"content-type":["application/grpc"],"error-code":["sfdc.platform.eventbus.grpc.subscription.fetch.replayid.corrupted"],"rpc-id":["f27efab0-ee6b-4f8c-949f-b6728e3b5412"],"type":["Subscribe"],"date":["Sun, 16 Jun 2024 14:00:43 GMT"]}}
[2024-06-16 17:00:43.859]: Error occurred during trigger or action processing

Link to the Slack conversation.
Link to the SMG conversation.
Component Version
2.8.3
Workaround(s)
Reset the snapshot and restart the flow. Note that the error will recur every 4 days.

https://github.com/elasticio/salesforce-component-v2/releases/tag/2.8.4


cloth config 2 required

cloth config 2 doesnt even exist, WTF is going on with launchers, are they on something?

It's confusing, yes. Cloth Config 2 refers to this mod: https://beta.curseforge.com/minecraft/mc-mods/cloth-config


Failed to launch external program msbuild

I followed the steps of Ankit Asthana from this video:
https://www.youtube.com/watch?v=B0xTgyCwsAo
with a simple "Hello World" example.
However, when I ">Run Build Task", I get an error:
"Failed to launch external program msbuild /property:GenerateFullPaths=true.
spawn msbuild ENOENT"
Tried to add MSBuild.exe (from VS 2017) to my PATH and the problem persists.
Any suggestions?

VSCode Version: Code 1.9.1 (f9d0c687ff2ea7aabd85fb9a43129117c0ecf519, 2017-02-09T00:26:45.394Z)
OS Version: Windows_NT ia32 10.0.14393
Extensions:

Extension
Author
Version

cpptools
ms-vscode
0.10.1

Steps to Reproduce:

@sursu can you please provide a git URL which contains your example. This helps tracking this down.

Hi, Dirk
Here is my example: https://github.com/sursu/vscode_test

Have you managed to replicate my error?

@sursu when trying with VS Code without any extension I get:
MSBUILD : error MSB1003: Specify a project or solution file. The current working directory does not contain a project or solution file.

with the cloned project https://github.com/sursu/vscode_test
Could this be a problem of the cpptools extension ?

With and without the extension, after I have added to the path the MSBuild from Visual Studio 2017, I get indeed:
MSBUILD : error MSB1003: Specify a project or solution file. The current working directory does not contain a project or solution file.

Closing then.

:) You may close it, but how do I deal with this problem?

I am not an msbuild expert but I think you either need to create a solution file or pass some other information. If you want to compile a single file then you can always call the compiler directly

Is it possible to call the compiler directly from VS Code? That is what I thought I'm doing with ">Run Build Task"...

Yes, but you need to list the compiler you want to call as a command in the tasks.json. You might want to have a look here for some examples: http://code.visualstudio.com/docs/editor/tasks

Hold up, does VS Code actually need MSBuild in the path? Or has it simply not been updated to find MSBuild in its new location for VS2017? If the latter this is a bug. If the former this is merely a usability bug (it should really look for MSBuild on Windows in reasonable, known locations...)

VS Code has not knowledge about msbuild location under Windows. All commands listed in tasks.json need to either be on the path or need to be references absolute.

I have now reinstalled VS Code.
The problem persists.
MSBUILD : error MSB1003: Specify a project or solution file. The current working directory does not contain a project or solution file.
I am curious if anyone in this world is able to build C++ applications from VS Code with msbuild and, if yes, how he/she/it does that.
I suspect that my problem has to do with the "includePath" from the c_cpp_properties.json file:
{
            "name": "Win32",
            "includePath": [
                "C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/include/*"
            ],
            "browse": {
                "limitSymbolsToIncludedHeaders": true,
                "databaseFilename": ""
            }
        }

My Visual Studio 14.0 is essentially non-existent (the folder has some files though).
When I change the path to the msbuild of Visual Studio 2017 no success is recorded.

try the following link its not flawless but it helps https://github.com/dunkan50/vscodecpp

@sursu the last error you received was valid, you did not provide a solution file (.sln) nor a project file (.csproj). For example, I got my project (WebUI) working with the latest 2.0.0 tasks configuration:
{
  "version": "2.0.0",
  "tasks": [
    {
      "taskName": "Build - Debug",
      "command": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Enterprise\\MSBuild\\15.0\\Bin\\amd64\\MSBuild.exe",
      "args": [
        "${workspaceRoot}/WebUI.csproj",
        "/t:Rebuild",
        "/p:Configuration=Debug;AutoParameterizationWebConfigConnectionStrings=False"
      ],
      "problemMatcher": []
    }
  ]
}

In my case msbuild was not in my Windows path, so I provided it in the task.


runOut: don't lose command output to stdout

Hi,
during my hackish attempt of adding gb to gin, I noticed that the command output of the runOut method is only printed to stdout in the error case.
This means that information and output of the compiler and other commands can't be read by tools using gb by themselfs. This PR adds a simple error type that captures the original error, the command args and their output.
Ps: see PR codegangsta/gin#73 if you want to play with it.

Thanks. This is fine but drop the output field, there is no standard way of printing an io.Writer.

drop the output field,

Hrm.. the juicy information (xyz is undefined, can't use foo as bar in assignment..) is in the command output. You only get exit status 1 without it.

there is no standard way of printing an io.Writer.

This was the most confusing thing to me when introducing the output field as string, I didn't notice is was a writer because it was printed like that.
Since only run() uses runOut() and it also didn't look at the passed buffers content afterwards, I dropped the run() wrapper and converted the exec.Command to use CominedOutput() instead of Run(). The output is only captured in the error case.
How does this look to you?

Have a look at the changes to this file landing in #103. I can break out
the changes to run/runOut if you need.
On Mon, May 18, 2015 at 7:35 PM, Henry<EMAIL_ADDRESS>wrote:

drop the output field,
Hrm.. the juicy information (xyz is undefined, can't use foo as bar in
assignment..) is in the command output. You only get exit status 1
without it.
there is no standard way of printing an io.Writer.
This was the most confusing thing to me when introducing the output field
as string, I didn't notice is was a writer because it was printed like that.
Since only run() uses runOut() and it also didn't look at the passed
buffers content afterwards, I dropped the run() wrapper and converted the
exec.Command to use CominedOutput() instead of Run(). The output is only
captured in the error case.
How does this look to you?
—
Reply to this email directly or view it on GitHub
https://github.com/constabulary/gb/pull/106#issuecomment-102994258.

I can break out the changes to run/runOut if you need.

Oh, don't bother. I'm happy to wait and rebase my changes ontop once cgo is merged. (yeeey! :smile:)

Aaah, libgcc() actually uses runOut() to output the generated content.. I'll adjust it accordingly.

Okay, the new custom error is now returned from run() and runOut() stays unchanged (apart from the Debugf() change I commented on above).

If you are okay with this change, I'd squash them to spare the history from this back and forth.
One thing still irks me, some output is still tunneled to os.Stderr directly. Some tool might leave the interesting information there but this is only hypothetical.

Do you want to continue with this PR ?

Do you want to continue with this PR ?

Yes, please! Did I miss something?

Wercker says no at the moment, do you want to merge master?
On Wed, 20 May 2015 17:48 Henry<EMAIL_ADDRESS>wrote:

Do you want to continue with this PR ?
Yes, please! Did I miss something?
—
Reply to this email directly or view it on GitHub
https://github.com/constabulary/gb/pull/106#issuecomment-103799146.

Okay - rebased on master and made one commit out of it.

I'm not sure this really adds much. This PR looses the command being executed, it just records the args. Does what exists today not do what you need ?

My problem is that when other tools use gb's code directly (not shelling out to the gb tool, like os/exec) their error output are lost to stdout and can't be reused.
In gin, I just wanted to output the errors to the browser as well. Other tools might want to process the source locations of the errors. The latter would still be possible through shelling out but If you already use gb as a package to find the project root and parse the packages in it, it feels weird to go back to os/exec for building a package.

This PR looses the command being execute

Whops.. Yes, sorry about that. It now does capture the command, too.

Thanks for sticking with me on this.

Thanks for pushing me to a better optimum.


instrallation issue

The SageMath team had an issue installing through pip install --user --dev pods
Installing collected packages: pods
Running setup.py install for pods
/projects/22c1772e-0ee1-418a-bb3d-26d649985c08/.local/lib/python3.4/site-packages/setuptools/dist.py:364: UserWarning: Normalizing 'v0.0.2-alpha' to '0.0.2a0'
normalized_version,
File "/projects/22c1772e-0ee1-418a-bb3d-26d649985c08/.local/lib/python3.4/site-packages/pods/test.py", line 3
def test_datasets:
^
SyntaxError: invalid syntax
but the package does end up in .local so we still managed to use it.

There was a typo above, the command was
pip3 install --user --pre pods
but the error message is correct

Thanks Magnus, @zhenwend, could you take a quick look at this?
On Wed, Nov 30, 2016 at 5:29 PM Magnus Rattray<EMAIL_ADDRESS>wrote:

There was a typo above, the command was
pip3 install --user --pre pods
but the error message is correct
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/sods/ods/issues/11#issuecomment-263938126, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAIKWixm45uYwLZUOGepaRjut_rclA7zks5rDbJngaJpZM4LAXur
.


[sgx-externalities]  compiles to std + move integritee-workers tests to here.

And execute them with cargo test in integration tests.

Which tests would you move here and why?

There are tests that check athe encode decode for externalities types. They are, in fact, failing since my upgrade here.

#30 solves the first part, of compiling externalities to std.

Closed, as sgx-runtime and externalities crates are moved back to the worker.


Add link to user community on Google+

Description:
Adds a link to the new AlexaPi Users community on G+.

@renekliment Done

Merging.

Thanks for merging.


Introduce wlr_output_layer

This series add a new wlr_output_layer API, an implementation of this API in the Wayland backend and an example compositor.
My plan is to make glider switch to this API to demonstrate how it can be implemented under DRM.
Introduce wlr_output_layer
This new API allows compositors to display buffers without needing to
perform rendering operations. This API can be implemented on Wayland
using subsurfaces and on DRM using KMS planes.
The goal is to make use of this API in a future scene-graph API.
backend/wayland: implement the output layer API
The output layer API is implemented using subsurfaces. I chose to
implement this API in the Wayland backend before doing so in the DRM
backend, because it's way easier on Wayland. On DRM, one needs to figure
out how buffers can be mapped to KMS planes (libliftoff can help) and
perform atomic test-only commits (our current DRM backend isn't ready
for this).
examples: add output-layers example
This new example demonstrates how to use the wlr_output_layer API. It's
a compositor that displays all client surfaces using wlr_output_layer.
To test, one can for instance run:
build/examples/output-layers -s 'weston-simple-dmabuf-egl & weston-simple-egl'

Under the Wayland backend (where layers work as long as clients use
DMA-BUFs), thsi should display two surfaces.

It seems like this is designed so that wlr_output_layer is entirely siloed and responsible for its own state. This is going to lead to copying down the line. Why not make it an interface that each backend can implement? DRM with libliftoff, for example, which looks very similar. DRY up the sources of truth. Thoughts?

It seems like this is designed so that wlr_output_layer is entirely siloed and responsible for its own state. This is going to lead to copying down the line. Why not make it an interface that each backend can implement? DRM with libliftoff, for example, which looks very similar. DRY up the sources of truth. Thoughts?

I agree state duplication is not great, but there are several issues with this approach:

A need a guarantee that nothing happens before wlr_output_commit. In particular, it should always be possible to cancel all pending changes with wlr_output_rollback (in case the commit fails, for instance). Rolling back is important because wlr_output_commit doesn't explain why a commit has failed, so the compositor wouldn't know what to do after a failed commit to perform a new non-failing commit. Neither Wayland nor libliftoff allow to rollback.
Ordering between layers needs to be tracked. On the DRM backend, the zpos property will be used. The backends code will be much simpler if the common code takes care of the wl_list stuff (backends can just iterate over the list of layers to update their state when the ordering changes). Once the ordering is tracked, it's very little work to keep track of other things like position or buffer.
The Wayland backend would need to track everything the common code currently tracks. One thing I think is important is making sure compositors can repeatedly change the pending state without any cost. It's pretty handy to be able to set the position of a layer at each frame, or to set the buffer to NULL then non-NULL in case importing it fails.

Let me know if any of the above is unclear or needs more explanation.

Thinking about this again, this plan is probably not a good idea. Output layers are not like the rest of the output state.
I think a better approach is not to make output layers part of the output state. This means wlr_output_rollback would not affect layers.

I think a better approach is not to make output layers part of the output state.

Hmm, this is likely a bad idea in the end. We still want output layer changes to be applied on wlr_output_commit to ensure atomicity. If we start introducing some special cases ("a commit applies pending output state, plus output layers" and "a rollback reverts pending state, except output layers state"), then the API will become hairy pretty quickly.

wlroots has migrated to gitlab.freedesktop.org. This pull request has been moved to:
https://gitlab.freedesktop.org/wlroots/wlroots/-/merge_requests/1985


[luafilesystem][lpeg] Bump versions

Describe the pull request
Bump versions of luafilesystem and lpeg to the latest available

Pushed the requested changes. Thanks @LilyWangL

I've removed the windows only gates. That was left over from when vcpkg only supported windows, nothing about these packages should be Windows only.