How to add a new (sub) section (e.g. build: noarch: python)

my_recipe["build"]["noarch"]

gives a message about needing to try to create the section first.

KeyError('Index noarch does not exist. Try to create it first.')

Is this the intended method for "creation"?
my_recipe["build"]["noarch"] = []

The following produced an error:
my_recipe["build"].add_section("noarch")

my_recipe["build"]["noarch"] = []
my_recipe["build"]["noarch"].append(

meta.yaml
build:
  number: 0
  script: {{ PYTHON }} -m pip install . -vv

  noarch: # new
    - python # new

(base) PS C:\Users\sterg\Documents\GitHub\sparks-baird\mat_discover\scratch>cconda build .
No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.16
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.16
Traceback (most recent call last):
  File "C:\Users\sterg\Miniconda3\Scripts\conda-build-script.py", line 10, in <module>
    sys.exit(main())
  File "C:\Users\sterg\Miniconda3\lib\site-packages\conda_build\cli\main_build.py", line 481, in main
    execute(sys.argv[1:])
  File "C:\Users\sterg\Miniconda3\lib\site-packages\conda_build\cli\main_build.py", line 470, in execute
    outputs = api.build(args.recipe, post=args.post, test_run_post=args.test_run_post,
  File "C:\Users\sterg\Miniconda3\lib\site-packages\conda_build\api.py", line 186, in build
    return build_tree(
  File "C:\Users\sterg\Miniconda3\lib\site-packages\conda_build\build.py", line 3064, in build_tree
    metadata_tuples = render_recipe(recipe, config=cfg, variants=variants,
  File "C:\Users\sterg\Miniconda3\lib\site-packages\conda_build\render.py", line 829, in render_recipe
    m = MetaData(recipe_dir, config=config)
  File "C:\Users\sterg\Miniconda3\lib\site-packages\conda_build\metadata.py", line 927, in __init__
    self.parse_again(permit_undefined_jinja=True, allow_no_other_outputs=True)
  File "C:\Users\sterg\Miniconda3\lib\site-packages\conda_build\metadata.py", line 1002, in parse_again
    self.meta = parse(self._get_contents(permit_undefined_jinja,
  File "C:\Users\sterg\Miniconda3\lib\site-packages\conda_build\metadata.py", line 363, in parse
    ensure_valid_noarch_value(res)
  File "C:\Users\sterg\Miniconda3\lib\site-packages\conda_build\metadata.py", line 276, in ensure_valid_noarch_value
    raise exceptions.CondaBuildException("Invalid value for noarch: %s" % build_noarch)
conda_build.exceptions.CondaBuildException: Invalid value for noarch: ['python']

I tried playing around with my_recipe["build"].add_section() as well.

I think I got it:
my_recipe["build"]["noarch"] = "python"

build:
  number: 0
  script: {{ PYTHON }} -m pip install . -vv

  noarch: python

yeah, noarch should be just one string
my_recipe["build"].add_section("noarch")

should work, I will investigate the reason it is not functioning, thanks for reporting it!

Fixed in:

https://github.com/marcelotrevisani/souschef/pull/31

I will generate a new version with those fixes and new functionalities.
Thank you very much for your input!

The version 0.2.0 is already on pypi

https://pypi.org/project/conda-souschef/

I will upload it for conda-forge as well, it should appear in the conda-forge channel in one hour or so

Awesome!! Thanks for the super-quick responses and updates! I've really appreciated the help with this. I now have some automated workflows going from flit to grayskull to conda build using conda-souschef.

I really appreciate your feedback, thanks a lot! If you have new suggestions or bug reports please feel free to open them :)
Pretty nice your workflow, I would love to take a look at it if possible


More exports

Right now, gimli::parser::Pointer is not exported. Because of that, it's actually not possible to get/use personality functions and lsda pointers.
I also needed gimli::parser::parse_encoded_pointer to parse the .eh_frame_hdr for my unwinder, but perhaps that code should just live inside gimli?

I'm working (in what little time I have) on a stack walker (not focused on actually unwinding the stack until walking it works well) as well: https://github.com/fitzgen/pancakes
Perhaps we should work together?

IIRC, .eh_frame_hdr isn't super reliable, but perhaps compilers have improved or I misremembered. In either case, it certainly seems like .eh_frame_hdr parsing belongs in this crate.

Perhaps we should work together?

Sounds good!
What I have at this point (https://github.com/main--/unwind-rs) can find .eh_frame_hdr at runtime, parse unwinding info from there and finally walk and unwind the stack (properly with personality functions etc). This works well for both debug and release builds for standard Rust panics (I'm exporting all relevant libunwind entry points and the linker prefers my versions over the genuine dynamically linked ones), but the implementation is still kind of a prototype and not portable at all (yet?). Right now, I'm mainly working on performance (binary search on the .eh_frame_hdr table works already but it's not committed yet) and actually building an clean and usable API.

@main-- I factored out the dl_iterate_phdr bits into the https://github.com/fitzgen/findshlibs crate.
My original goal was to allow iterating over sections within each mapped segment, but that isn't possible with ELF/dl_iterate_phdr, just mach-o and dyld APIs.
So I think instead it makes sense to build a tiny crate on top of findshlibs that is less general than "give me all the sections in this mapped segment" and is instead just "give me all the eh_frame (and eh_frame_hdr) sections mapped into memory" which is possible to implement for ELF with PT_GNU_EH_FRAME.
When performing actual unwinding, or inspecting the stack from a debugger, we need all the registers. When quickly capturing a stack trace for a profiler, we generally only need a couple of the registers.
When unwinding the stack for a panic, we're in the same process as the stack that is being unwound. When implementing a debugger or profiler, it is likely that we are in a different process from the stack being walked, and are presumably using something like https://crates.io/crates/read-process-memory
The stack walking / unwinding should be generic / customizable enough to handle all these different scenarios.
Also, I strongly typed different kinds of pointers that we encounter, before or after applying the loaders offset, etc: https://github.com/fitzgen/findshlibs/blob/master/src/lib.rs#L51-L72 I find this very helpful for wrapping my head around what kind of pointer I'm dealing with at what times. It might make sense to pull these definitions out into their own crate and use them in gimli too.
Additionally, I have a TaggedWord type that tracks which registers we've successfully recovered at what point during unwinding. I notice that there were some unwrap_or(0)s inside unwind-rs, and I think TaggedWord (or even just Option<usize> or something) is a slightly better approach. https://github.com/fitzgen/pancakes/blob/master/src/tagged_word.rs#L9
So that's kind of where I wanted to go / what my plans were. All that said, pancakes doesn't actually work yet :-p But it does have traits and infrastructure that was tailored with these different scenarios in mind.
Oh, one final question: why write assembly rather than use getcontext? Ultimately easier and more portable?
So where do you want to go from here?

"give me all the eh_frame (and eh_frame_hdr) sections mapped into memory"

AFAIK .eh_frame_hdr is the only way to find .eh_frame since dl only gives you ELF segments, not sections.
I don't actually have a clear understanding of how you're supposed to do this part properly, especially since gimli's BaseAddresses wants text+data+cfi base addrs. What I'm doing right now is: For EhFrameHdr, set CFI to .eh_frame_hdr. This somehow yields a correct eh_frame_ptr so who am I to complain. But before we declare victory, note that there is no length. Of course I can just make it infinitely large and call it a day but there has to be a better way to do this.
Now for all the EhFrame stuff, the CFI base is suddenly .eh_frame. Makes sense, but I still spent like 4 hours wondering why my unwinder is setting registers to incorrect values.
I can even use the same BaseAddresses to decode the .eh_frame_hdr table, except those pointers are tagged as data-relative where "data" really means .eh_frame_hdr.
Long story short: I hate this and I'm really surprised that it works.

When quickly capturing a stack trace for a profiler, we generally only need a couple of the registers.

What exactly do you mean? Since the encoding allows arbitrary register/frame layout the only optimization I can think of is evaluating registers only lazily, but I kindof doubt that that would make a significant difference.

Also, I strongly typed different kinds of pointers that we encounter, before or after applying the loaders offset, etc

Stronger types are always good, the only reason unwind-rs is lacking in that respect right now is that it's still a prototype and not a polished solution.

I notice that there were some unwrap_or(0)s inside unwind-rs, and I think TaggedWord (or even just Option<usize> or something) is a slightly better approach.

We have the same idea here as that's exactly what I'm doing! I assume you're talking about this function which is the lander glue that jumps to landing pads. I'm just setting all registers because I think that's the cleanest solution, zeroing out the undefined ones.

why write assembly rather than use getcontext? Ultimately easier and more portable?

Maybe? I didn't know about this API, but I also didn't search for anything like that as doing that part in plain asm was just easier (I don't have to worry about what the library function does). One obstacle that comes to my mind right now is how to ensure that everything ... well ... works? What I mean is that any function which returns context data is very hard - if not impossible - to use correctly since that means I'll be walking my own stack frame that I'm changing right now. This is especially tricky when you think about compiler optimizations (most notably inlining).
I couldn't really wrap my head around this so I implemented the one reliable mechanism I came up with: Save the calling state and call the unwinder so the frame where it starts unwinding is frozen.

It seems like we have two puzzle pieces here: I have a working stacktracer and unwinder while you have some nice abstractions. We could merge if you want to. Obviously the final dream is for unwind-rs to become a fast and portable unwinder and rule the world :laughing: .  An added bonus could be the project I'm working on right now (why I actually created unwind-rs) where I print a stack trace along with all live variables and their values.

AFAIK .eh_frame_hdr is the only way to find .eh_frame since dl only gives you ELF segments, not sections.

Yes, that is what I was getting at :) I ultimately realized that the API I originally imagined was only possible to implement on OSX / Mach-O, and not on Linux / ELF.

Long story short: I hate this and I'm really surprised that it works.

Yes. And it is all pretty terribly documented in many far out, disparate locations :(
I wonder if inverting how we handle relative pointers makes sense... Return a symbolic form of the pointer, eg pc + 0x1234, rather than pass base addresses in.
Not sure that would actually help at all.

What exactly do you mean? Since the encoding allows arbitrary register/frame layout the only optimization I can think of is evaluating registers only lazily, but I kindof doubt that that would make a significant difference.

In practice,

almost all register rules are the simple ones like CFA + offset, and don't need to evaluate DWARF expressions,

finding the CFA is almost always register + offset, and

the registers referenced in the above rules are one of rbp, rip, and rsp.

A fast path that only handles the above case is a boon for a stack sampling profiler. libunwind has such a path. Firefox's builtin profiler pretty much only supports this path, and gets away with it pretty well.
Implementation wise, I think it is just a different register set struct, that only has the common subset of registers and a cut down version of the unwind routine.

I couldn't really wrap my head around this so I implemented the one reliable mechanism I came up with: Save the calling state and call the unwinder so the frame where it starts unwinding is frozen.

Yeah, I have a similar-ish approach that involves calling a closure + some lifetimes to ensure that you can't return and stomp over everything. https://github.com/fitzgen/pancakes/blob/master/src/x86_64/registers.rs#L108

I have a working stacktracer and unwinder while you have some nice abstractions. We could merge if you want to.

Sounds good to me :)

Obviously the final dream is for unwind-rs to become a fast and portable unwinder and rule the world 😆

Indeed :)

So I guess we are left with two immediate questions:

Which name do you like better, pancakes or unwind-rs? :-P

Where should the repo live? Move it into gimli-rs? New organization? I think it would fit with the gimli-rs org, if @philipc @jonhoo @tromey don't object to the idea.

Some discussion of other people potentially interested in helping out: https://internals.rust-lang.org/t/is-there-any-pure-rust-solution-for-libgcc-s-so-in-libunwind/5998/10


ESP32-S2 idf building in eclipse with submodules (IDFGH-4570)

Environment

Development Kit:      [ESP32-S2-WROOM]
Module or chip used:  [ESP32-S2-WROOM]
IDF version (run git describe --tags to find it):  v4.3-dev-2399-g4206a0422
Build System:        Official eclipse with plugin ESP-IDF Eclipse Plugin (Version: <IP_ADDRESS>010091505)
Compiler version (run xtensa-esp32-elf-gcc --version to find it): (esp-2020r3 8.4.0?)
Operating System:     [Windows 10 Pro]
(Windows only) environment type: [eclipse built].
Using an IDE?: [Yes] Eclipse IDE for C/C++ Developers (includes Incubating components)

Version: 2020-09 (4.17.0)
Build id: 20200910-1200

Power Supply:         [problem with building idf...]

Problem Description
Hello I've got a big problem with my eclipse IDE and esp-idf (latest version, yesterday fetching).
I cannot build program with Eclipse (this program works my colleague with Linux and vscode installed - I preffer eclipse because some months ago I successfully run DEBUGGER with this board)
First of all: I need USB component program so I need to update idf to latest. I done it but I have strange logs in "git status" command:
$ git status
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
  (use "git push" to publish your local commits)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
  (commit or discard the untracked or modified content in submodules)
        modified:   components/asio/asio (modified content)
        modified:   components/bootloader/subproject/components/micro-ecc/micro-ecc (modified content)
        modified:   components/bt/controller/lib (modified content)
        modified:   components/bt/host/nimble/nimble (modified content)
        modified:   components/cbor/tinycbor (modified content)
        modified:   components/coap/libcoap (modified content)
        modified:   components/esp_wifi/lib (modified content)
        modified:   components/esptool_py/esptool (modified content)
        modified:   components/expat/expat (modified content)
        modified:   components/json/cJSON (modified content)
        modified:   components/libsodium/libsodium (modified content)
        modified:   components/lwip/lwip (modified content)
        modified:   components/mbedtls/mbedtls (modified content)
        modified:   components/mqtt/esp-mqtt (modified content)
        modified:   components/nghttp/nghttp2 (modified content)
        modified:   components/protobuf-c/protobuf-c (modified content)
        modified:   components/tinyusb/tinyusb (modified content)
        modified:   components/unity/unity (modified content)
        modified:   examples/build_system/cmake/import_lib/main/lib/tinyxml2 (modified content)
        modified:   examples/peripherals/secure_element/atecc608_ecdsa/components/esp-cryptoauthlib (mo

no changes added to commit (use "git add" and/or "git commit -a")

and
git  submodule update --init --recursive
do nothing...
Second when I build my project (I click on build icon in eclipse IDE) everything is compiled (with some warnings - normal) but building stops when linker starts work:
OUTPUT with error:
[2/3] cmd.exe /C "cd . && D:\Programy\Eclipse_CPP\.espressif\tools\xtensa-esp32s2-elf\esp-2020r3-8.4.0\xtensa-esp32s2-elf\bin\xtensa-esp32s2-elf-g++.exe  -mlongcalls   @CMakeFiles\eink-esp-proj.elf.rsp  -o eink-esp-proj.elf  && cd ."
FAILED: eink-esp-proj.elf
cmd.exe /C "cd . && D:\Programy\Eclipse_CPP\.espressif\tools\xtensa-esp32s2-elf\esp-2020r3-8.4.0\xtensa-esp32s2-elf\bin\xtensa-esp32s2-elf-g++.exe  -mlongcalls   @CMakeFiles\eink-esp-proj.elf.rsp  -o eink-esp-proj.elf  && cd ."
d:/programy/eclipse_cpp/.espressif/tools/xtensa-esp32s2-elf/esp-2020r3-8.4.0/xtensa-esp32s2-elf/bin/../lib/gcc/xtensa-esp32s2-elf/8.4.0/../../../../xtensa-esp32s2-elf/bin/ld.exe: esp-idf/esp_wifi/libesp_wifi.a(phy_init.c.obj):(.literal.esp_phy_load_cal_and_init+0x10): undefined reference to `get_phy_version_str'
d:/programy/eclipse_cpp/.espressif/tools/xtensa-esp32s2-elf/esp-2020r3-8.4.0/xtensa-esp32s2-elf/bin/../lib/gcc/xtensa-esp32s2-elf/8.4.0/../../../../xtensa-esp32s2-elf/bin/ld.exe: esp-idf/esp_wifi/libesp_wifi.a(phy_init.c.obj): in function `esp_phy_load_cal_and_init':
D:/Programy/esp-idf-v4.2-rc/components/esp_wifi/src/phy_init.c:474: undefined reference to `get_phy_version_str'
d:/programy/eclipse_cpp/.espressif/tools/xtensa-esp32s2-elf/esp-2020r3-8.4.0/xtensa-esp32s2-elf/bin/../lib/gcc/xtensa-esp32s2-elf/8.4.0/../../../../xtensa-esp32s2-elf/bin/ld.exe: esp-idf/wpa_supplicant/libwpa_supplicant.a(wpa.c.obj):(.literal.wpa_supplicant_install_ptk+0x4): undefined reference to `esp_wifi_get_sta_hw_key_idx_internal'
d:/programy/eclipse_cpp/.espressif/tools/xtensa-esp32s2-elf/esp-2020r3-8.4.0/xtensa-esp32s2-elf/bin/../lib/gcc/xtensa-esp32s2-elf/8.4.0/../../../../xtensa-esp32s2-elf/bin/ld.exe: esp-idf/wpa_supplicant/libwpa_supplicant.a(wpa.c.obj): in function `wpa_supplicant_install_ptk':
D:/Programy/esp-idf-v4.2-rc/components/wpa_supplicant/src/rsn_supp/wpa.c:688: undefined reference to `esp_wifi_get_sta_hw_key_idx_internal'
d:/programy/eclipse_cpp/.espressif/tools/xtensa-esp32s2-elf/esp-2020r3-8.4.0/xtensa-esp32s2-elf/bin/../lib/gcc/xtensa-esp32s2-elf/8.4.0/../../../../xtensa-esp32s2-elf/bin/ld.exe: esp-idf/wpa_supplicant/libwpa_supplicant.a(wpa.c.obj): in function `wpa_supplicant_install_gtk':
D:/Programy/esp-idf-v4.2-rc/components/wpa_supplicant/src/rsn_supp/wpa.c:824: undefined reference to `esp_wifi_get_sta_hw_key_idx_internal'
collect2.exe: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

Expected Behavior
I need to build project in eclipse...
Actual Behavior
Building stops on linking...
Steps to reproduce
as mention above... (Install eclipse, install plugin, update idf to latest, do a hello world, build everything together...
Code to reproduce this issue
any Hello world example in eclipse...
Please help me because this is very important project in my current job...
Greetings

I tried linux and the same problem. do you have any ideas?

I tried linux and the same problem. do you have any ideas?

Hi,
got the error /components/esp_wifi/src/phy_init.c:474: undefined reference to get_phy_version_str'as well. I just re-rungit submodule update --init --recursive and did afullclean`. This fixed it for me at least.

Hi,
got the error /components/esp_wifi/src/phy_init.c:474: undefined reference to get_phy_version_str'as well. I just re-rungit submodule update --init --recursive and did afullclean`. This fixed it for me at least.

Hi @wojciech-angelo ,
Sorry for the slow response. It looks like your git directory has gotten into a bad state somehow. What method did you use to update it?
Cloning ESP-IDF from github again may be the quickest way to get a working copy of the master branch.

Hi @wojciech-angelo ,
Sorry for the slow response. It looks like your git directory has gotten into a bad state somehow. What method did you use to update it?
Cloning ESP-IDF from github again may be the quickest way to get a working copy of the master branch.

Hi @wojciech-angelo ,
Sorry for the slow response. It looks like your git directory has gotten into a bad state somehow. What method did you use to update it?
Cloning ESP-IDF from github again may be the quickest way to get a working copy of the master branch.

Ok, I handle this...
thanks...

Hi @wojciech-angelo ,
Sorry for the slow response. It looks like your git directory has gotten into a bad state somehow. What method did you use to update it?
Cloning ESP-IDF from github again may be the quickest way to get a working copy of the master branch.

Ok, I handle this...
thanks...

@wojciech-angelo Thanks for letting me know. Is the issue resolved now?

got the error /components/esp_wifi/src/phy_init.c:474: undefined reference to get_phy_version_str'as well. I just re-rungit submodule update --init --recursive and did afullclean`. This fixed it for me at least.

Same error here after pulling from release/v4.2. Running git submodule update --init --recursive and a fullclean resolves the issue.

This happens if you do a git pull without updating submodules or if updating submodules fails due to a network timeout, etc.
If git status shows any changes in submodule folders, then git submodule update --init --recursive  should resolve it.

@Windoofs Thanks for reporting, would you please help share if any updates for this issue? Thanks.

The error has not occurred since then.

Closing as it seems like this is resolved.


Fix copy-webpack settings

Fixes copy-webpack-plugin settings to find the html & js files correctly on Windows. Uses the documentation found here: https://www.npmjs.com/package/copy-webpack-plugin

fix for #34

Thank you!


Fix textarea and validation on destination page

The destination configuration page did not previously support a textarea input as was supported on most other pages. The validation of string inputs was not being performed. This commit addresses both of those issues and corrects the validation expressions for the AWS and S3 destination plugins so that they continue to function. The SFTP destination plugin does not have any string validation. The Kubernetes plugin does not work at all as far as I can tell; there will be another PR in the coming days to address that.

Coverage remained the same at 60.913% when pulling 437d918cf795516142d5f7c6f0f385b2d4716bfa on wfhartford:destination-tpl-fix into afc7512914c2cedb987fa1219536c838f6a5aded on Netflix:master.


Multi-root support

Disclaimer: I used to work on single projects so far and haven't checked on multi-root support lately.
I have two git repos in subdirectories.
IIRC, when opening an R file and hitting the r.loadall shortcut, usually the R session was attached with whe root of PROJECT1 as the working directory instead of the workspace root.
Right now the session is always started in the workspace root and subsequently r.loadall fail because no DESCRIPTIOIN file is found.
Did something change lately or was my memory wrong in this case?

Basically, you are suggesting that the "R: Create R terminal" command should use the workspace folder of the active document as the working directory to start an R session if there's no active session. Does this make sense to you?

If we plan to provide better support for multi-root workspace, it seems that we could associate each R session created by "R: Create R terminal" with a workspace folder? When the user runs any command that needs to send text to the terminal, it should send to the terminal associated with the workspace folder where the current active document is contained.
This requires that we keep track of multiple terminals.

Basically, you are suggesting that the "R: Create R terminal" command should use the workspace folder of the active document as the working directory to start an R session if there's no active session. Does this make sense to you?

Yeah - if the workspace folder is the package root in this case.
This could be found out by searching for a description file upwards.

If none is found, the uppermost workspace root is used.
If one is found, this directory is used.

For example when I open WORKSPACE_ROOT/project1/R/file.R, the working directory in R should become WORKSPACE_ROOT/project1.
It then would be awesome if all R calls made in one of these R functions would be send to the active R session of this project (which is usually a package).
If I then open another file in WORKSPACE_ROOT/project2/R/file.R and hit r.loadall, it should start a new R session (if none exists yet for this project) and follow the same logic as described for project1.
All of this is an R package centric workflow. Users working on non-R packages may want to use one R session with scripts from different projects and use only one R session with all R commands be sent to this one.
Hence an option for this behavior might probably be needed.

This requires that we keep track of multiple terminals.

Yeah - probably a complex one. Not sure if any editor is currently capable of this.
Having this would be a blast 🚀

To be clear, are you using a mutli-root workspace or just a single-root workspace that contains two R packages?

In my example its a single-root workspace with two R packages - so the title might actually be wrong? Feel free to update and clarify if so  👍


[Bug]: requested access to the resource is denied

zot version
v2.0.0-rc5
Describe the bug
requested access to the resource is denied with not default docker repo
example I have docker repo https://docker.io/test
when I trying to pull from localhost:5000/test/hybrid-adm:hybrid it give error in pulling images.
To reproduce

Configuration
          "sync": {          "credentialsFile": "/secret/dockercfg.json",          "enable": true,          "registries": [          {            "urls": [              "https://docker.io/test"            ],            "content": [            {              "destination": "/test",              "prefix": "**"            }            ],            "onDemand": true,            "tlsVerify": true          }          ]        }
Client tool used
Seen error
{"level":"error","error":"reading manifest hybrid in docker.io/library/hybrid-adm: requested access to the resource is denied","repo":"hybrid-adm","reference":"hybrid","goroutine":77,"caller":"zotregistry.io/zot/pkg/extensions/sync/service.go:361","time":"2023-07-02T17:35:15.044407894Z","message":"couldn't get upstream image manifest details"} {"level":"error","error":"reading manifest hybrid in docker.io/library/hybrid-adm: requested access to the resource is denied","repository":"test/hybrid-adm","reference":"hybrid","goroutine":75,"caller":"zotregistry.io/zot/pkg/api/routes.go:1727","time":"2023-07-02T17:35:15.044625869Z","message":"error encounter while syncing image"}

Expected behavior
No response
Screenshots
No response
Additional context
No response

Hello @BorisKofman,
For syncing private docker registries the docker registry URL should be: https://docker.io
then pull localhost:5000/test/hybrid-adm:hybrid should work, if credentials are also correctly set in /secret/dockercfg.json
    "docker.io": {
        "username": <user>,
        "password": <password>
    }

Thank you for your feedback!

Hey Thank you
did the changes from:
    "docker.io/test": {
        "username": <user>,
        "password": <password>
    }

to:
    "docker.io": {
        "username": <user>,
        "password": <password>
    }

but it still same issue.

@BorisKofman did you also changed the url from https://docker.io/test to https://docker.io ?

If test is part of the repository name on docker, you don't need to add it under in the destination key.
That configuration is for creating a new folder on zot which si not present in the upstream. For example you want to place all images pulled from docker under docker/. So in this case it would end up under localhost:5000/test/test/hybrid-adm:hybrid. Notice the double test, one coming from original path on docker, one added by zot.
Can you try removing "destination": "/test" from zot configuration pull again?

@andaaron tested not working error: ImagePullBackOff

@BorisKofman just checking that you don't have any network access controls that can prevent downloads?
@peusebiu we should be able to test this locally, did something change upstream on dockerhub?

Yeah tested this locally and with his config I get the same exact error, after modifying the registry URL in both sync config and sync credentials, it's working.
The denied error comes from the fact that docker.io/library/hybrid-adm is not a public repo or doesn't exist.
my dockerhub user is peusebiu and I have a zot under repositories.
here is my zot config:
{
    "distspecversion":"1.1.0-dev",
    "storage": {
        "rootDirectory": "/tmp/zot_to_sync",
        "dedupe": false,
        "gc": false
    },
    "http": {
        "address": "<IP_ADDRESS>",
        "port": "8081"
    },
    "log": {
        "level": "debug"
    },
    "extensions": {
        "sync": {
            "credentialsFile": "/home/peusebiu/sync-auth-filepath.json",
            "registries": [
                {
			"urls": [
				"https://docker.io"
			],
			"onDemand": true,
			"tlsVerify": true,
                        "content": [
                            {
                                 "destination": "/peusebiu",
                                 "prefix": "**"
                           }
                    ]
                }
            ]
        }
    }
}

here are my sync credentials:
{
    "<IP_ADDRESS>:8080": {
        "username": "user",
        "password": "pass"
    },
    "docker.io": {
        "username": "peusebiu",
        "password": "password"
    }
}

This is how I queried, like @andaaron said I need to double "peusebiu" because of destination.
peusebiu@PEUSEBIU-FC130:~$ skopeo copy docker://<IP_ADDRESS>:8081/peusebiu/peusebiu/zot:tls5 oci:oci:oci --src-tls-verify=false
Getting image source signatures
Copying blob d0f82576f946 done
Copying blob a91c1ed460d2 skipped: already exists
Copying blob 98b9e799d13b done
Copying config 822bb49247 done
Writing manifest to image destination
Storing signatures
peusebiu@PEUSEBIU-FC130:~$

@BorisKofman can you give us your latest logs with the latest errors?

Wait so the image we want to sync is docker.io/test/library/hybrid-adm or docker.io/library/hybrid-adm?

@andaaron I have private repo so I am trying to sync docker.io/test/library/hybrid-adm
@peusebiu When I remove /test
Nginx:
nginx.ingress.kubernetes.io/rewrite-target=/v2/$2
I got error:
{"level":"error","error":"reading manifest hybrid in docker.io/library/hybrid-adm: requested access to the resource is denied","image":"docker.io/library/hybrid-adm:hybrid","goroutine":28,"caller":"zotregistry.io/zot/pkg/extensions/sync/utils.go:655","time":"2023-07-04T10:04:39.141247939Z","message":"couldn't get upstream image manifest details"}

@BorisKofman please give me the output of docker inspect

Assuming
nginx config:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-oci-ingress-v2
  namespace: test
  labels: k8slens-edit-resource-version: v1
  annotations:
     nginx.ingress.kubernetes.io/rewrite-target: /v2/test/$2
spec:
   ingressClassName: nginxv2
   rules:
   - host: test
     http:
       paths:
         - path: /v2(/|$)(.*)
           pathType: ImplementationSpecific
           backend:
              service:
                 name: oci-middleware
                 port:
                    number: 5000

sync config:
    "sync": {
      "enable": true,
      "credentialsFile": "/secret/dockercfg.json",
      "registries": [
        {
          "urls": ["https://docker.io"],
          "content": [
            {
              "prefix": "**",
              "destination": "/test"
            }
          ],
          "onDemand": true,
          "tlsVerify": true
        }
      ]
    }

So let me check if I get this right:

the client makes calls to zot:5000/hybrid-adm:hybrid
before the call reaches zot nginx configured with nginx.ingress.kubernetes.io/rewrite-target=/v2/test/$2 modifies the url to add test, resulting in zot:5000/test/hybrid-adm:hybrid
the call reaches zot, which is configured to use "destination": "/test", so zot requests the image docker.io/library/hybrid-adm in order to copy it under destination /test
and all of the above is working per your last message - and it makes sense to be working.

Now, assuming the above scenario is working, the ask is not clear to me, so.
A. You are confirming this is what you intend to configure and the issue is resolved.
OR
B. You want the client to have to use  zot:5000/test/hybrid-adm:hybrid to access the image originally from  docker.io/library/hybrid-adm. In which case you should

remove the nginx configuration entirely, and that should be enough
keep zot configuration the same as posted above

OR
C. You want the client to use  zot:5000/hybrid-adm:hybrid to copy the image originally at docker.io/test/library/hybrid-adm in which case I expect to work if you:

remove the "destination": "/test" configuration from zot, as the image already has test in the path, and that will be used in zot as well
keep nginx config the same as posted above

@andaaron
the client makes calls to zot:5000/test/hybrid-adm:hybrid
To access image in docker.io/test/libarary/hybrid-adm:hybrid
the original configuration for ingress is:
nginx.ingress.kubernetes.io/rewrite-target: /v2/$2
but the image access trying to pull from docker.io/libarary/hybrid-adm:hybrid in zot and got ImagePullBackOff
after debugging it and trying to understand the issue I have found that when I add rewrite-target:  /v2/test/$2 it work but it block from adding another repo example /test2
What I am trying to do:
Create Zot Middleware to pull our images from private docker.io repository diagram example:

RC4 works as expected without rewrite-target: test

For this use case you it looks to me like

You don't need to add the "destination": "/test" in the zot config, as the repo names in zot match entirely the repo name in dockerhub.
You don't need to add any nginx rewrites, for the same reason.

Did you try this out? If it fails what are the zot errors?

@andaaron Thanks Working!


Allow "anonymous" sign ins

Currently the preview requires anyone with an invite link to sign in to  join a collaboration session to ensure that the person starting the collaboration knows who is actually connected to the session. However, this adds an extra step for guests joining the collaboration session and may not be desirable in scenarios like open source projects.
Introducing an anonymous permissions level would resolve this problem where a participant could join the collaboration session without signing in (and then opt to sign in later).  The invite link is non-guessable which increases the overall security of the approach.

This would be extremely useful.  I work with a number of non-professional coders, so many don't have Microsoft or GitHub accounts.  Using Live Share is a great tool to assisting then when they ask questions, but the required account is one more hurdle to using it.

@pltrant What’s the use case you have in mind? Working with designers on HTML/CSS? Collaborating with technical writers on markdown? Etc. This feature is something we’ve heard a lot of interest in, and I just want to make sure we’re accounting for all scenarios this could potentially benefit 👍

I work on a product that has a custom scripting language.  It's simple enough that any employee who wishes to utilize it can, but it can also be complex at times, depending on what's trying to be accomplished.  I wrote a VSCode extension for it and assisted many of our employees in switching to it.  We also routinely have new employees every year and offer a brief training session on learning the scripting language and going forward, learning to utilize it with VSCode and the extension for it.  In such scenarios when these users ask for help or we want to show them something, we want the path to least resistance for jumping in and helping them.  Having to install the Live Share extension itself and reload VSCode for the first session is one big hurdle by itself, but a necessary and understandable step.  But for signing in, most of these users are not actual programmers and don't have such accounts already, so we also have to ask them to perform that step.  It's about reducing the time for setup and actual use.

Merging in #784 from @leidegre
Would it be possible to launch a VS Live Share session without an account? Simply by sending a one time use only link to a colleague or friend over, say Slack?
It would greatly lower the barrier of entry and encourage people to try the feature. I really don't like the idea that I have to sign in to use an extension which otherwise seem like a really nice thing.

It is not possible today to do this, the very purpose of liveshare is for collaboration with different user identities, to remove user identities from the collaboration experience amounts to anonymous access to your source code, and running programs on your machine.

@Priya91 obviously, you would 1.) not send an invite to someone you do not trust and 2.) not accept that they join the session if you do not trust that they are who they say they are.
Identities (that can be verified) can be provided in a multitude of ways, a Microsoft account is not the only way to do that nor is it necessary to have an external identity provider in the mix. You could just as easily set a password to protect the session, if you are really worried about the wrong person getting access.
As a final example, let use ssh. If I my colleague sent me a public SSH key we could use that as his identity. The burden lies on us to make this exchange in a secure manner but that doesn't seem like a big deal to me. Regardless, I'm assuming that I have to accept that my workspace is accessed before it actually is.

This has been addressed in the latest Live Share update 🚀 By default, when a guest tries to join a session, and they aren't already signed-in, they will be given the option to continue as a read-only guest, and then enter a name. The host will get a notification asking if they'd like to accept their join request or not (which is important, since we can't guarantee the guest's identity).
After the host accepts the request, the guest will only be given read-only access to the files/terminals/etc. If the guest wants to elevate to read/write access, they can simply sign-in.
We'd love to hear feedback/questions/thoughts on this experience. Otherwise, thanks again for all the feedback here!

how do you even change the name? I have logged in with my Microsoft Email into VS2019 and Liveshare automatically applied same credentials but when I test it, it displays the name wrong. On my Microsoft Account my name is correct but Live share displays wrong name for me. Is there some cache somewhere or where is the name retrieved from?


Combine yields incompatible tuple types

I was trying to extend your example to also accept #rgb colors, like so:
const rgb = mecha.convert(Rgb, Rgb.from, mecha.combine(.{
    mecha.char('#'),
    mecha.oneOf(.{
        rgbBytes,
        rgbNibbles,
    }),
}));
const rgbBytes = mecha.combine(.{ byte, byte, byte });
const rgbNibbles = mecha.combine(.{ nibble, nibble, nibble });
const byte = mecha.convert(u8, mecha.toInt(u8, 16), mecha.manyRange(2, 2, hex));
const nibble = mecha.convert(u8, mecha.toInt(u8, 16), mecha.manyRange(1, 1, hex));
const hex = mecha.digit(16);

And I'm getting this error when testing: error: expected type '.mecha.Result(.mecha.struct:323:23)', found '.mecha.Result(.mecha.struct:323:23)'
As far as I understand from the code and asking on Discord, combine generates its return type, but rgbBytes and rgbNibbles have distinct tuples even though they are both u8 triplets… (due to @TypeOf)

I worked around it by un-factoring convert from rgb down to rgbBytes and rgbNibbles, so those become Parser(Rgb) and the unique tuple types are immediately consumed.
I'm still wondering if there isn't a better way, but feel free to close.

👍
I can confirm it allows me to factor the conversion to Rgb back up :-)


Add minHoursPlayed, VacBan and Limited check from steam api

Write a Class to retrieve infos from http://steamcommunity.com/profiles/76561198201629499?xml=1
(also groups you are member of) to be used in plugins, or in rocket.

By ApokPT if you didn't notice on XMPP: http://pastebin.com/K7DtvNDs

@fr34kyn01535 for groups use:
http://steamcommunity.com/gid/103582791434248624/memberslistxml/?xml=1
<3 Apok

Done,  to be found in Rocket.Core.Steam


Adding StatsBeat implementation

how does the config look like for statsbeat?
this is for Java:
 "internal": { 	"statsbeat": { 		"instrumentationKey": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx",  		"endpoint": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", 		"intervalSeconds": 30 ,  		"featureIntervalSeconds": 30 	}   },

@heyams for Statsbeat here there is no configuration file available, everything is set as attributes or constants in Statsbeat class


BART: release XSum-tuned model

Hi
Do you plan to release this model?
Thanks

+1

The model has been released


Use Tab to Autocomplete, not Enter

x Is new feature
_ Concerns documentation

Description of suggestion or shortcoming:
Right now, autocomplete uses "enter". This is different from other programs - many of which use "tab".
This was requested by Jim. Also, in my own use, I found this frustrating - sometimes I want to skip the autocomplete, so I press enter, but this actually triggers autocomplete.

VS Code supports both "enter" and "tab", and I assume it's the most popular editor, so this is surprising. Skipping is on "esc", same as in Codemirror.


Fix table header grammar in annotations

Move "prefix" to out of the code backtick.

any updates? Is there anything that I need to do?

LGTM


Ignore all unknown format in v8 (unknownFormats: 'ignore')

What version of Ajv are you using? Does the issue happen if you use the latest version?
8.12.0
Ajv options object

Default
Issue
I'm currently upgrading one my project from using ajv v6 to ajv v8
I would like to keep all functionalities as close as possible as what we have right now
One of them is to ignore all unknown formats from the schema
From the documentation from the v8 I don't see a way to mimic the  unknownFormats: 'ignore' option anymore
I can only see that we can whitelist a specific format or just disable all format validation at once
But since I don't have control about how format are provided on schemas that I will compile, I reallly need a way to ignore unknown format while still being able to validate known ones
Is it something that is possible in v8 ?

You can use strictSchema: false or strictSchema: "log" to ignore unknown formats and keywords: https://ajv.js.org/strict-mode.html#unknown-formats.
Defining all used format as always valid (see the link) is better, as this way you avoid typos.

I agree this is very inconvenient @sl-nigelalmada.
In my case I'm taking schemas as input (I'm effectively using AJV to let users to validate their own data client side) so I have absolutely no way to know which unknown formats may be used in advance! I definitely want to ignore these issues regardless - in many cases schemas come with all sorts of extra metadata & odd formats that are used elsewhere but not relevant here, and I need to be able to do core validation immediately regardless, without that blocking anything or filling up the console logs with warnings.
unknownFormats: 'ignore' was perfect, and it's impossible to use the workaround you describe above @epoberezkin in this scenario.
Anyway, although it seems that nowadays AJV doesn't support this without unknownFormats, fortunately you can work around it with a JS proxy:
const ajv = new Ajv({
    formats: new Proxy({}, {
        // Return 'true' (ignore) for *all* undefined formats:
        get(target: any, name: string) {
            if (name in target) return target[name];
            else return true;
        }
    })
});

This will ignore any unknown formats (with no annoying logging - it will just ignore them) but will still correctly validate any formats that are actually explicitly defined, e.g. with ajv-format.
(I still really wish AJV supported this directly though)


Question: Cookie jar support?

Does this library have any sort of cookie jar support. Lets say I wanted to save multiple jars in my app and use them when I need them.
Is that possible here?

Actually, just yesterday cookie support was added into Needle. This is what you can do.
var needle = require('needle');

var cookies = { foo: 'bar', user: 'someone' };

// by passing the object in the cookies option, Needle will set them in the 'Cookies' request header.
needle.get('foobar.com/path', { cookies: cookies }, function(err, resp) {
  if (!err) console.log(resp.cookies); // will spit out an object with the response cookies (by parsing the set-cookie header)
})

// additionally, you can also ask needle to persist cookies on redirects by setting the follow_set_cookies option to true. this obviously requires the follow_max count to a number higher than one.
needle.get('foobar.com/path', { follow_max: 3, follow_set_cookies: true }, function(err, resp) {
  // if a redirect was followed, needle will parse and set the cookies accordingly
  // the resp.cookies object will be ones received on the last response
})

I hope this helps.

Hmmm, aight, is the cookies POJO updated after the requests are done. Example:
  var cookies = {foo: 'bar'};

  need.get('foobar.com/path', {cookies: cookies}, function (err, resp) {
    // Lets imagine the response headers had a Set-Cookie: new=cookie
    // Will this be the output?
    console.log(cookies); // {foo: 'bar', new: 'cookie'}
  });

Btw, I was looking for a request replacement, because the way they handle cookies is hideous and just plain wrong when you need to access them manually. This library looks pretty straightforward.
P.S. Are you by any chance from Chile? I think I've atended a talk you did.

Yup, from Chile like the porotos.
The cookie object you'll receive as part of the response object are just the response cookies (those contained in the set-cookie header). So in that case you should only get { new: 'cookie' } in resp.cookies.

Sure, I meant if the cookies object passed was updated in the process but I believe it doesn't. Btw,
here's a little bug I fixed for you #102
Need any further help with the library aperro cause I think I might stick to it ;)


bug: collapsible header resets scroll position when triggered by iOS overscroll

Bug Report
Ionic version:
[x] 4.x
Current behavior:
When using a collapsible header with a large title on a page that's not high enough to scroll only the iOS overscroll functionality takes effect. When the page is scrolled until the point where the opacity of the main and the scroll header is toggled the scroll position is reset which causes a quit ugly and janky behaviour.

Expected behavior:
It works the same way no matter if its normal scrolling or overscrolling.
Steps to reproduce:
Place a collapsible header in an otherwise empty ion-content, run it on an iOS device and try to scroll the page until the point where the scroll header should be hidden and the main header should be visible.
Related code:
If I remove all function calls where the scroll header is activated/deactivated:
https://github.com/ionic-team/ionic/blob/0a7aae28a7eb0270cdcd100933c01850403b66db/core/src/components/header/header.utils.ts#L104
https://github.com/ionic-team/ionic/blob/0a7aae28a7eb0270cdcd100933c01850403b66db/core/src/components/header/header.utils.ts#L119
and also the calls where the opacity of the main toolbar is changed:
https://github.com/ionic-team/ionic/blob/0a7aae28a7eb0270cdcd100933c01850403b66db/core/src/components/header/header.utils.ts#L63
...it works like expected.

Let me know if you should need an example repo, but it should be quite easy to reproduce it from the example code from the docs.
Other information:
Related to overscroll but more of a performance thing: you could possible save a few DOM writes here:
https://github.com/ionic-team/ionic/blob/0a7aae28a7eb0270cdcd100933c01850403b66db/core/src/components/header/header.utils.ts#L52
if you only actually scale the title if scrollTop is <= 0, overscroll(-top) is always a positive margin and always results in scale being 1.
Ionic info:

Ionic:
   Ionic CLI                     : 5.4.5 (/usr/local/lib/node_modules/ionic)
   Ionic Framework               : @ionic/angular 4.11.3
   @angular-devkit/build-angular : 0.803.12
   @angular-devkit/schematics    : 8.3.12
   @angular/cli                  : 8.3.12
   @ionic/angular-toolkit        : 2.0.0

Cordova:
   Cordova CLI       : 9.0.0<EMAIL_ADDRESS>   Cordova Platforms : ios 5.0.1
   Cordova Plugins   : cordova-plugin-ionic-keyboard 2.2.0, cordova-plugin-ionic-webview 4.1.2, (and 26 other plugins)

Utility:
   cordova-res : not installed
   native-run  : not installed

System:
   ios-deploy : 1.9.2
   ios-sim    : 8.0.2
   NodeJS     : v12.4.0 (/usr/local/bin/node)
   npm        : 6.11.3
   OS         : macOS Catalina
   Xcode      : Xcode 11.2 Build version 11B52

Thanks for the issue. I was able to reproduce this issue using the following markup:
    <ion-app>
      <ion-header>
        <ion-toolbar>
          <ion-title>Settings</ion-title>
        </ion-toolbar>
      </ion-header>

      <ion-content>
        <ion-header collapse="condense">
          <ion-toolbar>
            <ion-title size="large">Settings</ion-title>
          </ion-toolbar>
        </ion-header>
      </ion-content>
    </ion-app>

The issue goes away if disable this line: https://github.com/ionic-team/ionic/blob/0a7aae28a7eb0270cdcd100933c01850403b66db/core/src/components/header/header.utils.ts#L119
So far I haven't been able to reproduce this outside of Ionic, so I'm not sure if this is a WebKit bug that we'll need to find a workaround to, or a bug in Ionic Framework. I'll keep digging and update this thread when I have more info. Thanks!

I was able to reproduce this issue outside of Ionic: https://codepen.io/liamdebeasi/pen/xxxWNjM
I'm going to see if there's a way to work around this behavior. I will update this thread when I have more. Thanks!

Ok can you give this dev build a try and let me know if it fixes the issue for you?
npm i<EMAIL_ADDRESS>

@liamdebeasi Its fixed in the dev build - great work, thanks for all the effort you put into this!

But I'm seeing the issue described in #19682 again now, the main header title of the entering page is briefly visible when navigating:

This dev build didn't include the fix in https://github.com/ionic-team/ionic/pull/19682 so that's why it still has that behavior.

This issue has been resolved via https://github.com/ionic-team/ionic/pull/19850 and will be available in an upcoming release of Ionic Framework. Thanks!


PDF size for embedded image

I'm trying to convert a tif file that is 80k to a pdf and while the conversion to pdf works, it is 25MB in size.  Is there a way to enable compression or reduce the DPI?
Here's the code I'm using:
        public byte[] ImageToPdf(byte[] image)
        {
            var d = new CorePDF.Document();

            d.Images = new List<CorePDF.Embeds.ImageFile>
            {
                new CorePDF.Embeds.ImageFile
                {
                    FileData = image,
                    Name = "Image"
                }
            };

            d.Pages.Add(new CorePDF.Pages.Page
            {

                PageSize = Paper.PAGEA3PORTRAIT,
                Contents = new List<Content>() {
                new CorePDF.Contents.Image
                {
                    ImageName = "Image",
                    PosX = 50,
                    PosY = 100,
                    ScaleFactor = 0.25m
                },

                }
            });

            using (var stream = new MemoryStream())
            {
                d.Publish(stream);
                return stream.ToArray();
            }
        }

Image compression isn't something that has been implemented in this utility (but it is a good idea if you want to add it :-) ). One suggestion would be to use the ImageSharp library (for example) to compress the image before you add it to the document.
https://github.com/SixLabors/ImageSharp

I'll check it out thanks.  But just to confirm (I'm not an expert on pdfs or image processing), even though the image file is already small (80k), you are saying it is normal for the PDF size to then be 25MB if the image is not first compressed--yes?  It just seems strange that the pdf conversion process would blow it up like that.

I am not sure why the TIFF would grow from 80k to 25MB but the way images are saved in the pdf is to take the rgb bytes from the image and write them to the document so at most I would expect the image to grow (maybe) by 3x assuming that the TIFF is not itself compressed. If the TIFF is compressed then perhaps the act of uncompressing it to get the raw bytes is what has caused it to grow in such a way. Is the image large (in terms of the number of pixels)?

Yes it's 2544x3394 @ 300 dpi.

Ok, so that makes sense then. That comes to 8,634,336 bytes (approx. 8MB) per color x 3 (for RGB) is approx. 24MB. You could try setting the CompressContent property in the document (in your sample code that would be d.CompressContent) to true and see if that helps keep the size down. But I can't recall if that was implemented for images...

I just tried that--no difference.  I'll try to resize and convert to PNG and see if that makes any difference.  Thanks!

Resizing it to a more dimensions fixed my issue.  I used ImageSharp to resize then added it to the pdf and it came out as expected (< 2MB).
Thanks!


exploitdb: 2021-10-23 -> 2021-10-30

Motivation for this change
Update to latest upstream release 2021-10-30
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 143854 run on x86_64-linux 1

1 package built:

exploitdb

Takes over #143493


changeset version --ignore work not normal

Affected Packages
For instance, i have monorepo project which have package A and B, if A depends on B, i use changeset to generate B's changeset file, when i use changeset version --ignore A, the dependencies B in A will still change its version.
Problem
As Affected Packages description.
Proposed solution
I hope --ignore can totally ignore A's change, even A depends on B, i just want to update B but not A...

Changesets have some opinions about how it works and how it updates the packages etc. One of the main principles is that we favor safety whenever possible (and reasonable). Imagine this situation:

you release B without changing the dependency range in A
you develop A, starting to rely on something that has been recently added/fixed/whatever in B (you are within a monorepo after all, so u have a direct access to the most fresh code for each package at all times)
you release A, the dependency range for B stays untouched, and thus you are releasing for your consumers something that is partially broken

Of course, we could flip things and update dependency ranges when versioning dependants (whereas we mostly do that at the moment when versioning dependencies). I don't see a strong reason to do this though as you'd end up with mismatched dependency ranges and current versions of packages - which usually is a sign that a monorepo is in some kind of an inconsistent state.
I'm going to close this as part of cleaning up issues for housekeeping purposes. However, please don't hesitate to respond back here or in another issue - I will reopen this if you feel like the issue has not been addressed in full.


[explorer] switches getObjectInfo to batching function in Owned Objects

@666lcz --> this will need to be updated for the 'batching' function when this is ready.

This was resolved in another PR


⚠ Bofrost.at has degraded performance

In 0d3150b, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 3625 ms

Resolved: Bofrost.at performance has improved in 1fcd107.


[Event Request] table 5902 "Service Line" - OnBeforeTestQtyToConsumePositive

Hi, would it be possible to get an OnBeforeTestQtyToConsumePositive event in table 5902 (Service Line), like this:
        field(5917; "Qty. to Consume"; Decimal)
        {
            BlankZero = true;
            Caption = 'Qty. to Consume';
            DecimalPlaces = 0 : 5;

            trigger OnValidate()
            var
                IsHandled: Boolean;
            begin
                if CurrFieldNo = FieldNo("Qty. to Consume") then
                    CheckWarehouse;

                // New event begin
                IsHandled := false;
                OnBeforeTestQtyToConsumePositive(Rec, CurrFieldNo, IsHandled);
                if not IsHandled then begin
                // New event end
                    if "Qty. to Consume" < 0 then
                        FieldError("Qty. to Consume", Text029);
                end; // New event

                if "Qty. to Consume" = MaxQtyToConsume then
                    InitQtyToConsume
                else begin
                    "Qty. to Consume (Base)" := CalcBaseQty("Qty. to Consume", FieldCaption("Qty. to Consume"), FieldCaption("Qty. to Consume (Base)"));
                    UOMMgt.ValidateQtyIsBalanced(Quantity, "Quantity (Base)", "Qty. to Consume", "Qty. to Consume (Base)", "Quantity Consumed", "Qty. Consumed (Base)");

                    InitQtyToInvoice;
                end;

                if "Qty. to Consume" > 0 then begin
                    "Qty. to Ship" := "Qty. to Consume";
                    "Qty. to Ship (Base)" := "Qty. to Consume (Base)";
                    UOMMgt.ValidateQtyIsBalanced(Quantity, "Quantity (Base)", "Qty. to Ship", "Qty. to Ship (Base)", "Quantity Shipped", "Qty. Shipped (Base)");
                    "Qty. to Invoice" := 0;
                    "Qty. to Invoice (Base)" := 0;
                end;

                if ("Qty. to Consume" * Quantity < 0) or
                   (Abs("Qty. to Consume") > Abs(MaxQtyToConsume))
                then
                    Error(
                      Text028,
                      MaxQtyToConsume);
                if ("Qty. to Consume (Base)" * "Quantity (Base)" < 0) or
                   (Abs("Qty. to Consume (Base)") > Abs(MaxQtyToConsumeBase))
                then
                    Error(
                      Text032,
                      MaxQtyToConsumeBase);

                if (xRec."Qty. to Consume" <> "Qty. to Consume") or
                   (xRec."Qty. to Consume (Base)" <> "Qty. to Consume (Base)")
                then
                    Validate("Line Discount %");
            end;
        }

    [IntegrationEvent(false, false)]
    local procedure OnBeforeTestQtyToConsumePositive(var ServiceLine: Record "Service Line"; CallingFieldNo: Integer; var IsHandled: Boolean)
    begin
    end;

Thanks!

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


Intellisense on some websites automatically adds closing brackets whenever you add a opening bracket.

Remove the additional brackets added by intellisense, before autotyping again
Fix this behaviour, as it is leading to multiple extra closing brackets at the end of the autotyped code.

Hello @tushar5526 , can you provide an example to help me understand better?


Missing user warning - qt event loop should be started before launching napari

🐛 Bug

Failing to use the %gui qt magic in ipython/jupyter before launching napari no longer shows you a handy warning message explaining
To Reproduce
Steps to reproduce the behavior:

conda create -n napari-testenv python=3.7 pip
conda activate napari-testenv
Start ipython and then import napari and show the viewer.

import napari
viewer = napari.Viewer()

You'll see this blank, black napari window (not what we want), and there is no error message explaining that the user should try to use %gui qt before starting napari.

Expected behavior

I expected a warning message to be printed to the terminal, telling the user that they should start the qt loop (with %gui qt or similar) before starting napari.
Environment

Please copy and paste the information at napari info option in help menubar here:

napari: 0.2.4.dev736+g79ab358
Platform: Linux-5.8.0-41-generic-x86_64-with-debian-bullseye-sid
System: Ubuntu 20.04.2 LTS
Python: 3.7.9 | packaged by conda-forge | (default, Dec 9 2020, 21:08:20) [GCC 9.3.0]
Qt: 5.15.2
PyQt5: 5.15.2
NumPy: 1.20.1
SciPy: 1.6.0
Dask: 2021.02.0
VisPy: 0.6.6

OpenGL:
- GL version: 4.6 (Compatibility Profile) Mesa 20.2.6
- MAX_TEXTURE_SIZE: 16384

Screens:
- screen #1: resolution 1920x1080, scale 1.0
- screen #2: resolution 1920x1080, scale 1.0

Plugins:
- svg: 0.1.4

Any other relevant information:

Additional context

Hi @GenevieveBuckley,
The reason for this is that it's no longer an error to create a viewer without using gui_qt first (like it used to be).  But you do need to use either gui_qt or napari.run() (the latter is preferable) to actually show the viewer. Think of this now as similar to having to use plt.show in matplotlib.
Basically, we're phasing out gui_qt.  See https://github.com/napari/napari/pull/2056#discussion_r550924887 for more background and rationale on this new behavior. And let us know what you think

You'll see this blank, black napari window (not what we want), and there is no error message explaining that the user should try to use %gui qt before starting napari.

@tlambert03 but wouldn't you expect to not even see that black screen? I actually don't see it, but i do see focus leave ipython and then return. I guess that's because Window is created in the init?
The discussion https://github.com/napari/napari/pull/2225#issuecomment-775157902 is very relevant to I think, and I'm curious @GenevieveBuckley if you have ideas there too!!

yeah, I don't get it either... so I'd need to know more about how the script is being run.

I got the black screen on both Ubuntu 20.04 and on my mac laptop, following the "steps to reproduce" in my original comment.
I'm not sure what you mean by "how the script is being run" @tlambert03. I haven't been trying to run a python script, I've just been trying to launch napari from an interactive ipython console.

hmm, can you try again with the latest master @GenevieveBuckley? maybe we've fixed at least this part lately

I did try with master yesterday, I'll give it another go now

Ok, here's what's happening.
I open a terminal, activate my napari-dev conda environment, and start ipython.
This works as expected:
%gui qt

import napari

viewer = napari.Viewer()
# As expected, a new napari window is opened

But if you forget to use %gui qt or similar, you no longer see the helpful error message suggesting how to fix this.
import napari

viewer = napari.Viewer()  # this shows the blank, black window (see screenshot in original post)
viewer.show()  # this does nothing
napari.run()  # this now shows napari in what was the blank window
# but now the ipython console is locked and won't let you type anything else

This is very strange.  Without the ipython magic and without napari.run, there should be no event loop to show any window at all.
I'd say to try a new environment but it sounds like you've already seen this on two computers.  Can you give me the full output of your pip freeze?

And what you observed above is the expected behavior, with the exception of the black screen

Yeah, the black screen is weird.
pip freeze output, napari development environment, Ubuntu 20.04
alabaster==0.7.12
appdirs==1.4.4
asciitree==0.3.3
attrs==20.3.0
Babel==2.9.0
backcall==0.2.0
black==20.8b1
build==0.2.0
cachey==0.2.1
certifi==2020.12.5
cfgv==3.2.0
chardet==4.0.0
check-manifest==0.46
click==7.1.2
cycler==0.10.0
dask==2021.2.0
decorator==4.4.2
distlib==0.3.1
docstring-parser==0.7.3
docutils==0.16
fasteners==0.16
filelock==3.0.12
flake8==3.8.4
freetype-py==2.2.0
fsspec==0.8.5
HeapDict==1.0.1
identify==1.5.13
idna==2.10
imageio==2.9.0
imagesize==1.2.0
importlib-metadata==3.4.0
iniconfig==1.1.1
ipykernel==5.4.3
ipython==7.20.0
ipython-genutils==0.2.0
jedi==0.18.0
Jinja2==2.11.3
jupyter-client==6.1.11
jupyter-core==4.7.1
kiwisolver==1.3.1
magicgui==0.2.6
MarkupSafe==1.1.1
matplotlib==3.3.4
mccabe==0.6.1
mypy-extensions==0.4.3
-e git+git@github.com:GenevieveBuckley/napari.git@133ab7f70803b7db26f772335b30fb96e4a44b02#egg=napari
napari-plugin-engine==0.1.9
napari-svg==0.1.4
networkx==2.5
nodeenv==1.5.0
numcodecs==0.7.3
numpy==1.20.1
numpydoc==1.1.0
packaging==20.9
pandas==1.2.1
parso==0.8.1
pathspec==0.8.1
pep517==0.9.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.1.0
pluggy==0.13.1
pooch==1.3.0
pre-commit==2.10.1
prompt-toolkit==3.0.14
psutil==5.8.0
ptyprocess==0.7.0
py==1.10.0
pycodestyle==2.6.0
pydantic==1.7.3
pyflakes==2.2.0
Pygments==2.7.4
PyOpenGL==3.1.5
pyparsing==2.4.7
PyQt5==5.15.2
PyQt5-sip==12.8.1
pytest==6.2.2
pytest-faulthandler==2.0.1
pytest-qt==3.3.0
pytest-timeout==1.4.2
python-dateutil==2.8.1
pytz==2021.1
PyWavelets==1.1.1
PyYAML==5.4.1
pyzmq==22.0.2
qtconsole==5.0.2
QtPy==1.9.0
regex==2020.11.13
requests==2.25.1
scikit-image==0.18.1
scipy==1.6.0
six==1.15.0
snowballstemmer==2.1.0
Sphinx==3.4.3
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==1.0.3
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.4
tifffile==2021.2.1
toml==0.10.2
toolz==0.11.1
tornado==6.1
traitlets==5.0.5
typed-ast==1.4.2
typing-extensions==<IP_ADDRESS>
urllib3==1.26.3
virtualenv==20.4.2
vispy==0.6.6
wcwidth==0.2.5
wrapt==1.12.1
xarray==0.16.2
zarr==2.6.1
zipp==3.4.0

I'm sorry, I just tried this with python 3.7 and as similar of an environment as I could and couldn't repeat on mac.  my ubuntu setup is currently bonked so, will take me a while to rebuild that to test this out there.

Cross linking to #2238 as it will change some of this behaviour

I wonder if this is related to my random affinder window launch issue, which also could not be reproduced in a fresh env. @GenevieveBuckley could you try to create a minimal environment to see whether it also happens in a fresh env?

was able to try this on ubuntu 20.04 and saw the black screen.  will see what I can learn.  @jni, have you not seen this yet?

ok, so it looks like I can reproduce this without even using napari:
from qtpy.QtWidgets import QApplication, QMainWindow
app = QApplication([])
mw = QMainWindow()
mw.show()  # black screen shows
app.exec_()  # it turns into the "standard" white window (like `napari.run()`)

need to figure out why this happens on ubuntu, and whether it's unique to 20.04.  @goanpeca, any insight here?

@jni, have you not seen this yet?

I haven't encountered this because %gui=qt is part of my IPython default profile. 😂

@GenevieveBuckley, can you try something for me?
in qt_main_window.py, in Window.__init__ ... change this
        if show:
            self.show()

to this:
        if show:
            from qtpy.QtCore import QTimer
            QTimer.singleShot(0, self.show)

and see if it avoids the black screen until you use napari.run?

I've tried your suggestion @tlambert03 and it seems to make things worse, not better. Sorry!
(napari-dev) genevieve@genevieve-G5-5500:~$ ipython
Python 3.7.9 | packaged by conda-forge | (default, Dec  9 2020, 21:08:20)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.20.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import napari

In [2]: viewer = napari.Viewer()

In [3]: viewer.show()  # I still see the black screen appear here

In [4]: viewer.run()  # but now things are broken instead of me seeing the normal napari window
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-4-3a674e6670be> in <module>
----> 1 viewer.run()

AttributeError: 'Viewer' object has no attribute 'run'

it should be napari.run ...  not viewer.run
but in any case, the window shouldn't have shown at  viewer.show().  I'm really confused... do this without napari if you can:
from qtpy.QtWidgets import QApplication, QMainWindow
app = QApplication([])
mw = QMainWindow()

# do you see anything here?  If so, we're probably screwed.

# however... here's where I'd expect you to see the black screen
mw.show()

if you see the black screen there.  try this:
from qtpy.QtWidgets import QApplication, QMainWindow
from qtpy.QtCore import QTimer

app = QApplication([])
mw = QMainWindow()
QTimer.singleShot(0, mw.show)  # do you see a black screen?

# if not ... then do this:
app.exec_()  # do you see a non-black screen?

I just got my linux system back up and running, and this fix does work for me to prevent the black box.  @GenevieveBuckley, I'm not sure what went wrong when you tested it above, but maybe you can try #2276, which is what I was trying to suggest in https://github.com/napari/napari/issues/2233#issuecomment-777864183

it should be napari.run ... not viewer.run

Gah, of course - sorry!

do this without napari if you can:
from qtpy.QtWidgets import QApplication, QMainWindow
app = QApplication([])
mw = QMainWindow()

# do you see anything here?  If so, we're probably screwed.

# however... here's where I'd expect you to see the black screen
mw.show()

We're not screwed, this works the way you expected & wrote above.

if you see the black screen there. try this:
from qtpy.QtWidgets import QApplication, QMainWindow
from qtpy.QtCore import QTimer

app = QApplication([])
mw = QMainWindow()
QTimer.singleShot(0, mw.show)  # do you see a black screen?

# if not ... then do this:
app.exec_()  # do you see a non-black screen?

Yep, this also works like you expect. I don't see any kind of screen until the last line, when app.exec_() pops up a grey dialog box

Hi @GenevieveBuckley,
The reason for this is that it's no longer an error to create a viewer without using gui_qt first (like it used to be). But you do need to use either gui_qt or napari.run() (the latter is preferable) to actually show the viewer. Think of this now as similar to having to use plt.show in matplotlib.
Basically, we're phasing out gui_qt. See #2056 (comment) for more background and rationale on this new behavior. And let us know what you think

I apologize, I totally missed this earlier.
I think:

It's fantastic you're doing this work making sure headless napari can work well
I worry that for new users this might become a bit of a trap, and we don't have good ways to redirect them back to what they thought they were doing in the first place. It's only "not a mistake" if you meant to be running napari headless. If you didn't mean that, this is a very fast road for newcomers to "napari is a piece of crap, it doesn't even work for the intro tutorials".

Since the removal of the user warning was intended, I'll close this issue.
I would like it if we had a better way to address this barrier to adoption:

I worry that for new users this might become a bit of a trap, and we don't have good ways to redirect them back to what they thought they were doing in the first place. It's only "not a mistake" if you meant to be running napari headless. If you didn't mean that, this is a very fast road for newcomers to "napari is a piece of crap, it doesn't even work for the intro tutorials".

It's only "not a mistake" if you meant to be running napari headless.

would you then suggest that it's a mistake to make a matplotlib figure without showing it?

"napari is a piece of crap, it doesn't even work for the intro tutorials".

I'm not sure I agree on that ... surely, parity between the tutorials and the current state of master is an independent topic from how things "should work" ... meaning, we have to be able to evolve from how the tutorials were originally drafted, and yes, we'll need to update our tutorials from time to time.  (I'd be all for completing removing napari.gui_qt from our tutorials and moving to the new syntax!)
with that said ... all of the tutorials still work!  %gui qt works and napari.gui_qt works... we never suggest in our tutorials that you should use napari without one of those.
basically, the only "fix" to your request that I can think of here is that we autodetect whether someone is using ipython without having %gui qt active, and suggest to them that they might "like to try" %gui qt.  As I see it, that's a optional third party feature that napari happens to work well with, but it should not be an error (or even a warning) if you don't use it.

if you meant to be running napari headless.

it's also not even about running strictly headless or not.  If you're not using ipython's magic %gui qt, (which again, is an optional third party event loop feature), then you would definitely need to set up some additional things before starting the event loop with napari.run().  That's why we had the gui_qt context in the first place (because no one should have to use %gui qt)... but for all the reasons mentioned in https://github.com/napari/napari/pull/2056#discussion_r550924887, it was never well-suited to a context manager... so the preferred approach, if you're not using %gui qt, is now like matplotlib:
viewer = napari.Viewer()
# hook up all stuff that you'd like to do before you actually "run" the viewer
napari.run()

basically, the only "fix" to your request that I can think of here is that we autodetect whether someone is using ipython without having %gui qt active, and suggest to them that they might "like to try" %gui qt. As I see it, that's a optional third party feature that napari happens to work well with, but it should not be an error (or even a warning) if you don't use it.

This is the crux of it and I wonder if it's not a good idea to do this anyway? Especially given the large community we've built used to gui_qt()/%gui qt. ie if we're in IPython and %gui qt hasn't been launched when we instantiate the viewer, warn the user that they want to use napari.run() or %gui qt. What do you think @tlambert03?

as i laid out above, I'm not a fan... but it's a democracy and I'm happy to be outvoted! :joy:
I absolutely think we have an obligation to those who already use gui_qt() to make sure it doesn't break without a solid deprecation period.  And I also definitely think we should continue to tell people about %gui qt in our documentation, just as we always have, and to make sure that it works well with napari.   But I use napari in IPython all the time without %gui qt and I really don't want to be warned about it.  It feels akin to matplotlib warning you everytime you made a figure "hey you forgot to use plt.ion()!"

But I use napari in IPython all the time without %gui qt

You monster! 😂

I think it's a bit different to the matplotlib example because in my view the real power with napari is interactivity, whereas in matplotlib you generally want to construct the entire figure before seeing it.
But I'm not trying to start a philosophical argument here. I just missed that there was an intentional change here & was trying to be  helpful by flagging it.

I think it's the warning for completely acceptable usage that bothers me.  I feel like the "use %gui qt" suggestion belongs in the documentation and tutorials more than the code itself.   However, an alternative that I would be less opposed to is actually making  the napari.Viewer be interactive by default when running in IPython.  This could be done as follows:
# qt_main_window.py
class Window:
    ...

    def show(self, interactive=True):
        from .qt_event_loop import _ipython_has_eventloop
        from ..utils.misc import in_ipython

        if interactive and in_ipython() and not _ipython_has_eventloop():
            from IPython import get_ipython

            get_ipython().enable_gui('qt')
        ...

However, an alternative that I would be less opposed to is actually making the napari.Viewer be interactive by default when running in IPython.

I'd support this, on the basis that headless mode is likely to be less commonly used (and when it is used, those power users are likely to be people with stronger coding skills & ability to troubleshoot).


Potential links are not shown in MeshDB's map

The spreadsheet allows links to be created between potential installs, e.g:
https://www.nycmesh.net/map/nodes/15252-15253
This does not show up in our map:
http://map.grandsvc.mesh/nodes/15252-15253
This is because our data model only allows links between devices, and only allows devices for nodes, so we would have to consume an NN just to show planned links on the map (not good)
We will likely need to revise the data model in some way to fix this, eek

Willard, James, and I had a discussion about this tonight. We've decided that the concept we're representing here can be adequately represented by a new type of table, called "Sightline", "LineOfSight", or "LOS"
Each row in this table would represent a pair of buildings that are able to be potentially connected by a mesh antenna pair. Entries could be placed manually by volunteers, or in the future, automatically by LOS analyzer tools.
When we render the JSON object for the map, we would elect a representative install/network number (or include all permutations?) for each building on each end of each row in this table, and include them as potential links in the map.
Columns:

From Building
To Building
Date Checked
Source: Existing Link,Human Analysis (LOSTool in future?) (Maybe google earth vs panorama?)


Modified to allow multiple ignoreinds as a cell array

Our fixes to preprocessvols and addition of collapsedesign function.


How can I show Sub heading in the table?

When I am using htmltabletomd.convert_table(html_table) - I could see that Only headers and data are converted to table -  sub headers are not considerd in the data and they are printed as null

@Mian-Ahmed-Raza  Thanks for your help   - I Have a  similar structure in the website but the sub-headers are captured as null. Attaching the structure below.

@sree1658 The issue might be due to how the  tags are organized in the table, or how colspan and rowspan are being used. Let me help you with a structured version that should resolve the sub-header issue.
Fixed Table Structure:

Progress of Sowing of Kharif Crops (As of 23 Sep 2024)

            Actual area ('000 hectare)
            % change in actual area sown

            2022
            2023
            2024
            2023
            2024

            Agricultural products
            109,923
            108,826
            110,465
            -1.0
            1.5

Now, the sub-headers should appear correctly without being captured as null. If you still face issues or need further assistance with this table or any other aspect, let me know!

@Mian-Ahmed-Raza - I am trying to scrap a website so wont be able to change the html format - Thanks for the help anyway - Lesson learnt. 😊🙂

@sree1658 no problem dear😊😊


(BOLT-883) Create statedir

Without this, we get errors like
The task failed with exit code 1:
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:54:in `rescue in rescue in init_storage': Cannot remove /tmp/d20181003-855-156fn3k/state/state.yaml: No such file or directory @ unlink_internal - /tmp/d20181003-855-156fn3k/state/state.yaml (Puppet::Error)
	from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:50:in `rescue in init_storage'
	from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:45:in `init_storage'
	from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:205:in `run'
	from /tmp/c33af27e-1acd-495c-a495-304fd058ded3/apply_catalog.rb:81:in `block in <main>'
	from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
	from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:260:in `override'
	from /tmp/c33af27e-1acd-495c-a495-304fd058ded3/apply_catalog.rb:69:in `<main>'

I haven't figured out why this seems to be necessary yet.

Shouldn't that happen with https://github.com/puppetlabs/puppet/blob/master/lib/puppet/util/storage.rb#L45?

Ah yeah it should. That does make this a bit more mysterious.

Ok, I think this is better now.

Something deeper's going wrong here.

I've tested this with https://github.com/puppetlabs/bolt/pull/662, which gives me more confidence it'll work.

CLA signed by all contributors.


Does this work for huge audio files and their transcripts?

Thanks for creating this project. I have a doubt regarding its usage, suppose I have university lectures which are 45 minutes in size and the entire transcript for this lecture. Will this method still work for this case

hi @ramaneswaran! this now works when you use align-long, e.g.
timething align-long \
    --audio-file fixtures/audio/keanu.mp3 \
    --transcript-file fixtures/keanu.cleaned.txt \
    --alignments-dir aligned \
    --batch-size 10 \
    --n-workers 5

previously it did not work for 45 minute lectures, because of the length restrictions imposed by the underlying transformer model.


Change version 0.6.2 to 0.6.4 (devnet branch)

Please note, we work in the main branch rather than devnet. These updates should already exist there.


How to always receive context on probot during app start

Hi,
I want the bot to auth with github so I can get the context before any event even triggers:
app.on('start', async (context) => {

    const octokit = await app.auth();
    const { data } = await octokit.rest.repos.<>({
      org: 'org-name',
      type: 'public'
    });

    console.log(data);
  });

or auth outside of any event.
Right now I have to manually trigger something in github to get auth or context.
Thanks.

Ignore this, after adding the installation id it is retrieving even outside of events.
Thanks.


return all buckets if privacy bucket mappers operative field set is empty

…mpty

This change is

Thank you. While you are here, I think it will be good to also add a comment here that states if the operative fields are empty, normalization operation is not done.


chore: release 1.1.1

:robot: I have created a release *beep* *boop*
1.1.1 (2019-06-26)
Bug Fixes

docs: link to reference docs section on googleapis.dev (#356) (9deaea3)

This PR was generated with Release Please.

Release is at https://github.com/googleapis/nodejs-logging-winston/releases/tag/v1.1.1

The release build has started, the log can be viewed here. :sunflower:

:egg: You hatched a release! The release build finished successfully! :purple_heart:


added inorder tree traversal

Inorder traversal added. Will be adding more traversals too. Pls check @siddharth25pandey .

@shubhangi-singh21 Please edit the README file too with your name in Contributor's List

@shubhangi-singh21 Please edit the README file too with your name in Contributor's List

Sure, i'll do that after adding 2 more traversal techniques. Will that be okay?

@shubhangi-singh21 Please edit the README file too with your name in Contributor's List

Sure, i'll do that after adding 2 more traversal techniques. Will that be okay?

@shubhangi-singh21 Yeah fine, no worries

@siddharth25pandey Made all changes. Pls check now.

@all-contributors please add @shubhangi-singh21  for her code


Install fails with PySixel

Can't install the package because of PySixel-related build failure as described here https://github.com/saitoha/PySixel/issues/4. Do you have any workaround for that maybe?

@jwitos
I added a new version of dcmagick.  Please try it!


Definition for prefixed routes

All my services are versioned and available under the /api/{version}/ prefix.
I'm trying to create definition for my services, but they all end up in the api swagger group, is there a way to prevent this?

Not at the moment I think. One fix would be to add a config.prefix and then in https://github.com/feathersjs/feathers-swagger/blob/master/src/index.js#L65 set something like path = ${config.prefix}/${path}`

At least do you know if it's possible to override the api group during the definition settings? That would be an acceptable workaround.

Why not a pull request that adds what I suggested?

Why not a pull request that adds what I suggested?

Because it will not work when different prefix are used for services (like /api/v1 and /api/v2) which is the primary point of versioning 😉


Deletion of CRD with crd-install and before-hook-creation randomly fails (race condition)?

This is a follow-up to #5278 and is possibly related to #4925.
Output of helm version:
Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Output of kubectl version:
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.3", GitCommit:"721bfa751924da8d1680787490c54b9179b1fed0", GitTreeState:"clean", BuildDate:"2019-02-01T20:08:12Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.4", GitCommit:"f49fa022dbe63faafd0da106ef7e05a29721d3f1", GitTreeState:"clean", BuildDate:"2018-12-14T06:59:37Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
Cloud Provider/Platform (AKS, GKE, Minikube etc.):
Rancher2 on-premise
Issue description:
I'm installing a helm chart with a CRD and the annotations
  annotations:
    helm.sh/hook: crd-install
    helm.sh/hook-delete-policy: before-hook-creation

When I Install/remove the chart multiple times, sometimes it fails with the following error:
Release "hook-test" does not exist. Installing it now.
Error: object is being deleted: customresourcedefinitions.apiextensions.k8s.io "tests.my.test.com" already exists

A test to reproduce this follows.

yeah, this appears to be a duplicate of #4925. Can you test https://github.com/helm/helm/pull/5112 and see if that fixes it for you?

Here's a test chart that reproduces this error: https://github.com/micw/helm-hook-test
Just let run_test.sh run - it installs/removes the chart in a loop. On my cluster this fails randomly every 2nd or 3rd run.

@bacongobbler I worked out a test that makes this more or less reproducible. Unfortunately I have no idea how I could verify if this PR solves this issue. How can I obtain a helm binary that contains this fix?

You'll have to build from source following the developer guide. Let me know if you run into any issues.

I did it this way: https://gist.github.com/micw/30190083b3e5d7df8dc30bc1526bdb57 ;-)

@bacongobbler Now I have a custom build new helm/tiller version. Unfortunately, the tiller version which is deployed it one from a docker image, not my local one...
I can see no good way how I can easily test this.

I have build and pushed a tiller docker image this way:
https://gist.github.com/micw/30190083b3e5d7df8dc30bc1526bdb57
Then I changed my k8s to use this tiller image (via kubectl patch):
bin/helm version
Client: &version.Version{SemVer:"v2.11+unreleased", GitCommit:"57d589fb28c16f0e3418d0967bfb52772adc578e", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.11+unreleased", GitCommit:"57d589fb28c16f0e3418d0967bfb52772adc578e", GitTreeState:"clean"}

Then I changed my test to  use the custom built helm binary.
The issue still exists:
Release "hook-test" does not exist. Installing it now.
Error: object is being deleted: customresourcedefinitions.apiextensions.k8s.io "tests.my.test.com" already exists


[FEATURE] wash up ensure that the wasmcloud-host can be stopped gracefully

Is your feature request related to a problem? Please describe.
I  started two wasmclouds with wash up and they are connected to the same nats.
When I stop one of the wash up (without  running nats) with ctrl-c, I can still show the stopped host information in the page of the another wasmboard.
After some troubleshooting

I found that wasmcloud-host caches hosts and updates and deletes hosts based on event
https://github.com/wasmCloud/wasmcloud-otp/blob/1f276d070d57366135485a0b367e9620eadf172e/wasmcloud_host/lib/wasmcloud_host/lattice/state_monitor.ex#L30-L33

And the event type to delete hosts is com.wasmcloud.lattice.host_stopped.
https://github.com/wasmCloud/wasmcloud-otp/blob/1f276d070d57366135485a0b367e9620eadf172e/wasmcloud_host/lib/wasmcloud_host/lattice/state_monitor.ex#L325

VirtualHost.publish_host_stopped is called to send a stop event only when a Wasmcloud Host Application is gracefully stopped
https://github.com/wasmCloud/wasmcloud-otp/blob/1f276d070d57366135485a0b367e9620eadf172e/host_core/lib/host_core/vhost/virtual_host.ex#L435

wash up kill wasmcloud-host directly after receiving ctr-c, and does not gracefully stop wasmcloud-host
https://github.com/wasmCloud/wash/blob/389a7023b9a6c584d27e2b48573f21e7b09c41ba/src/up/mod.rs#L638-L645

Describe the solution you'd like
I want wash up to gracefully stop wasmcloud-host when it receives ctrl-c, for example by sending SIGTERM to it.
This allows the host to broadcast that it has stopped, and ensures that wasmcloud stops gracefully

@Iceber great catch here, this is in fact a bug in wasmcloud-otp where hosts are not removed from the washboard. Going to transfer that issue to that repository, for clarity.
For anyone looking to take on this issue, the host should be removed if:

The host_stopped event is received
The host hasn't been heard from in 2 or 3 heartbeat intervals


support all the .*ignore files

I just forked and release basically this exact same utility but for .dockerignore files. My modifications were basically just a simple search & replace.
It would probably be better to just generify this utility to handle any .gitignore-like files. Because, according to Intellij, there are quite a few of them:

It would involve moving the guts of this module to another one (ignore-core, etc). And each individual utility would simply require it, and pass in the particular file name it works with.
// npmignore index.js
module.exports = require('ignore-core')('.npmignore');

// npmignore cli.js
module.exports = require('ignore-core/cli')(require('./'), '.npmignore');

and similar for the other file types.
we could also create another group command that updates all the files present in the directory (npmignore, dockerignore, etc, all get updated with one command).
I haven't looked into all those file types yet (not sure they are all compatible with .gitignore).

crazy coincidence, I'm working on a parser for ignore files literally right now lol. ever since I released this lib yesterday, it's been bugging me that the gitignore patterns don't map directly to glob patterns - and none of the other tools I've found do it right.
if you create that lib, once I get that parser published it would be great to have it included if it's useful.


Fix filter pattern cheat sheet examples

Why:
fixes #855
In the Filter pattern cheat sheet examples section of https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-syntax-for-github-actions#filter-pattern-cheat-sheet:

Regex example for matching any number or lowercase letter is shown incorrectly as [0-9a-f] (any hexadecimal number)
Examples of patterns to match different paths are prepended with -, but because they are within table cells, they are shown as literal -'s instead of bullet points. Since many of the patterns actually include -, this is visually confusing.

What's being changed:

Correct the regex pattern for numbers/lowercase letters to [0-9a-z]
Remove the - before the pattern examples in the table cells (line breaks should be sufficient for separating the patterns)

Check off the following:

[x] All of the tests are passing.
[x] I have reviewed my changes in staging. (look for the deploy-to-heroku link in your pull request, then click View deployment)
[x] For content changes, I have reviewed the localization checklist
[x] For content changes, I have reviewed the Content style guide for GitHub Docs.

@tomjschuster Thanks so much for opening a PR! I'll get this triaged for review ⚡

Thanks for your patience! Our small team is working our way through reviewing all of the amazing contributions ✨

Just commenting to keep this open. Hope everyone is enjoying their holidays.


Trouble launching GPU_lidar

Hello,
The demo gpu_lidar.launch doesn't work.
The diff_drive and air pressure demos does work.

I tried using "ogre" instead of "ogre2" for the rendering (it worked for other demos like depth cameras) but the trouble persists :

ruby: /build/ogre-1.9-B6QkmW/ogre-1.9-1.9.0+dfsg1/OgreMain/include/OgreSingleton.h:80: Ogre::Singleton::Singleton() [with T = Ogre::LogManager]: Assertion `!msSingleton' failed.

The full console print is attached gpu_lidar_console.txt
I am trying to transfer my simulation from Gazebo to Ignition gazebo.
I installed Igntion Gazebo and this package following your documentation.
I am using ROS Melodic with Ubuntu 18.04.1

I've seen the Unable to create the rendering window error before. That usually means you're using an old version of OpenGL or have an old Nvidia driver. Could you check:

~/.ignition/rendering/ogre2.log and see if you have Failed to create an OpenGL 3+ context
Your OpenGL version, it should be 3+: glxinfo | grep "OpenGL version"
Your Nvidia driver version, it should be 384+: nvidia-smi

Thanks for your answer,

In the log, i have this :

14:29:01: Installing plugin: GL 3+ RenderSystem
14:29:01: OpenGL 3+ Rendering Subsystem created.
14:29:02: Plugin successfully installed

OpenGL version is : OpenGL version string: 3.0 Mesa 18.2.8

I installed  nvidia-384, nvidia-utils-390 and nvidia-driver-390 but nvidia-smi couldn't communicate with the NVIDIA driver.

Now in the .ignition log I got :

OGRE EXCEPTION(3:RenderingAPIException): OpenGL 3.3 is not supported. Please update your graphics card drivers.

But I did update it. My ubuntu kernel is 4.15.0-54. Is it possible that I need to upgrade ubuntu to the latest kernel?

If nvidia-smi didn't work, I think the problem is that you don't have the correct drivers enabled. Try switching drivers with the "Aditional Drivers" dialog?

dpkg -l | grep nvidia is giving me :

ii  libnvidia-cfg1-390:amd64
ii  libnvidia-common-390
ii  libnvidia-compute-390:amd64
ii  libnvidia-compute-390:i386
ii  libnvidia-decode-390:amd64
ii  libnvidia-decode-390:i386
ii  libnvidia-encode-390:amd64
ii  libnvidia-encode-390:i386
ii  libnvidia-fbc1-390:amd64
ii  libnvidia-fbc1-390:i386
ii  libnvidia-gl-390:amd64
ii  libnvidia-gl-390:i386
ii  libnvidia-ifr1-390:amd64
ii  libnvidia-ifr1-390:i386
rc  nvidia-340
ii  nvidia-384
ii  nvidia-compute-utils-390
ii  nvidia-dkms-390
ii  nvidia-driver-390
ii  nvidia-kernel-common-390
ii  nvidia-kernel-source-390
rc  nvidia-opencl-icd-340
ii  nvidia-prime
ii  nvidia-settings
ii  nvidia-utils-390
ii  xserver-xorg-video-nvidia-390

sudo lshw -C display gives me  :

*-display
description: VGA compatible controller
product: Intel Corporation
vendor: Intel Corporation
physical id: 2
bus info: pci@0000:00:02.0
version: 00
width: 64 bits
clock: 33MHz
capabilities: pciexpress msi pm vga_controller bus_master cap_list rom
configuration: driver=i915 latency=0

I don't know how to set nvidia driver.

There is nothing in the "additonal Drivers" dialog.

Setting up drivers can be complicated, I hope you figure it out.
I think we've established that the issue is in your local setup, and not a bug in this package, so I'm going to close this issue. Feel free to ask for help at http://answers.gazebosim.org/. Good luck :crossed_fingers:


Multiple (looped) panTo events happen too fast or only execute last panTo.

using the 1.0 RC1 version:
When I do a single panTO like this:
map.panTo(newLatLng, {animate: true, duration: 5.0} );
it works great, but if I loop through a list (Virtual Tour) and try to pan to each of the locations. it will only do the last one.
if I switch over and use:
map.flyTo(newLatLng, newZoom, {animate: true, duration: 5.0} );
same results, only the last one works. I Know the array values are good becuase each one works fine if used from the array as a single pan or fly.
I would expect to be able to loop through a list and have each one executed.
I also added a horrible sleep function to case a 2 second delay between each pan/fly and in the console I can clear see it executing the code, but nothing happens till the function doing the work exists.

This code can help you?
setTimeout(function(){map.panTo(L.latLng(-10.350002, -30.923930), {animate: true, duration: 1})}, 1000);
setTimeout(function(){map.panTo(L.latLng(10, 30), {animate: true, duration: 2})}, 2000);
setTimeout(function(){map.panTo(L.latLng(-10, 45), {animate: true, duration: 3})}, 5000);
setTimeout(function(){map.panTo(L.latLng(-10.350002, -30.923930), {animate: true, duration: 1})}, 8000);
The above solution is not the best but i thought to combine the event as below but I'm not sure yet how to work with a list of points
map.panTo(L.latLng(10, 30), {animate: true, duration: 3});
map.on('moveend', function(ev) {
map.panTo(L.latLng(-10.350002, -30.923930), {animate: true, duration: 1});
});

The problem you're having is that each call is immediately cancelling the one before it, so only the last one remains. You need to stagger them using a timeout as @EricRoque shows.
Here is how you can do it in a loop:
var _tourStops = [ L.latLng(-80, 40), L.latLng(80, 40), L.latLng(30, 0) ],
      _tourDelaySec = 5,
      _tourZoom = 13;

function runTour() {
    for (var i = 0; i < _tourStops.length; i++) {
        setTimeout(function() {
            map.flyTo(_tourStops[i], _tourZoom, {animate: true, duration: _tourDelaySec} );
        }, _tourDelaySec * i);
    }
}