Expanding Metadata Standards for Environmental and ToL Extensions - eDNAquaPlan T3.2

We are going to build on your DwC-MIxS alignment and are seeking further guidance beyond what is documented in your paper.

Dear GBWG,
My name is Camila, and I'm a Bioinformatician at BIOPOLIS-CIBIO, based in Portugal. As a research institution focused on biodiversity, we are collaborating with several EU partners on the eDNAqua-Plan project. This three-year initiative is dedicated to developing a comprehensive digital ecosystem for eDNA reference libraries aimed at marine and freshwater environments.
In this project, we serve as task leaders for Task T3.2. Our role involves providing a model that aligns eDNA (meta)data standards for all aquatic biodiversity biomonitoring workflow stages, such as MIxS, ToL, and DwC. We aim to achieve this by building on the existing work already available in this repository, as outlined in our project proposal and encouraged by Pier Luigi (@pbuttigieg), a member of the GBWG and the project's International Advisory Board.
Given that GBWG's work has focused on the MIxS core, we have discussed the possibility of expanding such an effort by mapping MIxS environmental package terms to standards in Earth and environmental sciences (ENVOs, SWEET, EOVs, GOOS, MoF?) and the ToL standards. We recognize and have reviewed the recommendation from the "Aligning Standards Communities for Omics Biodiversity Data: Sustainable Darwin Core-MIxS Interoperability" paper and intend to consider them in our work going forwards, particularly the suggestions outlined in "Recommendations for using the SSSOM mapping matrix" (inclusion of syntax_predicate_id, syntax_comment to capture syntax meaning along with the semantics, "or even definitions [subject_definition, object_definition], syntax requirements [subject_valueSyntax, object_valueSyntax"); and "Recommendations for the mapping of MIxS environmental package terms".
To make sure we are aligned with best practices, we would like to invite members of the GBWG to share any additional recommendations, insights, or cautionary advice you may have before we begin this task.
I'm looking forward to your insights and sharing what we do back to you and the wider community.
Best regards,
Camila Babo

@camilababo thank you for the description

In this project, we serve as task leaders for Task T3.2. Our role involves providing a model that aligns eDNA (meta)data standards for all aquatic biodiversity biomonitoring workflow stages, such as MIxS, ToL, and DwC. We aim to achieve this by building on the existing work already available in this repository...

As noted at the GA in Spala, this approach is strongly advised - working with standards bodies like the GSC and TDWG to persist such alignments will help sustain outcomes from your work, and allow quality control

Given that GBWG's work has focused on the MIxS core, we have discussed the possibility of expanding such an effort by mapping MIxS environmental package terms to standards in Earth and environmental sciences (ENVOs, SWEET, EOVs, GOOS, MoF?) and the ToL standards.

Extensions of the current mapping are welcome, but note that the resources listed above cannot and should not be considered standards in the same manner as GSC or TDWG standards. It is therefore important that the maps you create do not convolute very different resources.

We recognize and have reviewed the recommendation from the "Aligning Standards Communities for Omics Biodiversity Data: Sustainable Darwin Core-MIxS Interoperability" paper and intend to consider them in our work going forwards, particularly the suggestions outlined in "Recommendations for using the SSSOM mapping matrix" (inclusion of syntax_predicate_id, syntax_comment to capture syntax meaning along with the semantics, "or even definitions [subject_definition, object_definition], syntax requirements [subject_valueSyntax, object_valueSyntax"); and "Recommendations for the mapping of MIxS environmental package terms".

The reuse of mapping conventions like SSSOM is welcome. The statement above, however, is somewhat unclear. Currently, the GBWG mapping described in the paper you referenced has very particular conventions which the standards bodies have agreed to operate with. Any extensions must be compatible. We can reconvene the working group to ensure your project outputs contribute effectively.

To make sure we are aligned with best practices, we would like to invite members of the GBWG to share any additional recommendations, insights, or cautionary advice you may have before we begin this task.

There are several things to consider, that go beyond a single GitHub issue.
The most pressing concern for the moment is that we establish how your work can extend the existing DwC-MixS map (the other resources need more thought, and are generally outside of this group's remit).
My suggestion is that you create new SSSOM files for each extension/package that you wish to map and generate PRs that a GBWG team can assess and ratify.

@camilababo thanks for sharing this info!  I'm the US node manager for OBIS and GBIF, and there are a few similar efforts going on, so I'm tagging those folks, in case you haven't already crossed paths.  @tobiasgf is the main contact for DNA publishing at GBIF. I think you might already be working with @eboulanger from OBIS and eDNAqua-Plan.   @ksilnoaa is leading the development of Omics workflows at NOAA AOML. @miwa582 at CSIRO is leading the development of an eDNA checklist for MixS/DwC that might overlap quite a bit with what you're proposing.


Hope the sample can be changed to a specific URI folder rather than the / root

Great scenario sample, but one thing I'd like to add is change the default voting-app access URI to a specific path (maybe /vote) instead of root (/) for some istio deployments.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 04e91998-d53d-5074-5c67-0f9f031bc26c
Version Independent ID: 989fa4fe-e7a8-2003-6a89-73beaade1b80
Content: Intelligent routing and canary releases with Istio in Azure Kubernetes Service (AKS)
Content Source: articles/aks/servicemesh-istio-scenario-routing.md
Service: container-service
GitHub Login: @paulbouwer
Microsoft Alias: pabouwer

Thank you @JoshDaone for the feedback!
I've added an item to our backlog and we'll prioritize accordingly.
#please-close


Localhost not connecting running micro-service with docker.

Problem encountered on https://dotnet.microsoft.com/en-us/learn/aspnet/microservice-tutorial/run-docker
Operating System: macOS
MacOS version: 14.1.1 (23B81) on M2 air
I am having exact same version of micro-service and dockerfile provided in the guide. It is working perfect while running on local machine with downloaded SDK. But while running with docker it doesn't connects to localhost and I am getting following message on browser and terminal.

Follow on: same error still persist on blazor-webapp. Works completely fine on local machine SDK but have same error on running with docker.
I think it can be issue with the fact that M chips are arm64 based or some firewall issues. I was able to run spring boot docker image on my MacBook but none of the dotnet  docker is working.
Please help if anyone has encountered similar problem.

Follow on: I was able to run directly from docker hub with this command and it works completely fine.
docker run -it --rm -p 8000:8080 --name aspnetcore_sample mcr.microsoft.com/dotnet/samples:aspnetapp

I assume you are running on latest .NET 8? There was a breaking change in the default port used by the ASP.NET Core (from 80 to 8080), see
https://learn.microsoft.com/en-us/dotnet/core/compatibility/containers/8.0/aspnet-port. We didn't update the tutorials to reflect that. Instructions from https://github.com/dotnet/core/issues/8958 should apply here.
docker run -it --rm -p 3000:8080 --name mymicroservicecontainer mymicroservice

Thanks @rzikm, It is working now.

Glad it helped. I will close the issue, if you encounter other problems don't hesitate to open a new one.


WIP: adds AxisArray setindex! methods

Requires JuliaLang/julia#10331. But I think it is also running into a MAX_TUPLE_LEN issue, which is messing up my workaround for JuliaLang/julia#10191.
Needs tests, too.

Merged in 78e17ebb0f19519980b499f581448c7054240ae2 as I try to get AxisArrays back up to speed with 0.4.


Dockerfile and docker-compose added

Hard work to add Dockerfile and docker-compose yaml file to ensure we are able to run these amazing notebooks.
I've modified conda env config file a little bit, feel free to update it ;)
Dockerfile can be simplified, since I used a Dockerfile from jupyter  docker stack to create mine.
I've executed all cells from PyTorch without error. However it may be errors or missing packages.

@AnshulBasia Can you take a look and merge if all looks good? :)

@foxpowa  Thanks a lot. I was wondering if you could answer some questions:
1.) Is this docker file explicit for ubuntu or will it work for any unix based OS ?
2.) Could you also make another section in readme on how to use this docker file instead of a bunch of instructions already mentioned there ?

I think if it's docker it should work on any system supported by Docker, so that should be ok. Right @foxpowa ?
Also yeah, the edits in the readme with Docker support would be a great addition!

Hi folks,

It will work on any system that runs Docker, I've tested it on my mac and my ubuntu 16.04 linux computer.
Sure, I'll add some info to run it via docker-compose (basically, docker run command will be nicely stored in a yaml config file).


Boot logo cut in half

The boot logo is exactly cut in half by the upper part of the screen. The progress bar seems in place.
Here is an esplicative video
Thanks again!

Can you run these commands and tell me the results?
cat /sys/firmware/acpi/bgrt/yoffset

and
xrandr | grep '*' | grep -Po '(?<=x)(.*?)(?=\ )'

ofc, they are:
cat /sys/firmware/acpi/bgrt/yoffset 322
xrandr | grep '*' | grep -Po '(?<=x)(.*?)(?=\ )' 1800

okay, xrandr is giving a very high number.
can you run xrandr without anything in front of it and copy the whole output for me?

xrandr
Screen 0: minimum 320 x 200, current 3200 x 1800, maximum 8192 x 8192
eDP-1 connected primary 3200x1800+0+0 (normal left inverted right x axis y axis) 293mm x 165mm
   3200x1800     60.00*+  59.96    59.94    48.01
   2880x1620     59.96    59.97
   2560x1600     59.99    59.97
   2560x1440     59.99    59.99    59.96    59.95
   2048x1536     60.00
   1920x1440     60.00
   1856x1392     60.01
   1792x1344     60.01
   2048x1152     59.99    59.98    59.90    59.91
   1920x1200     59.88    59.95
   1920x1080     60.01    59.97    59.96    59.93
   1600x1200     60.00
   1680x1050     59.95    59.88
   1600x1024     60.17
   1400x1050     59.98
   1600x900      59.99    59.94    59.95    59.82
   1280x1024     60.02
   1440x900      59.89
   1400x900      59.96    59.88
   1280x960      60.00
   1440x810      60.00    59.97
   1368x768      59.88    59.85
   1360x768      59.80    59.96
   1280x800      59.99    59.97    59.81    59.91
   1152x864      60.00
   1280x720      60.00    59.99    59.86    59.74
   1024x768      60.04    60.00
   960x720       60.00
   928x696       60.05
   896x672       60.01
   1024x576      59.95    59.96    59.90    59.82
   960x600       59.93    60.00
   960x540       59.96    59.99    59.63    59.82
   800x600       60.00    60.32    56.25
   840x525       60.01    59.88
   864x486       59.92    59.57
   800x512       60.17
   700x525       59.98
   800x450       59.95    59.82
   640x512       60.02
   720x450       59.89
   700x450       59.96    59.88
   640x480       60.00    59.94
   720x405       59.51    58.99
   684x384       59.88    59.85
   680x384       59.80    59.96
   640x400       59.88    59.98
   576x432       60.06
   640x360       59.86    59.83    59.84    59.32
   512x384       60.00
   512x288       60.00    59.92
   480x270       59.63    59.82
   400x300       60.32    56.34
   432x243       59.92    59.57
   320x240       60.05
   360x202       59.51    59.13
   320x180       59.84    59.32
DP-1 disconnected (normal left inverted right x axis y axis)
HDMI-1 disconnected (normal left inverted right x axis y axis)
DP-2 disconnected (normal left inverted right x axis y axis)
HDMI-2 disconnected (normal left inverted right x axis y axis)

Apparently, my assumption is not generalizable and your EFI boot resolution is not the maximum resolution (ie. 3200x1800).
Can you run the command below?
cat /sys/class/graphics/fb0/virtual_size

$ cat /sys/class/graphics/fb0/virtual_size
3200,1800

Your assumption is reasonable but I run coreboot on a chromebook, it's definitely very uncommon.

Okay, I made a separate branch just for your case. Look at center-logo branch and install it.

Still not working properly, now the logo is not visible.
/usr/share/plymouth/themes/unified-bgrt/box.png this time is a box instead of a rectangle with the logo on top.
Also, oddly the /usr/share/plymouth/themes/unified-bgrt was now owned by root.
That said, the bar on it's own is already nice, so if I am bothering you, leave it like this.
Thank you vary much!

I am not being bothered. I would like to help with this and solve the problem.


chore: new devnet's addresses after upgrading PCSC to v6.1.0

Description
Add your description here, if it fixes a particular issue please provide a
link
to the issue.
Checklist

[x] Commit sequence broadly makes sense and commits have useful messages.
[x] New tests are added if needed and existing tests are updated.
[ ] Relevant logging and metrics added
[ ] CI passes. See note on CI.
[ ] Any changes are noted in the changelog.md for affected crate
[x] Self-reviewed the diff

Note on CI
If your PR is from a fork, the necessary CI jobs won't trigger automatically for security reasons.
You will need to get someone with write privileges.  Please contact IOG Partner Chains developers to do this
for you.

Please add some description


Write test for getHostState

This tests the getHostState method I refactored out from the existing ls command.
I also took the opportunity to move that method to the bottom of the file to follow the existing pattern of aux. functions in that file.
cc @ehazlett @bfirsh

@nathanleclaire go fmt

Fixed.  I think my editor's hook doesn't use -s which is annoying >_>

@ehazlett I changed to use a temp dir

LGTM


How to use it with trino after building it?

Steps mentioned to use it with trino are unclear. Please mention the steps to follow after successfully building it using ./mvnw clean install -DskipTests. What files are to be added to trino-server directory? Do we have to add storage.properties file in etc/catalog just like other connectors?

Do we have to add storage.properties file in etc/catalog just like other connectors?

Yes.


go-to-definition spawns new LSP server with root at definition

Description
When you use go-to-definition on a symbol that takes you outside of the current project (such as one defined in another crate in the Rust world), nvim-lspconfig will spawn a new LSP server (at least for rust-analyzer) with the current working directory set to the directory of the project that holds the symbol definition, rather than the current nvim working directory. This, in turn, causes issues when the working directory has implications for the LSP server's configuration (or the configurations of the tools it in turn invokes).
It's not entirely clear to me whether this a bug with neovim's lsp support, with nvim-lspconfig, with rust-analyzer, or with LSP more broadly, but I suspect it's a nvim-lspconfig issue as VSCode for example handles this correctly.
I also believe this is related to (but not quite the same as) https://github.com/neovim/nvim-lspconfig/issues/804.
I think the right solution here is to either find a way to continue using the same LSP server instance, or to spawn the new LSP server instances in :pwd, and then point it at the project directory for the definition. That way it'll still pick up the right configuration.
Neovim version
NVIM v0.7.2
Build type: Debug
LuaJIT 2.1.0-beta3
Compilation: /usr/bin/clang -DNVIM_TS_HAS_SET_MATCH_LIMIT -DNVIM_TS_HAS_SET_ALLOCATOR -g -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wdouble-promotion -Wmissing-noreturn -Wmissing-format-attribute -Wmissing-prototypes -Wimplicit-fallthrough -Wvla -fstack-protector-strong -fno-common -fdiagnostics-color=auto -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -DMIN_LOG_LEVEL=1

Features: +acl +iconv +tui
See ":help feature-compile"

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/usr/local/share/nvim"

Run :checkhealth for more info

Nvim-lspconfig version
d346335af862ede0cece75739d789cfdc46c4454
Operating system and version
Amazon Linux 2 (aarch64; 5.4 kernel)
Affected language servers
rust-analyzer, though likely also others
Steps to reproduce
For example, consider two Rust projects where one depends on the other, and both require nightly Rust, but only the consumer has a Rustup toolchain override file:
$ cargo new --lib needs-nightly
$ cargo new --lib consumer-of-needs-nightly
$ echo 'needs-nightly = { path = "../needs-nightly" }' >> consumer-of-needs-nightly/Cargo.toml
$ echo -e 'nightly' > consumer-of-needs-nightly/^C
$ echo '#![feature(allocator_api)]' > needs-nightly/src/lib.rs
$ echo 'pub struct Foo;' >> needs-nightly/src/lib.rs
$ echo 'use needs_nightly::Foo;' > consumer-of-needs-nightly/src/lib.rs
$ cd consumer-of-needs-nightly/ && rustup override set nightly
$ cargo check
...
    Finished dev [unoptimized + debuginfo] target(s) in 0.15s
$ nvim src/lib.rs

This is a pretty common scenario — consider if needs-nightly isn't actually a local project, but instead is depended on using a version number, and so its definition is actually somewhere in ~/.cargo/src/cache/whatever. The version that's there is unlikely to have a rust-toolchain.toml file.
Actual behavior
In the opened neovim session, all LSP features work just fine. However, if you "go to definition" on Foo in the use line, notice that the #![feature(allocator_api)] gets highlighted with the error
#![feature(allocator_api)]     _ `#![feature]` may not be used on the stable release channel

Despite :pwd having a rust-toolchain.toml that specifies nightly. There are many other ways for this to manifest, such as if the current directory has a .cargo/config.toml that defines other (required) rustflags, new Cargo registries, etc., that can even cause the LSP server to fail to start entirely.
(the relevant second LSP server launch is here)
Expected behavior
After go-to-definition, the configuration from :pwd should continue to apply, meaning rust-toolchain.toml should still be respected, so no errors should be shown in needs-nightly.
Minimal config
local on_windows = vim.loop.os_uname().version:match 'Windows'

local function join_paths(...)
  local path_sep = on_windows and '\\' or '/'
  local result = table.concat({ ... }, path_sep)
  return result
end

vim.cmd [[set runtimepath=$VIMRUNTIME]]

local temp_dir = vim.loop.os_getenv 'TEMP' or '/tmp'

vim.cmd('set packpath=' .. join_paths(temp_dir, 'nvim', 'site'))

local package_root = join_paths(temp_dir, 'nvim', 'site', 'pack')
local lspconfig_path = join_paths(package_root, 'test', 'start', 'nvim-lspconfig')

if vim.fn.isdirectory(lspconfig_path) ~= 1 then
  vim.fn.system { 'git', 'clone', 'https://github.com/neovim/nvim-lspconfig', lspconfig_path }
end

vim.lsp.set_log_level 'trace'
require('vim.lsp.log').set_format_func(vim.inspect)
local nvim_lsp = require 'lspconfig'
local on_attach = function(_, bufnr)
  local function buf_set_option(...)
    vim.api.nvim_buf_set_option(bufnr, ...)
  end

  buf_set_option('omnifunc', 'v:lua.vim.lsp.omnifunc')

  -- Mappings.
  local opts = { buffer = bufnr, noremap = true, silent = true }
  vim.keymap.set('n', 'gD', vim.lsp.buf.declaration, opts)
  vim.keymap.set('n', 'gd', vim.lsp.buf.definition, opts)
  vim.keymap.set('n', 'K', vim.lsp.buf.hover, opts)
  vim.keymap.set('n', 'gi', vim.lsp.buf.implementation, opts)
  vim.keymap.set('n', '<C-k>', vim.lsp.buf.signature_help, opts)
  vim.keymap.set('n', '<space>wa', vim.lsp.buf.add_workspace_folder, opts)
  vim.keymap.set('n', '<space>wr', vim.lsp.buf.remove_workspace_folder, opts)
  vim.keymap.set('n', '<space>wl', function()
    print(vim.inspect(vim.lsp.buf.list_workspace_folders()))
  end, opts)
  vim.keymap.set('n', '<space>D', vim.lsp.buf.type_definition, opts)
  vim.keymap.set('n', '<space>rn', vim.lsp.buf.rename, opts)
  vim.keymap.set('n', 'gr', vim.lsp.buf.references, opts)
  vim.keymap.set('n', '<space>e', vim.diagnostic.open_float, opts)
  vim.keymap.set('n', '[d', vim.diagnostic.goto_prev, opts)
  vim.keymap.set('n', ']d', vim.diagnostic.goto_next, opts)
  vim.keymap.set('n', '<space>q', vim.diagnostic.setloclist, opts)
end

-- Add the server that troubles you here
local name = 'rust_analyzer'
local cmd = { 'rust-analyzer' }
if not name then
  print 'You have not defined a server name, please edit minimal_init.lua'
end
if not nvim_lsp[name].document_config.default_config.cmd and not cmd then
  print [[You have not defined a server default cmd for a server
    that requires it please edit minimal_init.lua]]
end

nvim_lsp[name].setup {
  cmd = cmd,
  on_attach = on_attach,
}

print [[You can find your log at $HOME/.cache/nvim/lsp.log. Please paste in a github issue under a details tag as described in the issue template.]]

LSP log
https://gist.github.com/jonhoo/a4d400ee5d14e153904395e3eb12d497

Hello @jonhoo
This doesn't need nightly to triiger, you can use two crates that depend on each other and just jumping between them will spawn a new lsp server which is incorrect (vscode is the reference indeed)
The problem boils down to https://github.com/neovim/nvim-lspconfig/blob/master/lua/lspconfig/configs.lua#L227 and https://github.com/neovim/nvim-lspconfig/blob/master/lua/lspconfig/util.lua#L288 when we see a buffer that doesn't have a client attached to it, we spawn a new lsp server, the correct approach is to send 'textDocument/didOpen (which is vim.lsp_attach) instead
I think if this is all correct, the fix is to to detect here https://github.com/neovim/nvim-lspconfig/blob/master/lua/lspconfig/util.lua#L288 if there is already an active lsp of the same language we don't need to spawn a new one

The traceback for second start is like this (using vim.debug.traceback())
"stack traceback:
\t/usr/share/nvim/runtime/lua/vim/lsp/rpc.lua:660: in function 'start'
\t/usr/share/nvim/runtime/lua/vim/lsp.lua:1178: in function 'start_client'
\t.../pack/packer/start/nvim-lspconfig/lua/lspconfig/util.lua:288: in function 'add'
\t...ck/packer/start/nvim-lspconfig/lua/lspconfig/configs.lua:248: in function 'try_add'
\t...ck/packer/start/nvim-lspconfig/lua/lspconfig/configs.lua:76: in function <...ck/packer/start/nvim-lspconfig/lua/lspconfig/configs.lua:75>
\t[C]: in function 'nvim_cmd'
\t/usr/share/nvim/runtime/filetype.lua:23: in function </usr/share/nvim/runtime/filetype.lua:22>
\t[C]: in function 'nvim_buf_call'
\t/usr/share/nvim/runtime/filetype.lua:22: in function </usr/share/nvim/runtime/filetype.lua:11>
\t[C]: in function 'nvim_set_current_buf'
\t/usr/share/nvim/runtime/lua/vim/lsp/util.lua:1092: in function 'jump_to_location'
\t/usr/share/nvim/runtime/lua/vim/lsp/handlers.lua:359: in function 'handler'\n\t/usr/share/nvim/runtime/lua/vim/lsp.lua:1383: in function ''\n\tvim/_editor.lua: in function <vim/_editor.lua:0>"

I patched the code and indeed seems to fix the origianl issue, so for summery

this should become  spawn a new server if there is not already an active one for the same filetype https://github.com/neovim/nvim-lspconfig/blob/master/lua/lspconfig/configs.lua#L226
we can add a check here maybe, to check if clients have client for the same filetype  then just return that client_id https://github.com/neovim/nvim-lspconfig/blob/master/lua/lspconfig/util.lua#L252

I'm not sure exactly how to check and where for the filetype though

This might be related: https://github.com/neovim/nvim-lspconfig/issues/842.
Personally I think the design of root_dir is totally a mistake as it's out of spec and might result in confusion like this issue. And I use this workaround for setup of all the servers:
require('lspcofig')[name].setup {
   root_dir = function()
      return vim.loop.cwd()
   end
}

This will ensure that no multiple instances of the server will be spawned.

Just simply looked at the code of emacs lsp-bridge  lsp-mode and coc.nvim about the root dir, there are some problems in the design of the root dir of lspconfig. But I haven't thought of a good refactoring solution yet

look at the lsp initialize request we should send the rootPath and rootUri not the root dir

I removed the root_dir on the core. Re-made only the server initialization parameters. Use the current cwd. When the server starts successfully, add a unique identifier, which is generated by cwd. what will we get

The root dir does not meet the lsp standard
The root dir is only a basis for us to detect whether there is an existing client
Finding rootdir needs to use a lot of checks which is a bit wasteful of time.

Just for reference, here is how coc.nvim deals with workspace folders. It seems that at least coc.nvim also uses detection mechanism like root_dir, but much smarter to test cwd first.
The root cause, I think, should be the lack of support of workspace folders feature at core.

vscode handles multiple workspace folders by requiring user to define a special config file at root
sublime does the same

And neovim doesn't have such feature currently. Or more specifically, we do not have such a concept like "workspace" or "project" in editor. I'm not familiar with core and its roadmap, so I cannot dig on it further here. I always launch nvim at the root of the project, thus cwd is already the best approximation to "workspace" for me. I guess this is not the case for everyone, and directly using cwd should not be always safe.


Add andrew to contributors list

@ndrwum, I'm quite elated about your PR. I wanna evolve this project to addresses various problems faced by first-time contributors. I'd love to learn about your journey in open source community, the problems, pain points you had etc.
Could you explain how you felt when you went through the tutorial, made a PR and learned that I merged it?


Singleton does not work as expected on consumption plan

The following code runs with a queue trigger.
    [Singleton()]
    [FunctionName("ProcessWebSessionQueue")]
    public static async Task Run([QueueTrigger("websession-updates")]string myQueueItem, ILogger log)
    {
        log.LogInformation($"HELLO");
        await Task.Delay(5000);
        log.LogInformation($"GOODBYE");
    }

However in consumption plan it doesn't work as expected. The HELLO and GOODBYEs overlap as shown below. I only have one instance.
If this is expected behavior then you must document this more clearly. Very very unexpected and potentially devastating to data integrity.
2018-08-23T20:55:53.446 [Information] Executing 'ProcessWebSessionQueue' (Reason='New queue message detected on 'websession-updates'.', Id=5123da8d-04af-4b15-b332-89a4e224d3e0)
2018-08-23T20:55:53.458 [Information] HELLO
2018-08-23T20:55:53.419 [Information] Executing 'UpdateWebSessionQueue' (Reason='This function was programmatically called via the host APIs.', Id=19d9c020-4190-47e5-af2c-d865276a8faf)
2018-08-23T20:55:53.439 [Information] Executed 'UpdateWebSessionQueue' (Succeeded, Id=19d9c020-4190-47e5-af2c-d865276a8faf)
2018-08-23T20:55:53.446 [Information] Executing 'ProcessWebSessionQueue' (Reason='New queue message detected on 'websession-updates'.', Id=5123da8d-04af-4b15-b332-89a4e224d3e0)
2018-08-23T20:55:53.458 [Information] HELLO
2018-08-23T20:55:57.414 [Information] Executing 'UpdateWebSessionQueue' (Reason='This function was programmatically called via the host APIs.', Id=8b1f09ba-c686-4f30-a9d5-beff79167e64)
2018-08-23T20:55:57.438 [Information] Executed 'UpdateWebSessionQueue' (Succeeded, Id=8b1f09ba-c686-4f30-a9d5-beff79167e64)
2018-08-23T20:55:57.445 [Information] Executing 'ProcessWebSessionQueue' (Reason='New queue message detected on 'websession-updates'.', Id=8f2fb4f2-a525-4cf8-a88b-e6297795457e)
2018-08-23T20:55:57.414 [Information] Executing 'UpdateWebSessionQueue' (Reason='This function was programmatically called via the host APIs.', Id=8b1f09ba-c686-4f30-a9d5-beff79167e64)
2018-08-23T20:55:57.438 [Information] Executed 'UpdateWebSessionQueue' (Succeeded, Id=8b1f09ba-c686-4f30-a9d5-beff79167e64)
2018-08-23T20:55:57.445 [Information] Executing 'ProcessWebSessionQueue' (Reason='New queue message detected on 'websession-updates'.', Id=8f2fb4f2-a525-4cf8-a88b-e6297795457e)
2018-08-23T20:55:58.462 [Information] GOODBYE
2018-08-23T20:55:58.468 [Information] Executed 'ProcessWebSessionQueue' (Succeeded, Id=5123da8d-04af-4b15-b332-89a4e224d3e0)
2018-08-23T20:55:58.461 [Information] GOODBYE
2018-08-23T20:55:58.468 [Information] Executed 'ProcessWebSessionQueue' (Succeeded, Id=5123da8d-04af-4b15-b332-89a4e224d3e0)
2018-08-23T20:55:59.364 [Information] HELLO
2018-08-23T20:55:59.364 [Information] HELLO
2018-08-23T20:56:04.365 [Information] GOODBYE
2018-08-23T20:56:04.372 [Information] Executed 'ProcessWebSessionQueue' (Succeeded, Id=4040f930-c4c0-40bb-9850-f8db95d6038a)
2018-08-23T20:56:04.365 [Information] GOODBYE

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a4500b14-64b4-4d1b-aae3-f0fd4dde85f2
Version Independent ID: 8fb75741-37b9-b45f-c0ca-9948ddaf1c06
Content: How to use the Azure WebJobs SDK
Content Source: articles/app-service/webjobs-sdk-how-to.md
Service: app-service-web
GitHub Login: @ggailey777
Microsoft Alias: glenga

@simeyla Thanks for the feedback! We are currently investigating and will update you shortly.

@simeyla, Since you are executing this on consumption based plan, Please try SingletonMode.Listener setting, this is to ensure only a single instance of the function is running when the app scales out to multiple instances, this setting ensures only one instance acquires the lock and only one listener starts.
below is updated snap of your code with SingletonMode.Listener setting.
[Singleton(Mode = SingletonMode.Listener)]
[FunctionName("ProcessWebSessionQueue")]
public static async Task Run([QueueTrigger("websession-updates")]string myQueueItem, ILogger log)
{
    log.LogInformation($"HELLO");
    await Task.Delay(5000);
    log.LogInformation($"GOODBYE");
}

We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Unable to resolve f3586ed424d7bf5d94628332fbd0edebf1f5147f

Subject: Unable to resolve Git reference in SwiftLintPlugin
Hi Team,
I am encountering an issue with the SwiftLintPlugin in our project. Specifically, I am seeing the following error:
fatal: reference is not a tree: f3586ed424d7bf5d94628332fbd0edebf1f5147f (on 0.2.3)
This error occurs when I try to run xcodebuild  via Fastlane and it prevents me from resolving the package dependencies.
I have tried troubleshooting this issue on my end, but I have not been able to find a solution. I suspect that this issue is related to the Git reference that SwiftLintPlugin is using, as the error message mentions a specific reference.
Could you please investigate this issue and provide a solution? Please let me know if you need any additional information or if there are any further steps I can take to troubleshoot this issue.
Thank you for your time and assistance.

Best regards,
Karthik

Yes its still happening, i couldn't able to pull the plugin

@karthikAdaptavant maybe it's an issue with the swift-tools-version then. Is Swift 5.7 available on your CI runner?


CT-789 nest config under parameters

Jira: CT-789
KBC: N/A
Before asking for review make sure that:
Checklist

[x] New client method(s) has tests
[x] Apiary file is updated

Release

[x] I gave the PR a proper label:

major (BC break)
minor (new feature)
patch (backwards compatible fix)
no release (just test changes)


CivMC can't be logged into from Rosegold

World section height is set to 24
If we change the height to 18 https://github.com/grepsedawk/rosegold.cr/blob/main/src/rosegold/world/chunk.cr#L12 things work as expected

We need to parse the Dimensions field of the Join Game packet, particularly min_y and height.


⚠ AtmoBunker01 Server has degraded performance

In eaae3f1, AtmoBunker01 Server ($ATMOBUNKER01) experienced degraded performance:

HTTP code: 200
Response time: 8045 ms

Resolved: AtmoBunker01 Server performance has improved in 9f85bc5.


Local devstack hangs on get

Repro:
❯ go run . docker run ubuntu echo "hello world"
15:01:00.638 | INF bacalhau/utils.go:69 > Development client version, skipping version check
d7d4d23d-08ff-46f4-a695-f37647da67cc
❯ go run . list
 CREATION_TIME  ID        JOB                      STATE      RESULT
 15:01:00       d7d4d23d  Docker ubuntu echo h...  Published  /ipfs/QmW7TdjNEMzqmW...
❯ go run . get d7d4d23d
15:05:44.8 | INF bacalhau/get.go:66 > Fetching results of job 'd7d4d23d'...
15:05:52.874 | INF ipfs/downloader.go:76 > Found 1 result shards, downloading to temporary folder.

^Csignal: interrupt

Expected: Download files
Actual: Hangs forever

Actually, it does timeout... only after 600 seconds. Not sure if cutting that down to say 300 seconds makes any difference.
https://github.com/filecoin-project/bacalhau/blob/d99f401e256016ec7b65e44e5cb037c8ec6b008c/cmd/bacalhau/get.go#L42
And yes it's trying to use the 5001 port in devstack :(
13:31:31.357 | DBG ipfs/downloader.go:81 > Downloading result CID job-2c663534-7782-48dc-8bca-7b8070930f36-shard-0-host-QmVsJujnC2TdZGSC9cvp1mczM6457QGhdphAPcJULwQS7j 'Qmc98B7WyiJqaQ3bNro4zvosMBm67EQ5f6rkj53goXxyhA' to '/var/folders/jr/7z8x7_qn0x117gl7g8xhbv2m0000gn/T/bacalhau-ipfs-job-downloader1053426992/Qmc98B7WyiJqaQ3bNro4zvosMBm67EQ5f6rkj53goXxyhA'...

13:41:31.357 | ERR ipfs/downloader.go:92 > Timed out while downloading result.
Error: failed to get ipfs cid 'Qmc98B7WyiJqaQ3bNro4zvosMBm67EQ5f6rkj53goXxyhA': Post "http://<IP_ADDRESS>:5001/api/v0/files/stat?arg=%2Fipfs%2FQmc98B7WyiJqaQ3bNro4zvosMBm67EQ5f6rkj53goXxyhA": context deadline exceeded
13:41:31.488 | DBG ipfs/node.go:364 > node 'QmPtucdB5KEwhNJ8JPYpkRo2uoW2fWsfdXZKYP7vPatpz9' failed to serve ipfs api: accept tcp4 <IP_ADDRESS>:5001: use of closed network connection
exit status 1

It seems this failure occurs only after a first successful get run.

Fail with exit code 1 (when it timeouts):
❯ LOG_LEVEL=debug go run . get bebde00c --download-timeout-secs 10
10:34:23.84 | DBG system/environment.go:38 > BACALHAU_ENVIRONMENT is not set to a known value:
10:34:23.84 | DBG system/tracer.go:55 > error initializing http tracer: error creating honeycomb exporter: please ensure that "HONEYCOMB_KEY" has been set
10:34:23.84 | DBG system/tracer.go:56 > failed to initialize http tracer, falling back to debug tracer
10:34:23.864 | DBG system/config.go:182 > BACALHAU_DIR not set, using default of ~/.bacalhau
10:34:23.865 | INF bacalhau/get.go:74 > Fetching results of job 'bebde00c'...
10:34:23.869 | DBG ipfs/downloader.go:57 > Spinning up IPFS node...
10:34:24.729 | DBG ipfs/node.go:374 > IPFS node QmdEp6sS7LSo8AvGYWEkaAEFGPHPHxmfQ29rEnLKsCjArf has current peers: QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb, QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt, QmdEp6sS7LSo8AvGYWEkaAEFGPHPHxmfQ29rEnLKsCjArf, QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ, QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa
10:34:24.73 | DBG ipfs/node.go:375 > IPFS node QmdEp6sS7LSo8AvGYWEkaAEFGPHPHxmfQ29rEnLKsCjArf is connecting to new peers: [/ip4/<IP_ADDRESS>/tcp/1235/p2p/QmdZQ7ZbhnvWY1J12XYKGHApJ6aufKyLNSvf8jZBrBaAVL /ip4/<IP_ADDRESS>/tcp/1235/p2p/QmXaXu9N5GNetatsvwnTfQqNtSeKAD6uCmarbh3LMRYAcF /ip4/<IP_ADDRESS>/tcp/1235/p2p/QmYgxZiySj3MRkwLSL4X2MF5F9f2PMhAE3LV49XkfNL1o3]
10:34:25.33 | DBG ipfs/downloader.go:64 > Connecting client to new IPFS node...
10:34:25.33 | DBG ipfs/client.go:44 > Created IPFS client for node API address: /ip4/<IP_ADDRESS>/tcp/5001/p2p/QmdEp6sS7LSo8AvGYWEkaAEFGPHPHxmfQ29rEnLKsCjArf
10:34:25.332 | DBG ipfs/downloader.go:74 > Created download scratch folder: /var/folders/jr/7z8x7_qn0x117gl7g8xhbv2m0000gn/T/bacalhau-ipfs-job-downloader656313557
10:34:25.332 | INF ipfs/downloader.go:81 > Found 1 result shards, downloading to temporary folder.
10:34:25.332 | DBG ipfs/downloader.go:89 > Downloading result CID job-bebde00c-775d-48cd-bb8e-82d6fef6cc66-shard-0-host-QmTxnNJvjfcx8xbfFq4GgzMzyHYQFnxtMFof6vm6hXs6bQ 'Qmco4umWAcFBVmkpWxYrNU5eBabvLid566cuc2Ly5URNzU' to '/var/folders/jr/7z8x7_qn0x117gl7g8xhbv2m0000gn/T/bacalhau-ipfs-job-downloader656313557/Qmco4umWAcFBVmkpWxYrNU5eBabvLid566cuc2Ly5URNzU'...
10:34:35.337 | ERR ipfs/downloader.go:100 > Timed out while downloading result.
10:34:35.455 | DBG ipfs/node.go:364 > node 'QmdEp6sS7LSo8AvGYWEkaAEFGPHPHxmfQ29rEnLKsCjArf' failed to serve ipfs api: accept tcp4 <IP_ADDRESS>:5001: use of closed network connection
Error: failed to get ipfs cid 'Qmco4umWAcFBVmkpWxYrNU5eBabvLid566cuc2Ly5URNzU': Post "http://<IP_ADDRESS>:5001/api/v0/files/stat?arg=%2Fipfs%2FQmco4umWAcFBVmkpWxYrNU5eBabvLid566cuc2Ly5URNzU": context deadline exceeded

[...]

failed to get ipfs cid 'Qmco4umWAcFBVmkpWxYrNU5eBabvLid566cuc2Ly5URNzU': Post "http://<IP_ADDRESS>:5001/api/v0/files/stat?arg=%2Fipfs%2FQmco4umWAcFBVmkpWxYrNU5eBabvLid566cuc2Ly5URNzU": context deadline exceeded
exit status 1

Data is in IPFS:
❯ ipfs --api /ip4/<IP_ADDRESS>/tcp/56378 get Qmco4umWAcFBVmkpWxYrNU5eBabvLid566cuc2Ly5URNzU
Saving file(s) to Qmco4umWAcFBVmkpWxYrNU5eBabvLid566cuc2Ly5URNzU
 245 B / 245 B [=============================================================================================================] 100.00% 0s

Note: local IPFS nodes are running on ports 56378, 56386, 56392 while the get command is trying to use port 5001.

Fail with exit code 0 (when? unknown conditions):
❯ LOG_LEVEL=debug go run . get e55c4800 --download-timeout-secs 1000
10:27:03.399 | DBG system/environment.go:38 > BACALHAU_ENVIRONMENT is not set to a known value:
10:27:03.399 | DBG system/tracer.go:55 > error initializing http tracer: error creating honeycomb exporter: please ensure that "HONEYCOMB_KEY" has been set
10:27:03.399 | DBG system/tracer.go:56 > failed to initialize http tracer, falling back to debug tracer
10:27:03.425 | DBG system/config.go:182 > BACALHAU_DIR not set, using default of ~/.bacalhau
10:27:03.426 | INF bacalhau/get.go:74 > Fetching results of job 'e55c4800'...
10:27:03.432 | DBG ipfs/downloader.go:57 > Spinning up IPFS node...
10:27:04.335 | DBG ipfs/node.go:374 > IPFS node QmVzBRQqhmKnvXk28Dz3sUkkc3sWh9nX7vCY8bcRusLc8T has current peers: QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb, QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ, QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt, QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN, QmVzBRQqhmKnvXk28Dz3sUkkc3sWh9nX7vCY8bcRusLc8T, QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa
10:27:04.335 | DBG ipfs/node.go:375 > IPFS node QmVzBRQqhmKnvXk28Dz3sUkkc3sWh9nX7vCY8bcRusLc8T is connecting to new peers: [/ip4/<IP_ADDRESS>/tcp/1235/p2p/QmdZQ7ZbhnvWY1J12XYKGHApJ6aufKyLNSvf8jZBrBaAVL /ip4/<IP_ADDRESS>/tcp/1235/p2p/QmXaXu9N5GNetatsvwnTfQqNtSeKAD6uCmarbh3LMRYAcF /ip4/<IP_ADDRESS>/tcp/1235/p2p/QmYgxZiySj3MRkwLSL4X2MF5F9f2PMhAE3LV49XkfNL1o3]
10:27:04.939 | DBG ipfs/downloader.go:64 > Connecting client to new IPFS node...
10:27:04.939 | DBG ipfs/client.go:44 > Created IPFS client for node API address: /ip4/<IP_ADDRESS>/tcp/5001/p2p/QmVzBRQqhmKnvXk28Dz3sUkkc3sWh9nX7vCY8bcRusLc8T
10:27:04.94 | DBG ipfs/downloader.go:74 > Created download scratch folder: /var/folders/jr/7z8x7_qn0x117gl7g8xhbv2m0000gn/T/bacalhau-ipfs-job-downloader54141853
10:27:04.94 | INF ipfs/downloader.go:81 > Found 1 result shards, downloading to temporary folder.
10:27:04.94 | DBG ipfs/downloader.go:89 > Downloading result CID job-e55c4800-8c4c-474e-8725-f010c0ad8ae5-shard-0-host-Qmf9ho6F3jt8Te3zD9DtuJaH414s8rGqg5sd3tuvSED9rE 'QmW7TdjNEMzqmWxm5WPK1p6QCkeChxMLpvhLxyUW2wpjCf' to '/var/folders/jr/7z8x7_qn0x117gl7g8xhbv2m0000gn/T/bacalhau-ipfs-job-downloader54141853/QmW7TdjNEMzqmWxm5WPK1p6QCkeChxMLpvhLxyUW2wpjCf'...
10:27:06.01 | INF ipfs/downloader.go:117 > Combining shard from output volume 'outputs' to final location: '/Users/enricorotundo/winderresearch/ProtocolLabs/bacalhau'
10:27:06.186 | DBG ipfs/node.go:364 > node 'QmVzBRQqhmKnvXk28Dz3sUkkc3sWh9nX7vCY8bcRusLc8T' failed to serve ipfs api: accept tcp4 <IP_ADDRESS>:5001: use of closed network connection
❯ echo $?
0


"Fix all auto-fixable problems in the workspace" If you want to modifications to the function file that is not open, modify while maintaining a warning or error.

I believe the error will disappear after a while :thinking:
The fix all in workspace action is quite slow, I might want to add a new progress bar in the next version to show the progress of updating.

:tada: This issue has been resolved in version 2.1.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Let place-my-order-api go before done-serve

PMO's Promise shim doesn't like coming after can-zone.

Is there another issue open for fixing the actual/underlying issue?

Underlying issue is here: https://github.com/canjs/can-zone/issues/85

We need documented (for myself at least) ... why a promise polyfill that uses the underlying setTimeout and such doesn't work.
I remember ... Because it uses mutation observers and we don't monitor that.
Sent from my iPhone

On Aug 29, 2016, at 1:06 PM, Matthew Phillips<EMAIL_ADDRESS>wrote:
Underlying issue is here: canjs/can-zone#85
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


Add feature implementation to DG

Added module database feature implementation
Proposed more features and command implementations that can be added

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (d26e026) 80.79% compared to head (fee36f5) 80.79%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff            @@
##             master     #239   +/-   ##
=========================================
  Coverage     80.79%   80.79%
  Complexity      587      587
=========================================
  Files            78       78
  Lines          1755     1755
  Branches        178      178
=========================================
  Hits           1418     1418
  Misses          305      305
  Partials         32       32

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

LGTM 👍


Gambit message data relay

What's this PR do?
Relays message level data Twillio -> MoCo -> Blink -> Gambit Convers
How should this be manually tested?

yarn install
docker-compose up
yarn all-tests
yarn web
yarn worker gambit-message-data-relay

What are the relevant tickets?
Pivotal 150310810
Fixes #102

This is deployed to staging and ready for tests.


CommandResult.java:34: Implement succeed and failed...

The puzzle 1-f8cb5d0c from #1 has to be resolved:
https://github.com/shiruka/api/blob/e44cabfaa14f61e0de709b490fdf5f2aeadc2d89/src/main/java/net/shiruka/api/command/CommandResult.java#L34-L34
The puzzle was created by @portlek on 20-Jan-21.
Estimate: 15 minutes,  role: DEV.
If you have any technical questions, don't ask me, submit new tickets instead. The task will be "done" when the problem is fixed and the text of the puzzle is removed from the source code. Here is more about PDD and about me.

The puzzle 1-f8cb5d0c has disappeared from the source code, that's why I closed this issue.


[Feature]: support new embedding and preview models

Feature Request
New OpenAI embedding models have been added:
https://platform.openai.com/docs/models/embeddings
New GPT-4 and GPT 3.5 models are in preview now:
https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Thanks! You can use these with the "Override Model" setting, but I'll get them in for the next version too.


It emits a fatal error after added bootstrap by bower install bootstrap --save

It works well from I first use it. But It emits a fatal error as following.
Running "versionCopyBowerComponents" task Fatal error: Cannot read property 'name' of undefined
How can I fix it?

Hi @xiaoshao, did it start failing after you added a new bower component?  Try running the grunt task with -v (--verbose) and provide the output.  Thanks


[SR-93] Implement overlay for Glibc.open()

Previous ID
SR-93

Radar
None

Original Reporter
jpedrosa (JIRA User)

Type
Improvement

Additional Detail from JIRA

Votes
0

Component/s
Standard Library

Labels
Improvement, Linux, StarterBug

Assignee
None

Priority
Medium

md5: 5503e7dff6511ec3887463506f7df5c9

Issue Description:
The file at:
https://github.com/apple/swift/blob/8d9ef80304d7b36e13619ea50e6e76f3ec9221ba/stdlib/public/SDK/Darwin/Misc.mm
Enables a few functions for Darwin that don't seem to be available for Linux yet.
The problem seems to be related to variadic functions being harder to map to from Swift.
I was interested in the "open" function. I was actually interested in the "open64" function which I used from FFI in another language, but "open" may work OK too. Since the "open" function seems to be present in the Darwin version, it seems that people are interested in it.
In the absence of "open", it seems that people have tried to use "fopen" instead. But they take different parameters and different programming styles, resulting in perhaps noticeable differences.
Thanks.

is this moved to swift/stdlib/SwiftShims/swift/shims/LibcOverlayShims.h?


ref: clarify exp apply behavior

Noticed this issue while reviewing https://github.com/iterative/dvc.org/pull/2383
The docs for dvc exp apply currently state that:

Restores the results of a given experiment (name or hash are accepted, see dvc exp run for details) into the workspace. This means changing the appropriate metafiles, parameter files, metrics, plots, and corresponding DVC-tracked data.
This is similar to dvc checkout, but for dvc experiments.

exp apply will restore the changes to any (git or DVC tracked) file from a given experiment. It is not specific to DVC-related files and data. Rather than a simple dvc checkout, it's closer to:
git checkout <exp-sha>
dvc checkout

(specifically, it's more like a git stash apply or git merge --no-commit than git checkout, but typical DVC users are probably not familiar with those commands)
The docs should mention that exp apply is potentially destructive, and will overwrite any uncommitted changes to the user's (git) workspace, if they conflict with whatever changes exist in the experiment.

and will overwrite any uncommitted changes

will it drop changes that do not conflict?

@shcheklein no, any changes which don't conflict will be left alone. It basically just does a git merge/checkout with --theirs, meaning that if there are conflicts the version from the experiment is automatically preferred.
Originally, exp apply would error out on any conflicts and had a -f/--force option to do the --theirs overwrite automatically. But based on feedback from @dmpetrov, the typical use case for exp apply meant that users would just always need to run it with --force.

I would clarify this. It's important that changes are not wiped out but rather merged with the experiment. I'm a bit worried that pretty much silently user may end up with a mix of changes here.
Does it makes sense to go even further and do reset --hard  ?

important that changes are not wiped out but rather merged with the experiment. I'm a bit worried

Should we transfer to the core repo for now? I'll address part 1. for now (in #2417)

Merging/preserving the changes in an unrelated file is a useful feature IMO, especially with regard to the ability to create branches off of existing checkpoints. If I make a code change in one file and then want to test the changes against some other checkpoint branch, I can just exp apply the checkpoint and do exp run. To me this feels like a similar workflow to
<make changes in some code against one branch>
git checkout <branch> # checkout a different branch, those changes will be preserved and now applied to the newly checked out branch

With git checkout, it will preserve the changes unless there is a conflict w/the new branch - which is the same behavior we have with exp apply.

ability to create branches off of existing checkpoints

Wouldn't exp branch be a better/easier route for this? (Maybe less efficient)

With git checkout, it will preserve the changes unless there is a conflict

True, and git stash apply works similarly.
Up to you guys (but again, seems like a core issue/discussion). For now I have a Q on 1., @pmrowla :

exp apply is potentially destructive, and will overwrite any uncommitted changes to the user's (git) workspace

This seems to contradict with another https://github.com/iterative/dvc.org/pull/2417#discussion_r619906850:

It may also fail if there are git conflicts between changes in your workspace and in the experiment... We try to automatically resolve the conflicts by [git merge --theirs], but sometimes git may still fail to auto-resolve them.

A bit confused 🙃

Yes it is essentially git merge --theirs. git merge --theirs is destructive (it forcefully overwrites any of your conflicting changes)

git merge --theirs is destructive (it forcefully overwrites any of your conflicting changes)

it is dangerous but ok. inconsistency (mix of changes) might be even a bigger problem:

Does it makes sense to go even further and do reset --hard ?

It makes a lot of sense here.
@pmrowla what do you think about this even more aggressive (but consistent) strategy?

A suggestion/idea - run stash before doing hard reset, to make sure that nothing is completely lost and it can be reversed/merged.

A suggestion/idea - run stash before doing hard reset, to make sure that nothing is completely lost and it can be reversed/merged.

🤔 good idea.
However, it might confuse a user if he/she is an active stash user. Is there a way to reduce the confusion?

I'm still not convinced that doing the hard reset is what users would want here?
Let's say I've run some checkpoints and now I see that I want to change some params and re-run from a specific intermediate point. If I prefer editing my params files manually (vs using -S) I think I should be able to do
vim params.yaml # edit params
exp apply <intermediate-checkpoint>

in either order (which is consistent with being able to edit code and then use git checkout <different-branch> in either order). With reset --hard, I have to do the exp apply first or else I lose my changes.
We could consider doing git stash automatically, but if the user is unaware of this behavior or just forgets to always use stash drop/pop afterwards we will end up filling up their stash with a ton garbage entries very quickly.

ability to create branches off of existing checkpoints

Wouldn't exp branch be a better/easier route for this? (Maybe less efficient)

Also, @jorgeorpinel I missed this comment earlier. With regard to  ability to create branches off of existing checkpoints I wasn't talking about Git branches, I was talking about being able to make modifications to an intermediate checkpoint, and then start a new "checkpoint branch" off of that point which contains those modifications. Using exp branch here would forces users to create full Git branches for this kind of experimentation, which would defeats the purpose of the entire experiments feature in the first place.

OK! I'm still confused as to when exp apply can fail though. We currently say it may fail depending on Git being able to merge, but it should never fail if it overwrites conflicts with --theirs?
However, since the behavior may change soon, we can fix the docs once that is decided. Mind transferring this to the core repo, guys? Or please clarify what other doc updates are needed. Thanks!

Maybe I'm misremembering something, but I think there's still certain edge cases where using --theirs can fail and still require manually fixing conflicts (mostly involving binary files being tracked with git)?

We could consider doing git stash automatically, but if the user is unaware of this behavior or just forgets to always use stash drop/pop afterwards we will end up filling up their stash with a ton garbage entries very quickly.

@pmrowla you are right. it creates another layer of complexity. let's not do that.

but I think there's still certain edge cases where using --theirs can fail

I don't think git merge --ours/theirs can fail (unless there are file system issues or something else not worth considering). https://git-scm.com/docs/git-merge#Documentation/git-merge.txt-ours indicates that it's a forceful operation. I'll update the docs accordingly 👍 - a5de50ba

The docs should mention that exp apply is potentially destructive, and will overwrite any uncommitted changes to the user's (git) workspace, if they conflict with whatever changes exist in the experiment.

This is already indicated in https://dvc.org/doc/command-reference/exp/apply now so let's close the issue 👍


Is it possible to have the text under SpeedView

Is your feature request related to a problem? Please describe.
Currently, the SpeedView speed and units is between the start and end parts of the component. If I want to increase the font size for the speed, it would cover part of the speedview and that looks unprofessional. I was wondering if there's a way to put the speed and unit under the SpeedView.
Describe the solution you'd like
Either something similar to sv_unitUnderSpeedText where it's something like sv_SpeedTextUnderSv.
Another suggestion would be to not cut off the text when we add negative padding to the SpeedText.
Describe alternatives you've considered
I've tried adding negative padding to the SpeedText but that cuts off the text and I can't change the height because the SpeedView component is restricted to be rectangular.
Additional context
Add any other context or screenshots about the feature request here.

Hi @adabingw
If sv_speedTextPosition and sv_unitUnderSpeedText doesn't suit your requirement, you can use your own TextView to display your speed value. You can update the speed text by listening to speed change through Speed listeners.


failed to poll model : model not found in any model repository

Hi
I have a custom model of mine, which I am able to run successfully using tritonserver. It is a BLS kind of model. The main model is named pipe.
Docker command used to start the client/MA is as follows:
sudo docker run -it --gpus all -v /var/run/docker.sock:/var/run/docker.sock -v /home/ubuntu/tensor_rt/triton/client:/workspace/runs -v /home/ubuntu/tensor_rt/triton/output_models:/workspace/output_models -v /home/ubuntu/tensor_rt/triton/sd_test:/models/ --net host --name triton_client nvcr.io/nvidia/tritonserver:23.10-py3-sdk
Structure of my  /home/ubuntu/tensor_rt/triton/sd_test (on host)
|sd_test
|----aux_models
|--------meta
|----models
|--------model1
|--------model2
|--------model3
|--------model4
|--------pipe
Command used to start the model analzyer inside the container:
model-analyzer -v profile --model-repository /models/models/ --profile-models pipe --triton-launch-mode=docker --output-model-repository-path output_models/trial_1 --export-path runs/profile_results --override-output-model-repository --triton-docker-image triton_image --triton-output-path triton_log.txt --triton-docker-mounts "/models:/models:rw,/workspace/runs/profile_results:/profile_results:rw"
Output of the MA:
07:46:50 [Model Analyzer] DEBUG:
{'always_report_gpu_metrics': False,
 'batch_sizes': [1],
 'bls_composing_models': [],
 'checkpoint_directory': '/workspace/checkpoints',
 'client_max_retries': 50,
 'client_protocol': 'grpc',
 'collect_cpu_metrics': False,
 'concurrency': [],
 'config_file': None,
 'constraints': {},
 'cpu_only_composing_models': [],
 'duration_seconds': 3,
 'early_exit_enable': False,
 'export_path': 'runs/profile_results',
 'filename_model_gpu': 'metrics-model-gpu.csv',
 'filename_model_inference': 'metrics-model-inference.csv',
 'filename_server_only': 'metrics-server-only.csv',
 'gpu_output_fields': ['model_name',
                       'gpu_uuid',
                       'batch_size',
                       'concurrency',
                       'model_config_path',
                       'instance_group',
                       'satisfies_constraints',
                       'gpu_used_memory',
                       'gpu_utilization',
                       'gpu_power_usage'],
 'gpus': ['all'],
 'inference_output_fields': ['model_name',
                             'batch_size',
                             'concurrency',
                             'model_config_path',
                             'instance_group',
                             'max_batch_size',
                             'satisfies_constraints',
                             'perf_throughput',
                             'perf_latency_p99'],
 'latency_budget': None,
 'min_throughput': None,
 'model_repository': '/models/models/',
 'monitoring_interval': 1.0,
 'num_configs_per_model': 3,
 'num_top_model_configs': 0,
 'objectives': {'perf_throughput': 10},
 'output_model_repository_path': 'output_models/trial_1',
 'override_output_model_repository': True,
 'perf_analyzer_cpu_util': 1280.0,
 'perf_analyzer_flags': {},
 'perf_analyzer_max_auto_adjusts': 10,
 'perf_analyzer_path': 'perf_analyzer',
 'perf_analyzer_timeout': 600,
 'perf_output': False,
 'perf_output_path': None,
 'plots': [{'name': 'throughput_v_latency', 'title': 'Throughput vs. Latency', 'x_axis': 'perf_latency_p99', 'y_axis': 'perf_throughput', 'monotonic': True},
           {'name': 'gpu_mem_v_latency', 'title': 'GPU Memory vs. Latency', 'x_axis': 'perf_latency_p99', 'y_axis': 'gpu_used_memory', 'monotonic': False}],
 'profile_models': [{'model_name': 'pipe', 'cpu_only': False, 'objectives': {'perf_throughput': 10}, 'parameters': {'batch_sizes': [1], 'concurrency': [], 'request_rate': []}, 'weighting': 1}],
 'reload_model_disable': False,
 'request_rate': [],
 'request_rate_search_enable': False,
 'run_config_profile_models_concurrently_enable': False,
 'run_config_search_disable': False,
 'run_config_search_max_binary_search_steps': 5,
 'run_config_search_max_concurrency': 1024,
 'run_config_search_max_instance_count': 5,
 'run_config_search_max_model_batch_size': 128,
 'run_config_search_max_request_rate': 8192,
 'run_config_search_min_concurrency': 1,
 'run_config_search_min_instance_count': 1,
 'run_config_search_min_model_batch_size': 1,
 'run_config_search_min_request_rate': 16,
 'run_config_search_mode': 'brute',
 'server_output_fields': ['model_name',
                          'gpu_uuid',
                          'gpu_used_memory',
                          'gpu_utilization',
                          'gpu_power_usage'],
 'skip_detailed_reports': False,
 'skip_summary_reports': False,
 'triton_docker_args': {},
 'triton_docker_image': 'triton_image',
 'triton_docker_labels': {},
 'triton_docker_mounts': ['/models:/models:rw',
                          '/workspace/runs/profile_results:/profile_results:rw'],
 'triton_docker_shm_size': None,
 'triton_grpc_endpoint': 'localhost:8001',
 'triton_http_endpoint': 'localhost:8000',
 'triton_install_path': '/opt/tritonserver',
 'triton_launch_mode': 'docker',
 'triton_metrics_url': 'http://localhost:8002/metrics',
 'triton_output_path': 'triton_log.txt',
 'triton_server_environment': {},
 'triton_server_flags': {},
 'triton_server_path': 'tritonserver',
 'weighting': None}
07:46:50 [Model Analyzer] Initializing GPUDevice handles
07:46:51 [Model Analyzer] Using GPU 0 NVIDIA A10G with UUID GPU-53c85dc1-936c-a3e4-765c-60657311f1d0
07:46:51 [Model Analyzer] WARNING: Overriding the output model repo path "output_models/trial_1"
07:46:51 [Model Analyzer] Starting a Triton Server using docker
07:46:51 [Model Analyzer] Loaded checkpoint from file /workspace/checkpoints/1.ckpt
07:46:51 [Model Analyzer] GPU devices match checkpoint - skipping server metric acquisition
07:46:51 [Model Analyzer]
07:46:51 [Model Analyzer] Starting automatic brute search
07:46:51 [Model Analyzer]
07:46:51 [Model Analyzer] Creating model config: pipe_config_default
07:46:51 [Model Analyzer]
07:47:00 [Model Analyzer] DEBUG: Triton Server started.
07:47:08 [Model Analyzer] Model pipe_config_default load failed: [StatusCode.INTERNAL] failed to load 'pipe', failed to poll from model repository
07:47:12 [Model Analyzer] DEBUG: Stopped Triton Server.
07:47:12 [Model Analyzer] Saved checkpoint to /workspace/checkpoints/2.ckpt
07:47:12 [Model Analyzer] Creating model config: pipe_config_0
07:47:12 [Model Analyzer]   Setting instance_group to [{'count': 1, 'kind': 'KIND_GPU'}]
07:47:12 [Model Analyzer]   Setting max_batch_size to 1
07:47:12 [Model Analyzer]   Enabling dynamic_batching
07:47:12 [Model Analyzer]
07:47:13 [Model Analyzer] DEBUG: Triton Server started.
07:47:18 [Model Analyzer] Model pipe_config_0 load failed: [StatusCode.INTERNAL] failed to load 'pipe', failed to poll from model repository
07:47:22 [Model Analyzer] DEBUG: Stopped Triton Server.
07:47:22 [Model Analyzer] No changes made to analyzer data, no checkpoint saved.
Traceback (most recent call last):
  File "/usr/local/bin/model-analyzer", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/dist-packages/model_analyzer/entrypoint.py", line 278, in main
    analyzer.profile(
  File "/usr/local/lib/python3.10/dist-packages/model_analyzer/analyzer.py", line 124, in profile
    self._profile_models()
  File "/usr/local/lib/python3.10/dist-packages/model_analyzer/analyzer.py", line 242, in _profile_models
    self._model_manager.run_models(models=[model])
  File "/usr/local/lib/python3.10/dist-packages/model_analyzer/model_manager.py", line 145, in run_models
    self._stop_ma_if_no_valid_measurement_threshold_reached()
  File "/usr/local/lib/python3.10/dist-packages/model_analyzer/model_manager.py", line 239, in _stop_ma_if_no_valid_measurement_threshold_reached
    raise TritonModelAnalyzerException(
model_analyzer.model_analyzer_exceptions.TritonModelAnalyzerException: The first 2 attempts to acquire measurements have failed. Please examine the Tritonserver/PA error logs to determine what has gone wrong.

Output of the tritonserver log in triton_log.txt

=============================
== Triton Inference Server ==
=============================

NVIDIA Release 23.10 (build 72127154)
Triton Server Version 2.39.0

Copyright (c) 2018-2023, NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

WARNING: CUDA Minor Version Compatibility mode ENABLED.
  Using driver version 530.30.02 which has support for CUDA 12.1.  This container
  was built with CUDA 12.2 and will be run in Minor Version Compatibility mode.
  CUDA Forward Compatibility is preferred over Minor Version Compatibility for use
  with this container but was unavailable:
  [[System has unsupported display driver / cuda driver combination (CUDA_ERROR_SYSTEM_DRIVER_MISMATCH) cuInit()=803]]
  See https://docs.nvidia.com/deploy/cuda-compatibility/ for details.

Warning: '--strict-model-config' has been deprecated! Please use '--disable-auto-complete-config' instead.
I1107 07:33:56.080411 7 pinned_memory_manager.cc:241] Pinned memory pool is created at '0x7f8a18000000' with size 268435456
I1107 07:33:56.083827 7 cuda_memory_manager.cc:107] CUDA memory pool is created on device 0 with size 67108864
I1107 07:33:56.087180 7 server.cc:592]
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+

I1107 07:33:56.087203 7 server.cc:619]
+---------+------+--------+
| Backend | Path | Config |
+---------+------+--------+
+---------+------+--------+

I1107 07:33:56.087212 7 server.cc:662]
+-------+---------+--------+
| Model | Version | Status |
+-------+---------+--------+
+-------+---------+--------+

I1107 07:33:56.126179 7 metrics.cc:817] Collecting metrics for GPU 0: NVIDIA A10G
I1107 07:33:56.126464 7 metrics.cc:710] Collecting CPU metrics
I1107 07:33:56.126625 7 tritonserver.cc:2458]
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Option                           | Value                                                                                |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| server_id                        | triton                                                                                |
| server_version                   | 2.39.0                                                                                |
| server_extensions                | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data parameters statistics trace logging |
| model_repository_path[0]         | /models/models                                                                                |
| model_control_mode               | MODE_EXPLICIT                                                                                |
| strict_model_config              | 0                                                                                |
| rate_limit                       | OFF                                                                                |
| pinned_memory_pool_byte_size     | 268435456                                                                                |
| cuda_memory_pool_byte_size{0}    | 67108864                                                                                |
| min_supported_compute_capability | 6.0                                                                                |
| strict_readiness                 | 1                                                                                |
| exit_timeout                     | 30                                                                                |
| cache_enabled                    | 0                                                                                |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

I1107 07:33:56.127801 7 grpc_server.cc:2513] Started GRPCInferenceService at <IP_ADDRESS>:8001
I1107 07:33:56.128028 7 http_server.cc:4497] Started HTTPService at <IP_ADDRESS>:8000
I1107 07:33:56.168973 7 http_server.cc:270] Started Metrics Service at <IP_ADDRESS>:8002
E1107 07:34:03.823179 7 model_repository_manager.cc:1265] failed to poll model 'pipe': model not found in any model repository.

=============================
== Triton Inference Server ==
=============================

NVIDIA Release 23.10 (build 72127154)
Triton Server Version 2.39.0

Copyright (c) 2018-2023, NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

WARNING: CUDA Minor Version Compatibility mode ENABLED.
  Using driver version 530.30.02 which has support for CUDA 12.1.  This container
  was built with CUDA 12.2 and will be run in Minor Version Compatibility mode.
  CUDA Forward Compatibility is preferred over Minor Version Compatibility for use
  with this container but was unavailable:
  [[System has unsupported display driver / cuda driver combination (CUDA_ERROR_SYSTEM_DRIVER_MISMATCH) cuInit()=803]]
  See https://docs.nvidia.com/deploy/cuda-compatibility/ for details.

Warning: '--strict-model-config' has been deprecated! Please use '--disable-auto-complete-config' instead.
I1107 07:34:09.004050 7 pinned_memory_manager.cc:241] Pinned memory pool is created at '0x7f31d0000000' with size 268435456
I1107 07:34:09.007313 7 cuda_memory_manager.cc:107] CUDA memory pool is created on device 0 with size 67108864
I1107 07:34:09.010657 7 server.cc:592]
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+

I1107 07:34:09.010679 7 server.cc:619]
+---------+------+--------+
| Backend | Path | Config |
+---------+------+--------+
+---------+------+--------+

I1107 07:34:09.010688 7 server.cc:662]
+-------+---------+--------+
| Model | Version | Status |
+-------+---------+--------+
+-------+---------+--------+

I1107 07:34:09.049361 7 metrics.cc:817] Collecting metrics for GPU 0: NVIDIA A10G
I1107 07:34:09.049666 7 metrics.cc:710] Collecting CPU metrics
I1107 07:34:09.049846 7 tritonserver.cc:2458]
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Option                           | Value                                                                                |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| server_id                        | triton                                                                                |
| server_version                   | 2.39.0                                                                                |
| server_extensions                | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data parameters statistics trace logging |
| model_repository_path[0]         | /models/models                                                                                |
| model_control_mode               | MODE_EXPLICIT                                                                                |
| strict_model_config              | 0                                                                                |
| rate_limit                       | OFF                                                                                |
| pinned_memory_pool_byte_size     | 268435456                                                                                |
| cuda_memory_pool_byte_size{0}    | 67108864                                                                                |
| min_supported_compute_capability | 6.0                                                                                |
| strict_readiness                 | 1                                                                                |
| exit_timeout                     | 30                                                                                |
| cache_enabled                    | 0                                                                                |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

I1107 07:34:09.051480 7 grpc_server.cc:2513] Started GRPCInferenceService at <IP_ADDRESS>:8001
I1107 07:34:09.051715 7 http_server.cc:4497] Started HTTPService at <IP_ADDRESS>:8000
I1107 07:34:09.092854 7 http_server.cc:270] Started Metrics Service at <IP_ADDRESS>:8002
E1107 07:34:13.742246 7 model_repository_manager.cc:1265] failed to poll model 'pipe': model not found in any model repository.

=============================
== Triton Inference Server ==
=============================

NVIDIA Release 23.10 (build 72127154)
Triton Server Version 2.39.0

Copyright (c) 2018-2023, NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

WARNING: CUDA Minor Version Compatibility mode ENABLED.
  Using driver version 530.30.02 which has support for CUDA 12.1.  This container
  was built with CUDA 12.2 and will be run in Minor Version Compatibility mode.
  CUDA Forward Compatibility is preferred over Minor Version Compatibility for use
  with this container but was unavailable:
  [[System has unsupported display driver / cuda driver combination (CUDA_ERROR_SYSTEM_DRIVER_MISMATCH) cuInit()=803]]
  See https://docs.nvidia.com/deploy/cuda-compatibility/ for details.

Warning: '--strict-model-config' has been deprecated! Please use '--disable-auto-complete-config' instead.
I1107 07:47:00.596353 7 pinned_memory_manager.cc:241] Pinned memory pool is created at '0x7faa78000000' with size 268435456
I1107 07:47:00.599650 7 cuda_memory_manager.cc:107] CUDA memory pool is created on device 0 with size 67108864
I1107 07:47:00.603063 7 server.cc:592]
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+

I1107 07:47:00.603086 7 server.cc:619]
+---------+------+--------+
| Backend | Path | Config |
+---------+------+--------+
+---------+------+--------+

I1107 07:47:00.603095 7 server.cc:662]
+-------+---------+--------+
| Model | Version | Status |
+-------+---------+--------+
+-------+---------+--------+

I1107 07:47:00.641803 7 metrics.cc:817] Collecting metrics for GPU 0: NVIDIA A10G
I1107 07:47:00.642091 7 metrics.cc:710] Collecting CPU metrics
I1107 07:47:00.642261 7 tritonserver.cc:2458]
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Option                           | Value                                                                                |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| server_id                        | triton                                                                                |
| server_version                   | 2.39.0                                                                                |
| server_extensions                | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data parameters statistics trace logging |
| model_repository_path[0]         | /models/models                                                                                |
| model_control_mode               | MODE_EXPLICIT                                                                                |
| strict_model_config              | 0                                                                                |
| rate_limit                       | OFF                                                                                |
| pinned_memory_pool_byte_size     | 268435456                                                                                |
| cuda_memory_pool_byte_size{0}    | 67108864                                                                                |
| min_supported_compute_capability | 6.0                                                                                |
| strict_readiness                 | 1                                                                                |
| exit_timeout                     | 30                                                                                |
| cache_enabled                    | 0                                                                                |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

I1107 07:47:00.643902 7 grpc_server.cc:2513] Started GRPCInferenceService at <IP_ADDRESS>:8001
I1107 07:47:00.644138 7 http_server.cc:4497] Started HTTPService at <IP_ADDRESS>:8000
I1107 07:47:00.685100 7 http_server.cc:270] Started Metrics Service at <IP_ADDRESS>:8002
E1107 07:47:08.340650 7 model_repository_manager.cc:1265] failed to poll model 'pipe': model not found in any model repository.

=============================
== Triton Inference Server ==
=============================

NVIDIA Release 23.10 (build 72127154)
Triton Server Version 2.39.0

Copyright (c) 2018-2023, NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

WARNING: CUDA Minor Version Compatibility mode ENABLED.
  Using driver version 530.30.02 which has support for CUDA 12.1.  This container
  was built with CUDA 12.2 and will be run in Minor Version Compatibility mode.
  CUDA Forward Compatibility is preferred over Minor Version Compatibility for use
  with this container but was unavailable:
  [[System has unsupported display driver / cuda driver combination (CUDA_ERROR_SYSTEM_DRIVER_MISMATCH) cuInit()=803]]
  See https://docs.nvidia.com/deploy/cuda-compatibility/ for details.

Warning: '--strict-model-config' has been deprecated! Please use '--disable-auto-complete-config' instead.
I1107 07:47:13.523220 7 pinned_memory_manager.cc:241] Pinned memory pool is created at '0x7f85f0000000' with size 268435456
I1107 07:47:13.526703 7 cuda_memory_manager.cc:107] CUDA memory pool is created on device 0 with size 67108864
I1107 07:47:13.530155 7 server.cc:592]
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+

I1107 07:47:13.530177 7 server.cc:619]
+---------+------+--------+
| Backend | Path | Config |
+---------+------+--------+
+---------+------+--------+

I1107 07:47:13.530186 7 server.cc:662]
+-------+---------+--------+
| Model | Version | Status |
+-------+---------+--------+
+-------+---------+--------+

I1107 07:47:13.568680 7 metrics.cc:817] Collecting metrics for GPU 0: NVIDIA A10G
I1107 07:47:13.568988 7 metrics.cc:710] Collecting CPU metrics
I1107 07:47:13.569186 7 tritonserver.cc:2458]
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Option                           | Value                                                                                |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| server_id                        | triton                                                                                |
| server_version                   | 2.39.0                                                                                |
| server_extensions                | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data parameters statistics trace logging |
| model_repository_path[0]         | /models/models                                                                                |
| model_control_mode               | MODE_EXPLICIT                                                                                |
| strict_model_config              | 0                                                                                |
| rate_limit                       | OFF                                                                                |
| pinned_memory_pool_byte_size     | 268435456                                                                                |
| cuda_memory_pool_byte_size{0}    | 67108864                                                                                |
| min_supported_compute_capability | 6.0                                                                                |
| strict_readiness                 | 1                                                                                |
| exit_timeout                     | 30                                                                                |
| cache_enabled                    | 0                                                                                |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

I1107 07:47:13.570460 7 grpc_server.cc:2513] Started GRPCInferenceService at <IP_ADDRESS>:8001
I1107 07:47:13.570671 7 http_server.cc:4497] Started HTTPService at <IP_ADDRESS>:8000
I1107 07:47:13.611722 7 http_server.cc:270] Started Metrics Service at <IP_ADDRESS>:8002
E1107 07:47:18.258678 7 model_repository_manager.cc:1265] failed to poll model 'pipe': model not found in any model repository.

I think the docker volume mounts are not happening inside the container launched by the MA. Not very sure of what is happening in this.

The issue is that the absolute path to the model and output model repository are not identical, which is required when running in docker mode. See our installation documentation for further details.

I am having some trouble understanding exactly which paths and volumes need to be mounted.
Trying to follow your advice I have tried these commands:
Starting the model analyzer container with:
sudo docker run -it --gpus all -v /var/run/docker.sock:/var/run/docker.sock -v /home/ubuntu/tensor_rt/triton/client:/workspace/runs -v /home/ubuntu/tensor_rt/triton/output_models:/home/ubuntu/tensor_rt/triton/output_models -v /home/ubuntu/tensor_rt/triton/sd_test:/home/ubuntu/tensor_rt/triton/sd_test/ --net host --name triton_client nvcr.io/nvidia/tritonserver:23.10-py3-sdk
Running the analyzer inside the container:
model-analyzer profile     --model-repository /home/ubuntu/tensor_rt/triton/sd_test/models     --profile-models pipe --triton-launch-mode=docker     --output-model-repository-path /home/ubuntu/tensor_rt/triton/output_models/trial_1     --export-path profile_results --override-output-model-repository --triton-docker-image triton_image --triton-output-path triton_log.txt
I have tried to keep the paths the same. But I still get the same error as before. The MA is able to start the triton server, but it is unable to find the models and their paths.
When a sibling container container is launched by MA, which volumes and from where do they get mounted?

I'm not sure. Can you try launching in local or remote mode and see if that solves your issue?

Hi.. I think the issue was that this was a bls model and it was not loading the underlying models required. I changed a few things and made it an ensemble model and it works fine now.
Also can I not perform multi model concurrent analyzing when for an ensemble model?

That is correct. Model Analyzer does not currently support multi-model concurrent analysis with one or more ensemble models.

Closing this issue


[FT] Add System Prompt field in LightevalTaskConfig that can be used by model clients

Issue encountered
My models currently don't follow the template I give. I want to give a system prompt that nudges the models to provide output the way I want it.
Solution/Feature
We could add a new field in LightevalTaskConfig that can be consumed by model clients like LiteLLMClient to send to the API.
Possible alternatives
Alternatively, we could use the generation_grammar, but I think this may be a bit overkill and may be more difficult to implement for different API providers.

@clefourrier Happy to open a PR for this. Please let me know if you see a different solution.

Hi! I'm adding auto-scaling for inference endpoints atm, and will edit the generation mechanisms after that, possibly to get a single homogeneous system - if you need it super fast feel free to add it before, but I'll probably edit the system next week :)

Ok, I see. Thanks for letting me know. I will use a hack in the meantime in that case.

Just briefly asking to double check: This edit to the generation mechanism will also affect the openai/litellm model loaders?

Yes the aim is to have one better centralised system

Need to add to #428

Hi @JoelNiklaus , from what I can see, it's already available for all models with the latest CLI refacto: there's a system prompt param, which is then used when creating the requests from the samples. Do you need stg else wrt this?

I saw that, thanks. However, to me it is still not clear how to use the system prompt in the model_loader (e.g., litellm). The system prompt is not part of GreedyUntilRequest. Also it does not come through with the model_config.

Because you should not use it in the model_loader ^^
The system prompt is added to the request by the PromptManager, which creates the request from the sample, number of few shots, and system prompt if needed. (It's in ligtheval/tasks/prompt_manager)

The system prompt will appear in the details if you want to check this

Haha ok. Sorry if a stupid question, but how can I access the system prompt before I make a call to the API?
https://github.com/huggingface/lighteval/blob/main/src/lighteval/models/openai_model.py#L82

As of now there is no way to only grab the system prompt. One solution would be to add a system_prompt field to the requests when creating them in the PromptManager so that you can access them in litellm_model


[Docs] v0.5.0 Release: Meshery Extensibility

Current State:
Meshery Extensibility has been significantly expanded upon under v0.5.0 and will be included in the Meshery v0.5.0 Release Announcement
Desired State:
Documentation of this feature and its behavior is needed. The vast majority of this the Extension Point framework is already documented. Go Plugins need documented. GraphQL Resolvers.

Contributor Resources

Meshery documentation site
Meshery documentation source
Instructions for contributing to documentation

Done!


Deleting library videos leads to complete program failure

I deleted some videos that were in the Voracious Library and the program was rendered completely inoperable, giving the following error on program start:
Something has gone terribly wrong
Error: ENOENT: no such file or directory, scandir '(my video path)'
Also, a panel on the right should have opened with more debug info. Maybe take a screenshot of everything and send it to me.

In order to get it working again I had to go hunting for the preferences and ended up deleting both cookie files, as well as the preferences and voracious.db files in my appdata folder. A reinstall did not fix, because, I assume, whichever of those files that was causing the issue was still there.

Thanks for the report.


Hyprshade doesn't work (can't connect to "/tmp/hypr/*/.socket.sock.)

Hyprland Version
 System/Version info
Hyprland, built from branch  at commit 4cdddcfe466cb21db81af0ac39e51cc15f574da9  ().
Date: 2024-05-14
Tag: , commits:

flags: (if any)

System Information:
System name: Linux
Node name: aegis-r
Release: 6.6.30
Version: #1-NixOS SMP PREEMPT_DYNAMIC Thu May  2 14:32:50 UTC 2024

GPU information:
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU106 [GeForce RTX 2070] [10de:1f02] (rev a1) (prog-if 00 [VGA controller])

os-release: ANSI_COLOR="1;34"
BUG_REPORT_URL="https://github.com/NixOS/nixpkgs/issues"
BUILD_ID="24.05.20240512.2057814"
DOCUMENTATION_URL="https://nixos.org/learn.html"
HOME_URL="https://nixos.org/"
ID=nixos
IMAGE_ID=""
IMAGE_VERSION=""
LOGO="nix-snowflake"
NAME=NixOS
PRETTY_NAME="NixOS 24.05 (Uakari)"
SUPPORT_URL="https://nixos.org/community.html"
VERSION="24.05 (Uakari)"
VERSION_CODENAME=uakari
VERSION_ID="24.05"

plugins:
  split-monitor-workspaces by Duckonaut ver 1.1.0
  hyprtrails by Vaxry ver 1.0
  hyprfocus by Vortex ver 2.0

Bug or Regression?
Bug
Description
I run "hyprshade on {path_to_some_shader}", it doesn't enable the shader. Commands to get more info (e.g. "hyprshade ls", "hyprshade current") complain that they're getting invalid JSON from some call to hyprctl
How to reproduce
hyprshade ls
Crash reports, logs, images, videos
No response

hyprshade is not an official hyprland tool

I can understand the confusion due to the naming convention. I'll add a note to the README.


Run() fails when no schedule is added

Hi, I am receiving an error when calling Run() before any job has been scheduled / added. See the example below.
As soon as at least one schedule is added before calling Run() it all works.
More about my use case:
A developer in my software should be able to schedule a new cron at any time.
My approach: Starting the scheduler without any cron and allowing them to be added at any time. To do so, I have a small wrapper around Add() that stops and restarts the scheduler, if is was already running.
Desired behavior: the scheduler can start without error even, if nothing is added yet.
Error:  could not schedule jobs: could not build query to get the list of already scheduled jobs to clean them up: empty slice passed to 'in' query
func TestGueron(t *testing.T) {
	poolAdapter := pgxv5.NewConnPool(pgHandler.PGx())

	s, err := gueron.NewScheduler(poolAdapter)
	assert.NoError(t, err)

	wm := gue.WorkMap{}

	//s.MustAdd("@every 15m", "log-foo-bar", nil) // <- works if this line is uncommented
	wm["log-foo-bar"] = func(ctx context.Context, j *gue.Job) error {
		fmt.Printf("Working scheduled job: %d\n", j.ID)
		return nil
	}

	ctx, _ := context.WithCancel(context.Background())
	if err := s.Run(ctx, wm, 4); err != nil {
		assert.NoError(t, err)
	}
}

Version v2.2.0

pls check the version from the master solves your problem - made small adjustment at https://github.com/vgarvardt/gueron/pull/90

Yes it does. Works perfect for my situation now!
Thank you a lot!

released as https://github.com/vgarvardt/gueron/releases/tag/v2.2.1


Weekly Activities [13May-19May]

This weeks activities includes:
Decouple wikidata
Finalize question answering

Eduka decoupled (3 hours)
Question answering completed (4.5 hours)


graph doesn't update when series changes in highchart-ng 0.13

when a series is updated, the graph doesn't redraw. this is use to work in 0.11
an examples is in
http://jsfiddle.net/6p39mdye/1/

I added the setTimeout section.

You need to use angular $timeout: http://jsfiddle.net/uys5qrf2/
As far as I'm aware nothing has changed with this.


2D map not appear

Hi Verlab,
I am new to ROS. I have follow the step by step that you provided. However, on rviz, only tf odom appear. 2D map not generated. I am using ubuntu 18.04, ros Melodic, cuda 10.2.

Hello @darissa , sorry for the late reply. I hope you succeed running and trying the code. The packages and launches were developed and mostly tested with ROS Kinetic. We provided some portability instructions for the turtlebot2 and TF's for Melodic here https://github.com/verlab/3DSemanticMapping_JINT_2020/blob/master/docs/INSTALL_MELODIC.md. Maybe the map/turtlebot2 TF can be the reason why the 2D occupancy map don't show up.
We are currently refactoring this code and creating a new branch to be used with ROS Noetic (with a simpler running scripts) likely this week. We keep you posted as soon as we have created the noetic branch. Best

Yes, I have followed the installation guide for melodic. Already surrender =D. But went back to your page as I still cannot solve my problem. I have same objective like yours. Just need to replace the config & weight file as I want to detect Tree. Plus I need display Tree in 2D map just like Doors in your 2D map. Please help if you have any idea or link or somewhere that I can refer. Thank you.

@renatojmsdh Btw, what version opencv do you used? I got error during compile:

error: ‘cvResizeWindow’ was not declared in this scope
cvResizeWindow("YOLO V3", 640, 480);

error: no matching function for call to ‘_IplImage::_IplImage(cv::Mat&)’

I am using opencv-4.5.3.

Hello @darissa,
The opencv version was the one used in darknet_ros package: <IP_ADDRESS> (it also the one provided by default in the software manager of ubuntu xenial). And there have been indeed strong changes that happened from opencv 2.x to opencv 3.x and to opencv 4.x. I think it is unlike you will be able to install this version of darknet with opencv4.x because of c++11 support required now on. If you can use opencv 3.4 I think these modifications can work for compiling darknet_ros:
Please replace the following lines in darknet_ros/darknet_ros/src/YoloObjectDetector.cpp for fixing opencv renamed functions including resizeWindow, namedWindow, ... since opencv 3.x:

line 431:   shell int c = cv::waitKey(waitKeyDelay_);
lines 522 - 528:

  if (!demoPrefix_ && viewImage_) {
	  cv::namedWindow("YOLO V3", cv::WINDOW_NORMAL);
    if (fullScreen_) {
	    cv::setWindowProperty("YOLO V3", cv::WND_PROP_FULLSCREEN, cv::WINDOW_FULLSCREEN);
    } else {
	    cv::moveWindow("YOLO V3", 0, 0);
	    cv::resizeWindow("YOLO V3", 640, 480);

lines 562 - 567: (https://stackoverflow.com/questions/4664187/converting-cvmat-to-iplimage)

IplImage* YoloObjectDetector::getIplImage()
{
  boost::shared_lock<boost::shared_mutex> lock(mutexImageCallback_);
  IplImage ROS_img = cvIplImage(camImageCopy_);
  return &ROS_img;
}

best

Were you able to finish the porting process for noetic? I'm attempting to set this on melodic. map frame seems to be missing from tf

Same question +1, thank you!

sorry guys as I was not able to solve this.


Stricter Meta Schema, Fixed Issues Found

As discussed in #357, we have a couple of inconsistent schemas that declare one thing in meta:* and then disregard the implementation. This was partly caused by a bug in our tests that skipped the meta.schema.json validation, so that even invalid schemas would pass.
This PR:

makes sure every schema is validated against meta.schema.json
adds rules to meta.schema.json, in particular

when using meta:extends, make sure there is an allOf with at least two $refs and each $ref points either to a URL or to something in #definitions
meta:status is present for every schema

fixes all violates of the above rules (our metric definitions are a mess)
fixes two remaining metrics that have not been converted into metric definitions

This is great, thanks Lars for adding.

@cdegroot-adobe  - With this change all of those solution extensions would need to include in their allOf the definition of the base context/experienceevent.  In doing this they are no longer smaller schema fragments that include only the new fields, but instead represent a complete schema which includes the fields of the schema it is extending plus the new fields.  It's really a pre extended version of experienceevent.
I'm wondering how this affects the workflow you've talked about where these were intended to be only small fragments that are mixed in later via and API call and creation of a customer extended schema. Do we need another way to define extension fragments that can be selected and mixed in later rather?

We already have a way of creating schema fragments: these are abstract schemas, i.e. definitions: yes, $ref: no.

@trieloff - From what I understand the UI is to eventually allow a user to select the base context/profile schema, and then a second drop down or selection shows the extensions that can be added. How will the UI/API know what schemas to show in that second selection? I think the PM group has been thinking the meta:extends contains the list it CAN extend.

@cmathis please open a separate issue for this, paging @kstreeter I'm not opposed to changing the semantics of meta:extends,  especially that right now it is redundant to allOf/…/$ref


Error when trying to run transcribe.py

Hello, the first time I tried running brainrot.js locally everything worked except my missing openAI tokens, so I returned the next day only to get greeted with this peculiar missing modules error when I wanted to run transcribe.py:

I haven't made any changes overnight or anything, ran pip install again and nothing was missing, tried switching to a different python version, followed the instructions again, all to no avail. As this same command worked perfectly fine just a day before I am perplexed. Any help would be greatly appreciated! :)

I'll try to help. So you ran "pip install -r requirements.txt" inside the generate/ folder? And you created a virtual environment before doing that and activated it? The venv folder should be in generate/venv/
Is this all correct?

yes, that is exactly what I've done.

weird. I've never had this complaint so I imagine it works fine typically on windows. Try using python in place of python3, not really sure. This is a python configuration issue more than a brainrot config issue and I'm no python expert haha

See: https://discuss.pytorch.org/t/failed-to-import-pytorch-fbgemm-dll-or-one-of-its-dependencies-is-missing/201969
Seems to be a Windows issue solvable by installing the latest Windows C++ Redistributable.


Update Thunderbird.app to v38.1.0

Thank you for the contribution. It was merged directly as 230d40ea473ae3a1d7ba6008c5fafb8302013982 to keep commit history cleaner. Your contribution is still credited to you.