Bootstrap Form Layout : class on formGoup

Hello,
What do you think about having the possibility to specify a class on the group element ?
If I want to use the grid system, I have to specify the class for both control and label:
...
        {
            grid: {
                control: 'col-sm-6',
                label: 'col-sm-6'
            }
        }

Is there a way to work on a group ?

    COMPANY
    <input type="text" class="form-control" id="inputCompany" placeholder="Enter company" formControlName="company">

It could be:
...
        {
            grid: {
                group: 'col-sm-6'
            }
        }

I really appreciate your library. Especially because you are taking care of the form layout which is not easy when we talk about dynamic forms.
I think it could be good also to specify the row class (I already read #173 and the second solution is ok but it's a lot of code if we have many lines)

@jogelin

I really appreciate your library. Especially because you are taking care of the form layout which is not easy when we talk about dynamic forms.

Thank you very much! Although a dynamic forms approach always limits your layouts in some way it's a major concern of mine to provide as much freedom as possible.
Concerning your issue I think you can already achieve this by using the container property in the grid object:
{
    grid: {
        container: "col-sm-9"
    }
}

Always appreciate someone's Bootstrap expertise here!

@jogelin
By the way, here's sample model on horizontal Bootstrap forms that also contains a group of two form controls inlined in a row.

Indeed, container works to apply a style to the form-group.
Just the name was not clear for me because "container" is also used in bootstrap grid to specify the breaking points in a responsive design.
For the "row" subject, it's important to specify the row class because bootstrap is using the negative padding which is corrected by the col class:
.row {
    margin-right: -15px;
    margin-left: -15px;
}

.col ....... {
    padding-right: 15px;
    padding-left: 15px;
}

If you use only the col class, you will add paddings that you don't want.
I tried a way by using groups and doing something like:

    <dynamic-form-ng-bootstrap-control *ngFor="let controlModel of groupModel.group"
                                       [group]="formGroup"
                                       [model]="controlModel"
                                       [hasErrorMessaging]="controlModel.hasErrorMessages"></dynamic-form-ng-bootstrap-control>

But it fails on the dynamic-form-ng-bootstrap-control component  with:
ERROR TypeError: Cannot read property 'valueChanges' of null
    at DynamicFormNGBootstrapComponent.DynamicFormControlComponent.ngOnInit (core.es.js:1198)
    at checkAndUpdateDirectiveInline (core.es5.js:10705)
    at checkAndUpdateNodeInline (core.es5.js:12084)
    at checkAndUpdateNode (core.es5.js:12052)
    at debugCheckAndUpdateNode (core.es5.js:12681)
    at debugCheckDirectivesFn (core.es5.js:12622)

My other way for now is to create an array of DynamicFormGroupModel[] in my formModel that can be used in the template and flattenize it when I need to createFormGroup

@udos86
When I use the row class, the grid config on a control is not enough. You have to apply a col class on the dynamic-form-ng-bootstrap-control component to make it works. In my case, the structure of the generated dom is:
<form ... >
    ....

    <dynamic-form-ng-bootstrap-control ...>

            ...
            <input class="form-control" />

    </dynamic-form-ng-bootstrap-control>

    ....
</form>

After some hacks, my reflexion is that the grid design approach is maybe in the wrong way. We should not apply a layout on a control. The layout should be specified on top of the controls and arrange them on the grid. In this way, the grid specifications could be extracted from the controls configurations. Then we can define controls and re-use them in many dynamic forms using different grid.
My current (and ugly) approach: https://plnkr.co/edit/rG9XLGZKfsxB0QZtkFoq?p=preview
For example, the layout could be specified when we do this.formService.createFormGroup(this.formModel, this.formGrid)
What do you think ?

@jogelin First of all THANKS for all your thoughts and providing sample code!
I can understand your point here. Making the grid configuration UI-agnostic would be a great thing to have. On the other side it would make the template logic much more complex than it is at the moment. Another scenario that still is a bit unclear to me  when someon would like to mix a grid system of a lib like Bootstrap with a different UI lib like Material.
Concerning your sample I still think you could have achieved this by using a model structure like this. Am I wrong?

@udos86
My thinking is not to make the grid configuration UI agnostic, but splitting the grid configuration from  the components configuration. It exists also in the material library:

Material
...

Like you did for the component, you could do it for the layout configurations. Create abstract class for the layout configurations and put the implementations in each library.
But I can understand that it's not the point of this library.
Concerning the rows, yes I can do it in your way but it is not the way of bootstrap with class "row". You will have extra paddings with your solution....

@jogelin

But I can understand that it's not the point of this library.

I really like your approach and I'll keep it in mind for further refactorings.

Concerning the rows, yes I can do it in your way but it is not the way of bootstrap with class "row". You will have extra paddings with your solution....

I respectfully disagree ;-)
Please have a look at this live demo . It's the same form with three different approaches (template-driven, model-driven and  dynamic). It uses the Bootstrap grid as officially described here.
But I may be wrong...

@udos86
Thank you !
Ok I understand now, you are talking about bootstrap v3, I am talking about bootstrap v4... You can see in the documentation that for column sizing in forms, they use row
Maybe you should precise in the README that you are using bootstrap v3.
May I advise to use bootstrap v4 ? I think that if people are using bootstrap with Angular v2+, they use bootstrap v4....

@jogelin Alright, just a misunderstanding :-)

May I advise to use bootstrap v4 ? I think that if people are using bootstrap with Angular v2+, they use bootstrap v4....

I don't know, I mean, as long as Bootstrap 4 is still in alpha I would not want to update the ui-bootstrap package. It's development progress seems pretty slow to me at the moment.

@udos86 I could agree for ui-bootstrap but ng-bootstrap is using bootstrap 4 so maybe ui-ng-bootstrap should use the bootstrap 4 rules.

@jogelin

so maybe ui-ng-bootstrap should use the bootstrap 4 rules

I thought it' already doing so ;-) Maybe there are still some flaws but ui-ng-bootstrap is definately using Bootstrap 4.


Where's the icons-only css?

I loved the collapsing function of this userchrome. Where'd that go?
Specifically pieces like --uc-autohide-sidebar-delay

Hey Xan, that version can be found inside of the legacy folder

Why was that feature removed?

I decided to support Sidebery v5 beta since it has significant upgrades in comparison to v4, but it has an incompatibility with dragging/moving tabs around and the auto-shrink feature. It was easier to deal with it by removing it than coordinating a workaround. I'll bring it back by default once it's fully compatible.

Awesome. Well then, I wanna say that I love the work you're doing and it makes Firefox (LibreWolf) my favorite browser experience ever ✨ by far!

That makes all the effort and time put into it, and making the decision to share it, well worth it. Thanks for your kind words!


Problems [SOLVED] when compiling nanopolish

Today I managed to install nanopolish but I experienced some issues I had to overcome so I wanted to share with you just in case this could help anybody.
First of all, nanopolish is suposed to download htslib and fast5 and compile them for you. However, after running make, I found both folders but they were completely empty, so I manually downloaded and compiled them into nanopolish folder.
The tricky one is the reference to some libraries in the Makefile. When running make, the compiler was complaining about "undefined references to some lzma and bz2 libraries?). I'm not sure about this as I know little about C programming. In any case I modified the 11th line of the Makefile in order to "include" these libraries like this:
FROM
LIBS=-lz
TO
LIBS=-lz -llzma -lbz2
Hope this helps!
Galo

Hi,
The htslib and fast5 folders were empty because you probably did not use the --recursive flag when running git clone. Thanks for the other tips though - what operating system are you using?
Jared

It is a CentOS 7.3 server

Just for my curiosity if you start from git clone --recursive https://github.com/jts/nanopolish.git do you still need to edit the makefile to add -llzma -lbz2?

Yes, I still need to edit the makefile.

Ok, thanks for the feedback!


Add extra subdomains

Is it possible to add extra subdomains later?

Yes, it is possible.
Just do the same as if you were adding a new website.
Let's Encrypt will detect that the site already exists and ask if you just want to update.

But you have to include all the domains you've added before again, right?

Yes, you have to include all domains and subdomains.

Is there the capability of adding new domains for different serverpilot apps on the same server?
For instance, I have 3 websites with a few different domain names for each all on the same server managed by serverpilot. For one of the sites (with 4 other domain names), I ran your script and it worked perfectly.
Now, I'm wanting to add SSL to the other two websites (different apps) that have two different sets of domain names. Can I just run the script again twice for each of these? Or do I need to run it once and include every domain name on the server, regardless of which app/website it's for?

@justinbowers
You need to run the script for each domains (ex: domain.com) and include all subdomains (ex: subdomain1.domain.com subdomain2.domain.com ...).
It would look like this on the command line: domain.com subdomain1.domain.com subdomain2.domain.com ...
If you then need to add SSL to other domains, you repeat the same process.

@pedrorodbit Thanks for the reply. That's what I did with the first site on the server. I just didn't know if it would remove the existing config if I ran it again for the other sites.


Fix func annotations by signarure

It allows to have a function without either complete type validation or only partial type
validation, also it enables these kinds of parameters/functions to have support on Browse API.
Notice, the function without the type validation option setting equal true, the return type of
the function always will be the typing.Any if the type annotation is not set on the function.

github-actions-workflow-tests


Tagging users doesn't work

"Replying to someone doesn't tag them (potentially on purpose). But also I can't tag people in general. The menu doesn't show up, the usernames aren't underlined and aren't clickable." (skmmcj)

Covered by other tickets.


Bug in the Profile Section

Description
There is a bug in the profile section under Basic Information. One can be student and sometime professional both as well but one cannot be an entry level ,mid senior level or senior level professional all at the same time. So this is sort of a minor bug in the system.
Kindly check that.
I am attaching the image screenshot for the reference.
How to Reproduce

Go to 'https://courses.openmined.org/users/settings'
Click on 'Basic Information Tab'
Scroll down to 'Section which says Which of the following best describes you?'
Try selecting multiple radio buttons . You can be able to select all of it.
There you can select multiple radio buttons and there  you can select all of it so which technically is a bug as one cannot be all at the same time.

Expected Behavior
One can be either student or mid/entry/senior level professional or student and entry/mid/senior level professional but not all so by max 2 radio buttons selection is allowed not all.
Screenshots

System Information

OS: Windows
OS Version: 11
Browser (if applicable): Microsoft Edge
Browser Version : Version 96.0.1054.43 (Official build) (64-bit)

Additional Context
Add any other context about the problem here.

Yeah just noticed it!
CC: @tcp


docs: simplify lazy installation

I'm aware of this feature of Lazy, but I'm not sure it's that obvious for inexperienced users. You would still need to change it in case you need custom options. In that case, maybe opts option makes more sense:
{
    "ivanjermakov/troublesum.nvim",
    opts = {}
}

I think the installation instructions should show them how to get up and running as quickly as possible. Then the configuration section can show them how to apply any deviations, as you've done in your example above.


403 (Unauthorized) response from AcquireOBOToken request in SPFx Teams Tab App

Target SharePoint environment
SharePoint Online
What SharePoint development model, framework, SDK or API is this about?
💥 SharePoint Framework
Developer environment
None
What browser(s) / client(s) have you tested

[ ] 💥 Internet Explorer
[ ] 💥 Microsoft Edge
[x] 💥 Google Chrome
[ ] 💥 FireFox
[ ] 💥 Safari
[ ] mobile (iOS/iPadOS)
[ ] mobile (Android)
[ ] not applicable
[ ] other (enter in the "Additional environment details" area below)

Additional environment details

SPFx version 1.18.2
Node.js version 18.17.1

Describe the bug / error
A user is reporting that our SPFx Teams Tab app is failing to load with the following error:
GET https://[site]/_api/Microsoft.SharePoint.Internal.ClientSideComponent.Token.AcquireOBOToken?resource=%27https://graph.microsoft.com%27&clientId=[client_id] 403 (Forbidden)
They reported this issue is taking place in a private channel and in a browser. Public channels do not appear to be affected. We cannot recreate in our own environments, and other users have not reported this issue.
I've noticed similar issues being tracked in this repo, but they appear to be outdated and reportedly fixed.
Has there been any recent updates to this issue? Is there anything I should check with the user?
Thank you.
Steps to reproduce
Underlying code:
const client = await this.context.msGraphClientFactory.getClient("3");
const rootWebUrlResponse = await client
          .api(`groups/${groupId}/sites/root/weburl`)
          .version("v1.0")
          .get();

Expected behavior
Graph token is acquired without issue.

@nick-pape Any update on this?

Same error for me, with same SPFx code inside Teams Tab;
const clientGraph:MSGraphClientV3 = await this.context.msGraphClientFactory.getClient('3');
const meBlobPhoto = await clientGraph.api(`/users/jbasterra@ayesa.com/photo/$value`).get();

First request seems to be ok, but second ends with error.

Kind Regards.

We are also experiencing the same issue. SPFx app used in Teams Tab, getting 403 Forbidden when requesting OBO token for the SharePoint Web Application Principal app. Only happens in one of three environments, the one it is happening in is a targeted release (early access) tenant.


Not JSON

Really cool experiment, I tried to do the same the other way around; inserting scripts into CSS. However, I have something to note. This is not JSON:
new Descartes({
  "html": {
    "margin": 0,
    "padding": 0,
    "body": {
      "margin": 0,
      "padding": 0,
      "section": {
        "max-width": "800px"
      }
    }
  }
})

It's just a javascript object (also noted as literal object). It'd be JSON if it was all a big string, but in this way it's just an object. See one explanation and another.

You're right - quite honestly, I was just lazy about my terminology and inconsistent with the copy on the homepage. It's a JS object literal, and not JSON.

How about storing the "object" data in a separate .json file? You could import and JSON.parse() the file as needed. This might improve maintainability by keeping data separate from application logic.

Yeah but now the question is whether or not the method also accepts a string to convert it to an object with JSON.parse(). But it'd be quite easy to do so and also quite useful.


python310Packages.frigidaire: 0.18.5 -> 0.18.12

Diff: https://github.com/bm1549/frigidaire/compare/refs/tags/0.18.5...0.18.12
Changelog: https://github.com/bm1549/frigidaire/releases/tag/0.18.12
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 226117 run on x86_64-linux 1

4 packages built:

python310Packages.frigidaire
python310Packages.frigidaire.dist
python311Packages.frigidaire
python311Packages.frigidaire.dist


Make capabilities an option in crio.conf

The list of capabilities to be added can be defined in
/etc/crio/crio.conf. If the option is commented out, the
default capabilities defined in the spec will be used.
Fixes https://github.com/kubernetes-incubator/cri-o/issues/1536
Signed-off-by: umohnani8<EMAIL_ADDRESS>

Let us add an integration test that uses this configuration.

I think there should be a --default-capabilities flag for CRI-O so distros can specify the list of capabilities to add by default

/test all

@mrunalp
The error is:
Detected go version: go version go1.10.1 linux/amd64.
Kubernetes requires go1.10.2 or greater.
Please install go1.10.2 or later.

Does this mean we need to update the AMI?

/test all

/test critest_fedora
/test critest_rhel

/test critest_rhel

/test integration_rhel

/test integration_fedora

/test kata-containers

Tests pass!
@rhatdan @mrunalp PTAL

LGTM

@umohnani8 Sadly this needs a rebase, also could you check to make sure the seccomp handling is done correctly.  IE Seccomp rules are supposed to reflect the current settings in the capabilities list.

rebased.

/test all

/test e2e_rhel

LGTM


Show no results text when no matching elements

How do i add a record that says No records found when no elements are matched?

Hi,
At the moment that's not supported natively. I'll implement that in a future release.
It's possible to implement it, though, using the event after:flexdatalist.search  and onblur (to remove the "no records found").

Hi,
On release 1.8.0 it shows "No results found". You're able to customize the text with the option noResultsText.

Works great, thanks!


Second Largest

❗Write Code in Only One Language ( CPP , C , Java , Python )
Give File Name :  Second_largest  ✅ In respective language folder
📑 DESCRIPTION
Second Largest

Given an array Arr of size N, print second largest distinct element from an array.
Example 1:
Input:
N = 6
Arr[] = {12, 35, 1, 10, 34, 1}
Output: 34
Explanation: The largest element of the
array is 35 and the second largest element
is 34.

Example 2:
Input:
N = 3
Arr[] = {10, 5, 10}
Output: 5
Explanation: The largest element of
the array is 10 and the second
largest element is 5.

Expected Time Complexity: O(N)
Expected Auxiliary Space: O(1)

Assign this issue to me I am from RCPIT


Update outdated build badges

#9225

@trivialfis let's deal with the documentation in other Pull Request. It deserves more attention as xgboost has more than two README.md files providing an important context in few places.


Switch to Cachix

@romac , CI has already pushed the cosmos-nix cache, so you should be able to switch hermes to use this now, once you have we can merge this and it will stop pushing the old S3 cache

Just switched the Hermes CI to using Cachix, so I guess we can merge this.
@soareschen Or do you need to switch Hermes SDK over as well?


TODO

Oh my intention was Cloud. I think this is an interesting point would like to delve in with you and make sure we are on the same page.
We've discussed in previous ontology discussions and examples about a node being a single "software component" or at least that's how I felt each node in an ontology represented.
But as I've been thinking about this more and as we've been exploring this more at Intel a node in an ontology is not really a software component for us, it's a thing we want to measure separate from everything else we are measuring.
So at Intel, we might want to work with a client and break a software component into the processes that run on AMD vs the processes that tun on Intel. So we can see how much energy the same process consumes when running on AMD chips vs Intel chips.
It's subtle but I think that's it's an important characteristic of the ontology, the lowest level of granularity isn't the process that's run, it's just a clustering of things we want to measure and that might not even relate to a process. Does that make sense?

Closing thius


Vendor with specific tag / commit?

Shouldn't gb vendor support a tag / commit option?  Otherwise it just does exactly the same thing as go get.

Gb vendor is just a proof of concept at the moment, it just shells out to go get to help bootstrap your project.
It would be nice to teach it about tags and versions, possibly so I can be used to update an existing vendored package.
For the moment the work around is to fetch the source manually.
+help wanted

Are you open to the idea of gb being able to:

Work for libraries? i.e. someone can fetch my library and all deps and
(should they wish to) use gb build mylib to use those deps instead of
what's on their global GOPATH?
Store a reference to those tags in some kind of .vendordotfilething?
etc?

I write more libraries than 'package mains' but being able to provide a
reproducible version  would be extremely useful. The workflow would
probably be go get (which should fetch the vendor/ path too) and then gb build to build off ./vendor.
On Thu, Apr 30, 2015 at 10:43 AM Dave Cheney<EMAIL_ADDRESS>wrote:

Gb vendor is just a proof of concept at the moment, it just shells out to
go get to help bootstrap your project.
It would be nice to teach it about tags and versions, possibly so I can be
used to update an existing vendored package.
For the moment the work around is to fetch the source manually.
+help wanted
—
Reply to this email directly or view it on GitHub
https://github.com/constabulary/gb/issues/17#issuecomment-97607868.

Matt, can you please make a new issue for this. I want to address this
separately.
On Thu, 30 Apr 2015 09:06 Matt Silverlock<EMAIL_ADDRESS>wrote:

Are you open to the idea of gb being able to:

Work for libraries? i.e. someone can fetch my library and all deps and
(should they wish to) use gb build mylib to use those deps instead of
what's on their global GOPATH?
Store a reference to those tags in some kind of .vendordotfilething?
etc?

I write more libraries than 'package mains' but being able to provide a
reproducible version would be extremely useful. The workflow would
probably be go get (which should fetch the vendor/ path too) and then gb build to build off ./vendor.
On Thu, Apr 30, 2015 at 10:43 AM Dave Cheney<EMAIL_ADDRESS>wrote:

Gb vendor is just a proof of concept at the moment, it just shells out to
go get to help bootstrap your project.
It would be nice to teach it about tags and versions, possibly so I can
be
used to update an existing vendored package.
For the moment the work around is to fetch the source manually.
+help wanted
—
Reply to this email directly or view it on GitHub
https://github.com/constabulary/gb/issues/17#issuecomment-97607868.

—
Reply to this email directly or view it on GitHub
https://github.com/constabulary/gb/issues/17#issuecomment-97610724.

Imo a plugin which tackles source packaging and library version compatibility via a package file might be cool. Something like cargo/leiningen/maven. We could also have a package server to go with the plugin.

We could also have a package server to go with the plugin.

That sounds much larger in scope than I imagined, but that's why it's
implemented as a plugin, anyone can write a plugin, they don;t need to be
part of the main gb distribution.
On Thu, Apr 30, 2015 at 11:20 AM, andrewchambers<EMAIL_ADDRESS>wrote:

Imo a plugin which tackles source packaging and library version
compatibility via a package file might be cool. Something like
cargo/leiningen/maven. We could also have a package server to go with the
plugin.
—
Reply to this email directly or view it on GitHub
https://github.com/constabulary/gb/issues/17#issuecomment-97627963.

Yeah, I guess whoever wants to step up can just do it in their own plugin repo. If a plugin gets traction, it can get referenced from the official readme here.

What I am planning on doing is moving gb-vendor to it's own repository so it can evolve independent of gb. This will happen when I've got the new github pages' site up with updated documentation. I'll close this then.

gb-vendor has been moved to it's own repository, as discussed in #29


errors when running concurrently

When trying to load a given scenario in more than one thread, we see the following issue:
foo.py contains:
import ixmp

p = ixmp.Platform()
p.Scenario('MESSAGEix-GLOBIOM_BvR_OFR1', 'NPi2020_Net0')

Then we run:
In [1]: import multiprocessing, subprocess

In [2]: n=2; pool=multiprocessing.Pool(n); pool.map(subprocess.call, [('python', 'foo.py')] * n)
INFO:root:launching ixmp.Platform using config file at '/home/gidden/work/iiasa/message/properties_files/enedb.properties'
INFO:root:launching ixmp.Platform using config file at '/home/gidden/work/iiasa/message/properties_files/enedb.properties'
Nov 21, 2018 9:39:29 AM org.flywaydb.core.internal.util.VersionPrinter printVersion
INFO: Flyway 4.2.0 by Boxfuse
Nov 21, 2018 9:39:29 AM org.flywaydb.core.internal.util.VersionPrinter printVersion
INFO: Flyway 4.2.0 by Boxfuse
Nov 21, 2018 9:39:30 AM org.flywaydb.core.internal.dbsupport.DbSupportFactory createDbSupport
INFO: Database: jdbc:oracle:thin:@gp3.iiasa.ac.at:1521:GP3 (Oracle 12.1)
Nov 21, 2018 9:39:30 AM org.flywaydb.core.internal.dbsupport.DbSupportFactory createDbSupport
INFO: Database: jdbc:oracle:thin:@gp3.iiasa.ac.at:1521:GP3 (Oracle 12.1)
Nov 21, 2018 9:39:30 AM org.flywaydb.core.internal.command.DbValidate validate
INFO: Successfully validated 11 migrations (execution time 00:00.030s)
Nov 21, 2018 9:39:30 AM org.flywaydb.core.internal.command.DbValidate validate
INFO: Successfully validated 11 migrations (execution time 00:00.031s)
2018-11-21 09:39:31,049  INFO at.ac.iiasa.ixmp.Platform:111 - Welcome to the IX modeling platform!
2018-11-21 09:39:31,050  INFO at.ac.iiasa.ixmp.Platform:112 -  connected to database<EMAIL_ADDRESS>/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py:171: UserWarning: The constructor `mp.Scenario()` is deprecated, please use `ixmp.Scenario(mp, ...)`
  warnings.warn('The constructor `mp.Scenario()` is deprecated, '
2018-11-21 09:39:31,112  INFO at.ac.iiasa.ixmp.objects.Scenario:218 - loading Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' by version id (version: 1, runid: 18489)...
2018-11-21 09:39:31,113  INFO at.ac.iiasa.ixmp.Platform:111 - Welcome to the IX modeling platform!
2018-11-21 09:39:31,115  INFO at.ac.iiasa.ixmp.Platform:112 -  connected to database<EMAIL_ADDRESS>/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py:171: UserWarning: The constructor `mp.Scenario()` is deprecated, please use `ixmp.Scenario(mp, ...)`
  warnings.warn('The constructor `mp.Scenario()` is deprecated, '
2018-11-21 09:39:31,179  INFO at.ac.iiasa.ixmp.objects.Scenario:218 - loading Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' by version id (version: 1, runid: 18489)...
2018-11-21 09:39:31,541  INFO at.ac.iiasa.ixmp.objects.Scenario:241 - done loading Scenario from the database!
2018-11-21 09:39:31,571  INFO at.ac.iiasa.ixmp.objects.MsgScenario:457 -  spatial and temporal disaggregation maps assignment
2018-11-21 09:39:31,590  INFO at.ac.iiasa.ixmp.objects.MsgScenario:573 -  period mapping assignment
2018-11-21 09:39:31,612  INFO at.ac.iiasa.ixmp.objects.Scenario:241 - done loading Scenario from the database!
2018-11-21 09:39:31,613  INFO at.ac.iiasa.ixmp.objects.MsgScenario:599 -  duration period assignment
2018-11-21 09:39:31,636  INFO at.ac.iiasa.ixmp.objects.Scenario:1826 - committing changes of Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' to the database (runid: 18489)...
Traceback (most recent call last):
  File "foo.py", line 4, in <module>
    p.Scenario('MESSAGEix-GLOBIOM_BvR_OFR1', 'NPi2020_Net0')
  File "/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py", line 174, in Scenario
    return Scenario(self, model, scen, version, scheme, annotation, cache)
  File "/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py", line 460, in __init__
    self._jobj = mp._jobj.getScenario(model, scenario)
jpype._jexception.IxExceptionPyRaisable: at.ac.iiasa.ixmp.exceptions.IxException: This Scenario is currently locked by user gidden
2018-11-21 09:39:32,907  INFO at.ac.iiasa.ixmp.objects.Scenario:601 - done updating Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' to the database (runid: 18489, version: 1)!
2018-11-21 09:39:32,912  INFO at.ac.iiasa.ixmp.objects.MsgScenario:457 -  spatial and temporal disaggregation maps assignment
2018-11-21 09:39:32,912  INFO at.ac.iiasa.ixmp.objects.MsgScenario:573 -  period mapping assignment
2018-11-21 09:39:32,913  INFO at.ac.iiasa.ixmp.objects.MsgScenario:599 -  duration period assignment
/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py:468: UserWarning: Using `ixmp.Scenario` for MESSAGE-scheme scenarios is deprecated, please use `message_ix.Scenario`
  warnings.warn('Using `ixmp.Scenario` for MESSAGE-scheme scenarios '
Out[2]: [0, 1]

There are a few things happening here that @OFR-IIASA and I don't understand, but to start, there is a commit happening when loading the scenario which could be culprit (don't know why it happens):
2018-11-21 09:39:31,636  INFO at.ac.iiasa.ixmp.objects.Scenario:1826 - committing changes of Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' to the database (runid: 18489)...

any thoughts @zikolach or @danielhuppmann ?

When touching a scenario that was created with MESSAGEix v1.0 for the first time with MESSAGEix v1.1, the scenario will be updated (new sets and parameters initialised). This should happen only once for any scenario.
It could be that two processes happening virtually simultaneously result in a conflict here, as both processes load the same scenario and try to update it.
Did you touch this scenario first with only one process? If not, try n=1 first, and then again n>1.

Yes, I also just came to this conclusion. I agree with you that it should work. Let me test and I'll get back to you.

Unfortunately, it looks like it's somehow not actually being saved?
In [3]: n=1; pool=multiprocessing.Pool(n); pool.map(subprocess.call, [('python', 'foo.py')] * n)
INFO:root:launching ixmp.Platform using config file at '/home/gidden/work/iiasa/message/properties_files/enedb.properties'
Nov 21, 2018 10:04:52 AM org.flywaydb.core.internal.util.VersionPrinter printVersion
INFO: Flyway 4.2.0 by Boxfuse
Nov 21, 2018 10:04:52 AM org.flywaydb.core.internal.dbsupport.DbSupportFactory createDbSupport
INFO: Database: jdbc:oracle:thin:@gp3.iiasa.ac.at:1521:GP3 (Oracle 12.1)
Nov 21, 2018 10:04:53 AM org.flywaydb.core.internal.command.DbValidate validate
INFO: Successfully validated 11 migrations (execution time 00:00.034s)
2018-11-21 10:04:53,395  INFO at.ac.iiasa.ixmp.Platform:111 - Welcome to the IX modeling platform!
2018-11-21 10:04:53,401  INFO at.ac.iiasa.ixmp.Platform:112 -  connected to database<EMAIL_ADDRESS>/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py:171: UserWarning: The constructor `mp.Scenario()` is deprecated, please use `ixmp.Scenario(mp, ...)`
  warnings.warn('The constructor `mp.Scenario()` is deprecated, '
2018-11-21 10:04:53,477  INFO at.ac.iiasa.ixmp.objects.Scenario:218 - loading Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' by version id (version: 1, runid: 18489)...
2018-11-21 10:04:53,797  INFO at.ac.iiasa.ixmp.objects.Scenario:241 - done loading Scenario from the database!
2018-11-21 10:04:53,838  INFO at.ac.iiasa.ixmp.objects.MsgScenario:457 -  spatial and temporal disaggregation maps assignment
2018-11-21 10:04:53,877  INFO at.ac.iiasa.ixmp.objects.MsgScenario:573 -  period mapping assignment
2018-11-21 10:04:53,886  INFO at.ac.iiasa.ixmp.objects.MsgScenario:599 -  duration period assignment
2018-11-21 10:04:53,913  INFO at.ac.iiasa.ixmp.objects.Scenario:1826 - committing changes of Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' to the database (runid: 18489)...
2018-11-21 10:04:55,250  INFO at.ac.iiasa.ixmp.objects.Scenario:601 - done updating Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' to the database (runid: 18489, version: 1)!
2018-11-21 10:04:55,254  INFO at.ac.iiasa.ixmp.objects.MsgScenario:457 -  spatial and temporal disaggregation maps assignment
2018-11-21 10:04:55,254  INFO at.ac.iiasa.ixmp.objects.MsgScenario:573 -  period mapping assignment
2018-11-21 10:04:55,255  INFO at.ac.iiasa.ixmp.objects.MsgScenario:599 -  duration period assignment
/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py:468: UserWarning: Using `ixmp.Scenario` for MESSAGE-scheme scenarios is deprecated, please use `message_ix.Scenario`
  warnings.warn('Using `ixmp.Scenario` for MESSAGE-scheme scenarios '
Out[3]: [0]

In [4]: n=4; pool=multiprocessing.Pool(n); pool.map(subprocess.call, [('python', 'foo.py')] * n)
INFO:root:launching ixmp.Platform using config file at '/home/gidden/work/iiasa/message/properties_files/enedb.properties'
INFO:root:launching ixmp.Platform using config file at '/home/gidden/work/iiasa/message/properties_files/enedb.properties'
INFO:root:launching ixmp.Platform using config file at '/home/gidden/work/iiasa/message/properties_files/enedb.properties'
INFO:root:launching ixmp.Platform using config file at '/home/gidden/work/iiasa/message/properties_files/enedb.properties'
Nov 21, 2018 10:04:59 AM org.flywaydb.core.internal.util.VersionPrinter printVersion
INFO: Flyway 4.2.0 by Boxfuse
Nov 21, 2018 10:04:59 AM org.flywaydb.core.internal.util.VersionPrinter printVersion
INFO: Flyway 4.2.0 by Boxfuse
Nov 21, 2018 10:04:59 AM org.flywaydb.core.internal.util.VersionPrinter printVersion
INFO: Flyway 4.2.0 by Boxfuse
Nov 21, 2018 10:04:59 AM org.flywaydb.core.internal.util.VersionPrinter printVersion
INFO: Flyway 4.2.0 by Boxfuse
Nov 21, 2018 10:05:00 AM org.flywaydb.core.internal.dbsupport.DbSupportFactory createDbSupport
INFO: Database: jdbc:oracle:thin:@gp3.iiasa.ac.at:1521:GP3 (Oracle 12.1)
Nov 21, 2018 10:05:00 AM org.flywaydb.core.internal.dbsupport.DbSupportFactory createDbSupport
INFO: Database: jdbc:oracle:thin:@gp3.iiasa.ac.at:1521:GP3 (Oracle 12.1)
Nov 21, 2018 10:05:01 AM org.flywaydb.core.internal.command.DbValidate validate
INFO: Successfully validated 11 migrations (execution time 00:00.071s)
Nov 21, 2018 10:05:01 AM org.flywaydb.core.internal.dbsupport.DbSupportFactory createDbSupport
INFO: Database: jdbc:oracle:thin:@gp3.iiasa.ac.at:1521:GP3 (Oracle 12.1)
Nov 21, 2018 10:05:01 AM org.flywaydb.core.internal.dbsupport.DbSupportFactory createDbSupport
INFO: Database: jdbc:oracle:thin:@gp3.iiasa.ac.at:1521:GP3 (Oracle 12.1)
2018-11-21 10:05:01,384  INFO at.ac.iiasa.ixmp.Platform:111 - Welcome to the IX modeling platform!
2018-11-21 10:05:01,385  INFO at.ac.iiasa.ixmp.Platform:112 -  connected to database<EMAIL_ADDRESS>/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py:171: UserWarning: The constructor `mp.Scenario()` is deprecated, please use `ixmp.Scenario(mp, ...)`
  warnings.warn('The constructor `mp.Scenario()` is deprecated, '
Nov 21, 2018 10:05:01 AM org.flywaydb.core.internal.command.DbValidate validate
INFO: Successfully validated 11 migrations (execution time 00:00.040s)
2018-11-21 10:05:01,468  INFO at.ac.iiasa.ixmp.objects.Scenario:218 - loading Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' by version id (version: 1, runid: 18489)...
Nov 21, 2018 10:05:01 AM org.flywaydb.core.internal.command.DbValidate validate
INFO: Successfully validated 11 migrations (execution time 00:00.080s)
Nov 21, 2018 10:05:01 AM org.flywaydb.core.internal.command.DbValidate validate
INFO: Successfully validated 11 migrations (execution time 00:00.077s)
2018-11-21 10:05:01,620  INFO at.ac.iiasa.ixmp.Platform:111 - Welcome to the IX modeling platform!
2018-11-21 10:05:01,623  INFO at.ac.iiasa.ixmp.Platform:112 -  connected to database<EMAIL_ADDRESS>/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py:171: UserWarning: The constructor `mp.Scenario()` is deprecated, please use `ixmp.Scenario(mp, ...)`
  warnings.warn('The constructor `mp.Scenario()` is deprecated, '
2018-11-21 10:05:01,698  INFO at.ac.iiasa.ixmp.objects.Scenario:218 - loading Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' by version id (version: 1, runid: 18489)...
2018-11-21 10:05:01,785  INFO at.ac.iiasa.ixmp.Platform:111 - Welcome to the IX modeling platform!
2018-11-21 10:05:01,786  INFO at.ac.iiasa.ixmp.Platform:112 -  connected to database<EMAIL_ADDRESS>/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py:171: UserWarning: The constructor `mp.Scenario()` is deprecated, please use `ixmp.Scenario(mp, ...)`
  warnings.warn('The constructor `mp.Scenario()` is deprecated, '
2018-11-21 10:05:01,818  INFO at.ac.iiasa.ixmp.Platform:111 - Welcome to the IX modeling platform!
2018-11-21 10:05:01,821  INFO at.ac.iiasa.ixmp.Platform:112 -  connected to database<EMAIL_ADDRESS>/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py:171: UserWarning: The constructor `mp.Scenario()` is deprecated, please use `ixmp.Scenario(mp, ...)`
  warnings.warn('The constructor `mp.Scenario()` is deprecated, '
2018-11-21 10:05:01,881  INFO at.ac.iiasa.ixmp.objects.Scenario:218 - loading Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' by version id (version: 1, runid: 18489)...
2018-11-21 10:05:01,947  INFO at.ac.iiasa.ixmp.objects.Scenario:218 - loading Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' by version id (version: 1, runid: 18489)...
2018-11-21 10:05:02,002  INFO at.ac.iiasa.ixmp.objects.Scenario:241 - done loading Scenario from the database!
2018-11-21 10:05:02,105  INFO at.ac.iiasa.ixmp.objects.MsgScenario:457 -  spatial and temporal disaggregation maps assignment
2018-11-21 10:05:02,129  INFO at.ac.iiasa.ixmp.objects.MsgScenario:573 -  period mapping assignment
2018-11-21 10:05:02,144  INFO at.ac.iiasa.ixmp.objects.MsgScenario:599 -  duration period assignment
2018-11-21 10:05:02,198  INFO at.ac.iiasa.ixmp.objects.Scenario:1826 - committing changes of Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' to the database (runid: 18489)...
2018-11-21 10:05:02,378  INFO at.ac.iiasa.ixmp.objects.Scenario:241 - done loading Scenario from the database!
Traceback (most recent call last):
  File "foo.py", line 4, in <module>
    p.Scenario('MESSAGEix-GLOBIOM_BvR_OFR1', 'NPi2020_Net0')
  File "/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py", line 174, in Scenario
    return Scenario(self, model, scen, version, scheme, annotation, cache)
  File "/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py", line 460, in __init__
    self._jobj = mp._jobj.getScenario(model, scenario)
jpype._jexception.IxExceptionPyRaisable: at.ac.iiasa.ixmp.exceptions.IxException: This Scenario is currently locked by user gidden
2018-11-21 10:05:02,513  INFO at.ac.iiasa.ixmp.objects.Scenario:241 - done loading Scenario from the database!
Traceback (most recent call last):
  File "foo.py", line 4, in <module>
    p.Scenario('MESSAGEix-GLOBIOM_BvR_OFR1', 'NPi2020_Net0')
  File "/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py", line 174, in Scenario
    return Scenario(self, model, scen, version, scheme, annotation, cache)
  File "/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py", line 460, in __init__
    self._jobj = mp._jobj.getScenario(model, scenario)
jpype._jexception.IxExceptionPyRaisable: at.ac.iiasa.ixmp.exceptions.IxException: This Scenario is currently locked by user gidden
2018-11-21 10:05:02,642  INFO at.ac.iiasa.ixmp.objects.Scenario:241 - done loading Scenario from the database!
Traceback (most recent call last):
  File "foo.py", line 4, in <module>
    p.Scenario('MESSAGEix-GLOBIOM_BvR_OFR1', 'NPi2020_Net0')
  File "/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py", line 174, in Scenario
    return Scenario(self, model, scen, version, scheme, annotation, cache)
  File "/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py", line 460, in __init__
    self._jobj = mp._jobj.getScenario(model, scenario)
jpype._jexception.IxExceptionPyRaisable: at.ac.iiasa.ixmp.exceptions.IxException: This Scenario is currently locked by user gidden
2018-11-21 10:05:03,686  INFO at.ac.iiasa.ixmp.objects.Scenario:601 - done updating Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' to the database (runid: 18489, version: 1)!
2018-11-21 10:05:03,692  INFO at.ac.iiasa.ixmp.objects.MsgScenario:457 -  spatial and temporal disaggregation maps assignment
2018-11-21 10:05:03,693  INFO at.ac.iiasa.ixmp.objects.MsgScenario:573 -  period mapping assignment
2018-11-21 10:05:03,698  INFO at.ac.iiasa.ixmp.objects.MsgScenario:599 -  duration period assignment
/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py:468: UserWarning: Using `ixmp.Scenario` for MESSAGE-scheme scenarios is deprecated, please use `message_ix.Scenario`
  warnings.warn('Using `ixmp.Scenario` for MESSAGE-scheme scenarios '
Out[4]: [0, 1, 1, 1]

See the warning message
/home/gidden/.local/lib/python3.5/site-packages/ixmp/core.py:468: UserWarning: Using `ixmp.Scenario` for MESSAGE-scheme scenarios is deprecated, please use `message_ix.Scenario`   warnings.warn('Using `ixmp.Scenario` for MESSAGE-scheme scenarios '
Try using message_ix.Scenario().

More to the point, the function updateMsgScheme() is called at every loading of a MESSAGE-scheme scenario, and it does a check-out, validates the version (sets and parameters), and then either discards (if no changes were made) or commits the changes.
Having a lag of one second between each process should do the trick.

@danielhuppmann why we cannot do blocking there instead letting users shoot into their leg?

@zikolach, the blocking on the database side is exactly what is causing the problem.

I updated foo.py to look like
import ixmp
import message_ix

p = ixmp.Platform()
message_ix.Scenario(p, 'MESSAGEix-GLOBIOM_BvR_OFR1', 'NPi2020_Net0')

The error persists, unfortunately.

So, to get the conversation back on track, here is my understanding of the issue:
This is an old scenario with scheme == 1.0.0. It needs to be updated to 1.1.0.
The hypothesis is that by loading it once, the scheme is updated and committed. Thus all should be fine.
However, this hypothesis is incorrect. When loading it once (n=1 above), all is well. Then it is attempted to be loaded multiple times ( n > 1). In this second round, again commits are generated.
So, what do we think is the best way to address this issue? I agree that explicitly putting in a sleep would work, but that, I would argue, is highly unsatisfactory. I would expect to be able to concurrently load (read only) many instances of a scenario as long as its scheme is up to date. Right?

Well, the hypothesis is partially correct. Touching a scenario generated with v1.0 with an MESSAGEix installation v1.1 will automatically upgrade the scenario.
The way this was implemented is the process described above - check-out, test whether upgrades are necessary, discard or commit. This seems to take about one second if no upgrade is made on a full-scale MESSAGE scenario, not sure how much more if an upgrade is made.
Options:

Change the function updateMsgScheme() to only do the check-out if upgrades are made. I'm not sure if this actually works and whether changes are required down the rabbit hole.
Add an explicit version tag to the RUN table and test for that instead of the current workflow. Question is whether this will require a more elaborate version-control-and-upgrade process, "flyway"-style. Definitely a rabbit hole.

Is it possible that the updated version is not being labeled default?

Ok, looking over @OFR-IIASA's shoulder I think this is indeed the issue. In each of the above examples, the "default" is always version 1, e.g.,: INFO at.ac.iiasa.ixmp.objects.Scenario:218 - loading Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' by version id (version: 1, runid: 18489
Thus it is always getting a stale version here. Is this how we expect it to work? I thought there was an automatic setting to default somewhere in the pipeline of a new clone?

Ok, way too many issues that are being tangled up in this discussion...
From the conversation so far, I have no reason to believe that the upgrade to v1.1 doesn't work as intended - but you can test this by looking whether any new set or parameter from the hackathon developments exist in the scenario after touching it with MESSAGEix v.1.1.
What is confusing (and a minor bug) is that a message is written to the log file about updating a scenario even if no change to the scenario is saved to the db. The commit() function on the database does verify (afaik) whether any change has been made and only touches the db if that is true.
What needs to be double-checked is whether an entry is added to the CHANGELOG table every time a scenario is loaded, even if no change was made by the upgrade. You can verify this by looking at the changelog table for scenario id=18489, there should be one entry "Upgrade to MESSAGEix scheme version 1.1" (or similar). I tested this (in an unstructured way) prior to making the lastest ixmp PR. That would be a medium bug if the log entry was added more than once.
As I tried to explain earlier, the "checking whether an upgrade is necessary" function performs a check-out, so no other process can start loading the scenario while that verification is in progress (~ 1 sec).
Now, to the other issues: the version number you see is the identification from multiple runs with a particular model/scenario name. It has nothing to do with the MESSAGEix version number.
Also, the upgrade to MESSAGEix v1.1 does not clone a scenario (it just makes changes to an existing scenario) and does not have any impact on whether that scenario is defined as default or not.

Yes, agreed, we've got a few issues wound up in this, notably:

Reading is not strictly thread-safe (due to the fact that check outs occur, which is similar to a write in thread-safety even if no writing occurs)
Model/scenario combinations are allowed to have no defaults (thus will always grab the  version: 1 dataset)
Making changes to to the schema, e.g., updating from V1.0 to V1.1, does not update the default status of the updated scenario (i.e., V1.0 will still be the default)

Just for posterity, there are a number of questions (from expert users!) about when exactly to set_as_default(), so it would probably be good to have an FAQ or similar page on that. Also, what is the difference between scen.clone() (without new model/scenario name) and making edits and committing vs just making edits and committing. Frankly, we could probably all use a refresher course on the pseudo-version-control implemented on the ixmp backend.

Re item 3 in the list above - this is nonsense. It confuses the model/scenario version number with the MESSAGEix version number.

just to elaborate on Matts 2nd point from above. Looking in the database I observe the following:

This clearly shows that the default is not set, in fact, none of the version of this scenario have a default set, yet it is clear that this is actually loaded when using it in another process. This begs the question, why version 42, if there are also older and newer version?
__
2018-11-20 16:05:03,749  INFO at.ac.iiasa.ixmp.Platform:108 - Welcome to the IX modeling platform!
2018-11-20 16:05:03,750  INFO at.ac.iiasa.ixmp.Platform:109 -  connected to !...
2018-11-20 16:05:03,847  INFO at.ac.iiasa.ixmp.objects.Scenario:227 - loading Scenario 'MESSAGEix-GLOBIOM_BvR_OFR|NPi2020_Net0' by version id (version: 42, runid: 18473)...
2018-11-20 16:05:04,245  INFO at.ac.iiasa.ixmp.objects.Scenario:249 - done loading Scenario from the database!
2018-11-20 16:05:04,254  INFO at.ac.iiasa.ixmp.objects.Scenario:400 -  spatial and temporal disaggregation maps assignment
2018-11-20 16:05:04,254 DEBUG at.ac.iiasa.ixmp.database.DbDAO:1744 - loading elements for set 'map_spatial_hierarchy' from database...
2018-11-20 16:05:04,265 DEBUG at.ac.iiasa.ixmp.database.DbDAO:1744 - loading elements for set 'map_node' from database...
2018-11-20 16:05:04,272 DEBUG at.ac.iiasa.ixmp.database.DbDAO:1744 - loading elements for set 'map_temporal_hierarchy' from database...
2018-11-20 16:05:04,279 DEBUG at.ac.iiasa.ixmp.database.DbDAO:1744 - loading elements for set 'map_time' from database...
2018-11-20 16:05:04,287  INFO at.ac.iiasa.ixmp.objects.Scenario:530 -  period mapping assignment
2018-11-20 16:05:04,287 DEBUG at.ac.iiasa.ixmp.database.DbDAO:1744 - loading elements for set 'cat_year' from database...
2018-11-20 16:05:04,298  INFO at.ac.iiasa.ixmp.objects.Scenario:557 -  duration period assignment
2018-11-20 16:05:04,299 DEBUG at.ac.iiasa.ixmp.database.DbDAO:1744 - loading elements for parameter 'duration_period' from database...
2018-11-20 16:05:04,322 DEBUG at.ac.iiasa.ixmp.database.DbDAO:1744 - loading elements for variable 'OBJ' from database...
2018-11-20 16:05:04,337  INFO at.ac.iiasa.ixmp.objects.Scenario:227 - loading Scenario 'MESSAGEix-GLOBIOM_BvR_OFR|noBCCS' by version id (version: 0, runid: 18473)...
__

@OFR-IIASA https://github.com/iiasa/ixmp_source/blob/4f96133a8ad152aa6aadf9436ab7bb3010f0c0de/src/main/java/at/ac/iiasa/ixmp/database/DbDAO.java#L235

As indicated by @zikolach, it uses the run with the highest version number by default.
But in the case posted by @OFR-IIASA, see

2018-11-20 16:05:03,847 INFO at.ac.iiasa.ixmp.objects.Scenario:227 - loading Scenario 'MESSAGEix-GLOBIOM_BvR_OFR|NPi2020_Net0' by version id (version: 42, runid: 18473)...

It says that you specially select version 42...

@zikolach yes, that could be true now, but I believe that 42 could have been the most recent version at the time. But is this then the expected behaviour? If there is no default set, always the most recent version is used? @danielhuppmann I dont specify a version in the command when loading the scenario.

@OFR-IIASA yes - according to code it uses maximum version number if no default version found (see link I posted above)

@OFR-IIASA, at least when I wrote the different functions for loading a scenario, the logging message

loading Scenario 'model|scenario' by version id (version: x, runid: x)`

was intended for ixmp.Scenario(model, scenario, version), while ixmp.Scenario(model, scenario) was showing a message like

loading default version of Scenario 'model|scenario' (version: x, runid: x)`.

But maybe this got jumbled at some point.

@danielhuppmann so, here are the two statements i execute (no version of the scenario has a default set, which is different to the example you gave above - so this only serves for clarfication when there is no default set)
scen = message_ix.Scenario(mp, "MESSAGEix-GLOBIOM_BvR_OFR1", "NPi2020_Net0")
__
2018-11-21 16:50:40,469  INFO at.ac.iiasa.ixmp.objects.Scenario:218 - loading Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' by version id (version: 2, runid: 18494)...
2018-11-21 16:50:40,669  INFO at.ac.iiasa.ixmp.objects.Scenario:241 - done loading Scenario from the database!
2018-11-21 16:50:40,693  INFO at.ac.iiasa.ixmp.objects.MsgScenario:457 -  spatial and temporal disaggregation maps assignment
2018-11-21 16:50:40,705  INFO at.ac.iiasa.ixmp.objects.MsgScenario:573 -  period mapping assignment
2018-11-21 16:50:40,710  INFO at.ac.iiasa.ixmp.objects.MsgScenario:599 -  duration period assignment
2018-11-21 16:50:40,718  INFO at.ac.iiasa.ixmp.objects.Scenario:1826 - committing changes of Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' to the database (runid: 18494)...
2018-11-21 16:50:41,525  INFO at.ac.iiasa.ixmp.objects.Scenario:601 - done updating Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' to the database (runid: 18494, version: 2)!
2018-11-21 16:50:41,653  INFO at.ac.iiasa.ixmp.objects.MsgScenario:457 -  spatial and temporal disaggregation maps assignment
2018-11-21 16:50:41,653  INFO at.ac.iiasa.ixmp.objects.MsgScenario:573 -  period mapping assignment
2018-11-21 16:50:41,654  INFO at.ac.iiasa.ixmp.objects.MsgScenario:599 -  duration period assignment
__
scen = message_ix.Scenario(mp, "MESSAGEix-GLOBIOM_BvR_OFR1", "NPi2020_Net0", version=2)
__
2018-11-21 16:50:41,686  INFO at.ac.iiasa.ixmp.objects.Scenario:218 - loading Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' by version id (version: 2, runid: 18494)...
2018-11-21 16:50:41,884  INFO at.ac.iiasa.ixmp.objects.Scenario:241 - done loading Scenario from the database!
2018-11-21 16:50:41,896  INFO at.ac.iiasa.ixmp.objects.MsgScenario:457 -  spatial and temporal disaggregation maps assignment
2018-11-21 16:50:41,906  INFO at.ac.iiasa.ixmp.objects.MsgScenario:573 -  period mapping assignment
2018-11-21 16:50:41,909  INFO at.ac.iiasa.ixmp.objects.MsgScenario:599 -  duration period assignment
2018-11-21 16:50:41,914  INFO at.ac.iiasa.ixmp.objects.Scenario:1826 - committing changes of Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' to the database (runid: 18494)...
2018-11-21 16:50:42,644  INFO at.ac.iiasa.ixmp.objects.Scenario:601 - done updating Scenario 'MESSAGEix-GLOBIOM_BvR_OFR1|NPi2020_Net0' to the database (runid: 18494, version: 2)!
2018-11-21 16:50:42,650  INFO at.ac.iiasa.ixmp.objects.MsgScenario:457 -  spatial and temporal disaggregation maps assignment
2018-11-21 16:50:42,650  INFO at.ac.iiasa.ixmp.objects.MsgScenario:573 -  period mapping assignment
2018-11-21 16:50:42,650  INFO at.ac.iiasa.ixmp.objects.MsgScenario:599 -  duration period assignment
__

@OFR-IIASA here is when it outputs to log and here is implementation of isDefaultVersion(runId) method

Hi all, I have created two separate issues brought up in this thread. If you think there are other actionable issues here, please open them as new issues. Closing now as we have found the root cause of the observed behavior.


Python: build a package take 1

I was able to get this installed with Pipfile using the following magic (since it's only on test.pypi.org right now):
› cat Pipfile
[[source]]
url = "https://test.pypi.org/simple/"
verify_ssl = true
name = "test"

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[dev-packages]
mypy = "*"

[packages]
cattrs = {version = "*",index = "pypi"}
LookerSDK = {version = "*",index = "test"}


Api to check server info/status

Hi,
there is a command or an api to check the server name if it is online and how many people there are, basically the info that one sees when opening the game and looking for the server name.
Thanks

I'm assuming you mean "Is there a command", in which case not yet.
You can see if the server process is running at least from the health info from docker status, but not the name nor the number of players online. I'm planning on adding that functionality later and providing the information through odin once I find the spare time (and if it gets merged).

Yes if there is a command but it could also be a public api from Valheim game which could allow us to perform a search for servers outside the game.
I think it's very convenient to quickly check if your server is online and working without entering the game or the server

At this time, there is no API to check if the server is running. This could potentially be added via the steam API and process watching which is something I have been planning to do but I havent stubbed out the requirements yet.

For reference, I have baked a healthcheck into the dockerfile. If you run docker ps you can see if its healthy or not.

I'm not sure if this is the same thing as this issue, but it would be great to have some sort of liveness probe so that I could easily run this properly in Kubernetes. It seems that [Kubernetes can't make use of the heathcheck in the Dockerfile].(https://github.com/kubernetes/kubernetes/issues/25829)

In other servers, like arkmanager, you can run a status check. Could an odin status be added?
I have a Discord bot for Ark that I use to run commands in docker to get server info by doing things like: docker exec -it arkmanager status right now. It would be great if I could do this with odin.

In other servers, like arkmanager, you can run a status check. Could an odin status be added? I think an odin status would provide the details being asked for here.
I have a Discord bot for Ark that I use to run commands in docker to get server info by doing things like: docker exec -it arkmanager status right now. It would be great if I could do this with odin.

I love arkmanager and I have built a container + web interface before for them :) Of which I agree, unfortunately though, Valheim does not support RCON nor does it have a good way to get information out of the server. Currently, the only way to get information about your server would be to query steams API but that would require everyone to generate an API key to access it. Even then though, I wouldnt be able to provide how many people are connected or anything quite yet.

I would love to help out and potentially create a PR that does it. But I have no idea how to query it.
I ran odin status within the docker and that's unavailable. Here are the available odin commands (when running within the container):
# odin help
odin 1.2.0
mbround18
Installs and Runs Valheim

USAGE:
    odin [FLAGS] [SUBCOMMAND]

FLAGS:
    -d, --debug      Sets the logger to log debug events.
    -r, --dry-run    Will output the commands that it intends to run.
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    backup       Backups the current saves to a specifc location
    configure    Initializes Odin with its configuration variables.
    help         Prints this message or the help of the given subcommand(s)
    install      Installs Valheim with steamcmd
    start        Starts Valheim
    stop         Stops Valheim

I would love to help out and potentially create a PR that does it. But I have no idea how to query it.
I ran odin status within the docker and that's unavailable. Here are the available odin commands (when running within the container):
# odin help
odin 1.2.0
mbround18
Installs and Runs Valheim

USAGE:
    odin [FLAGS] [SUBCOMMAND]

FLAGS:
    -d, --debug      Sets the logger to log debug events.
    -r, --dry-run    Will output the commands that it intends to run.
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    backup       Backups the current saves to a specifc location
    configure    Initializes Odin with its configuration variables.
    help         Prints this message or the help of the given subcommand(s)
    install      Installs Valheim with steamcmd
    start        Starts Valheim
    stop         Stops Valheim

Currently expected, this issue is to create the odin feature called status. What Valheim lacks is RCON or some other api to query the interface.

I'm not sure if this is the same thing as this issue, but it would be great to have some sort of liveness probe so that I could easily run this properly in Kubernetes. It seems that Kubernetes can't make use of the heathcheck in the Dockerfile.(kubernetes/kubernetes#25829)

At this current time there is no status check available other than pinging your game/query port for livelness

@mbround18 I mentioned it a bit in another issue, but I think we can get the player count and server name through a steam query and then we can also check if the server process' cpu and ram usage.
Would you be open to exposing this info through a odin status subcommand? If so then I can open a PR for it in a few days. It would also be nice to expose a --format json flag to make it easier for people to parse the output
The output could be something like
$ odin status
{{ NAME }} is online on {{ PORT }}
Version: {{ VALHEIM_VERSION }}
Player count: {{ NUM_PLAYERS }} / 10

I think that's the best status message we can provide without being able to extract more information from the server.

@mbround18 I mentioned it a bit in another issue, but I think we can get the player count and server name through a steam query and then we can also check if the server process' cpu and ram usage.
Would you be open to exposing this info through a odin status subcommand? If so then I can open a PR for it in a few days. It would also be nice to expose a --format json flag to make it easier for people to parse the output
The output could be something like
$ odin status
{{ NAME }} is online on {{ PORT }}
Version: {{ VALHEIM_VERSION }}
Player count: {{ NUM_PLAYERS }} / 10

I think that's the best status message we can provide without being able to extract more information from the server.

I dig it, I would suggest the output be something more akin to
Server Name: {{ NAME }}
IP Address: {{ IP:PORT }}
Version: {{ VALHEIM_VERSION }}
Players Online: {{ NUM_PLAYERS }} // I would keep it just the current count, some people use mods to alter the limit

BepInEx: {{ TRUE }} // only show if BepInEx is installed, useful for us to debug
Installed Mods: {{ List dlls in the BepInEx folder }} // Only show if BepInEx is installed

What Valheim lacks is RCON or some other api to query the interface.

Valheim lacks RCON, but you can absolutely send a valid A2S query to the server and it will respond with bunch of server information. I'm currently using it to display player count and server version on Discord. One thing that doesn't work yet is player list (it correctly show player count and even lists them, but player information is empty).
A2S reference: https://developer.valvesoftware.com/wiki/Server_queries
You can test it using for example: https://github.com/gamedig/node-gamedig
There is also RCON feature request on public bug tracker here: https://valheimbugs.featureupvote.com/suggestions/159703/feature-rcon-support-or-a-proper-server-console so you can upvote/comment on it. :)

@mhwcat that is what I was referring to in my comment above talking about steam queries :D

Doyou think its possible to perhaps add the status function or rcon ability through the BepInEx integration?  We might be able to build a plugin with that and use its hooks to query the info?  I'm just starting to explore BepInEx plugins, if I find a way to do that I will come back and write more on it.

Doyou think its possible to perhaps add the status function or rcon ability through the BepInEx integration? We might be able to build a plugin with that and use its hooks to query the info? I'm just starting to explore BepInEx plugins, if I find a way to do that I will come back and write more on it.

Absolutely, with bepinex you could gain any information about the game that youd like but Odin would not support it unless it met certain criteria. I can dive into the server status feature after this weekend as I will be out of town.

Check it,
Without --json

With --json

Command will exit with code 1 if it fails
You can test this feature early with the mbround18/valheim:development-status image but not I am going to be out of internet range until next week. I might be able to see some responses but it will be a bit before I can review them

Would it be possible to also export server status + player count in prometheus exposition format so this info can be scraped into a prometheus/grafana stack?

Would it be possible to also export server status + player count in prometheus exposition format so this info can be scraped into a prometheus/grafana stack?
Example:
http://localhost:2459/metrics

valheim_player_count{name="Created With Valheim Docker", version="<IP_ADDRESS>"} = 10
valheim_up{name="Created With Valheim Docker", version="<IP_ADDRESS>"} = 1

Could you provide more context? Would this just need to be a simple http server that displays text similar to your bullet points ?

Could you provide more context? Would this just need to be a simple http server that displays text similar to your bullet points ?
Yep, that is exactly how it would work. Prometheus is an industry standard for collecting and monitoring metrics for web services, supported by the CNCF. There are tons of well maintained opensource libraries that let you capture metrics and expose in the prometheus format, along with server framework plugins to help stand up the http server.

Additionally, I feel like this is something outside the scope of this project and if you are running on kubernetes, docker compose, etc. You could bring in another container to run this sort of metrics gathering process.

This is true that this could be achieved via a separate container. If that is the direction you think this should go, I'd suggest looking at this repo as an example on how this could be done with dockerized minecraft.
https://github.com/itzg/mc-monitor
Prometheus does have a middleware plugin for expressJS,  so your idea of using nodeJS/Express/ShellJS would probably be pretty easy to setup.

The thing I see there is that the Bedrock MC servers use RakNet protocol. For the Valheim servers, it's either a2s, or we implement a custom BepInEx plugin that exposes an actually useful API.
I kind of think that is the way to go. A framework to go around the basic Valheim server that implements the APIs and protocols needed to correctly monitor and manage a server. I don't think the Valheim devs are going to get around to doing that anytime soon.
I'm more of a golang guy, and so creating a c# plugin along the lines of BePinEx is not something I can dive into, but I am hoping someone creates something.  I see a lot of quick and dirty plugins/mods happening, but not really seeing anyone think longer term.

Hi,
there is a command or an api to check the server name if it is online and how many people there are, basically the info that one sees when opening the game and looking for the server name.
Thanks

I'm running a small sidecar container [1] along the gameserver which polls my server regularly using gamedig and provides this data as JSON via http and optionally sends a Telegram message if someone joines or leaves the server.
[1] https://github.com/Aldjinn/valheim-server-status

Keep in mind gamedig uses Valves a2s protocol and there are some issues with the reliability of the info specified with that.  For example, the players count, and coming/going is not always accurate.  the version is wrong, but the right version is in extra data keywords field.  The rules query doesnt work at all.
I'm hoping the devs put the version stuff right and fix what ever is making the player stuff unreliable (works sometimes, othertimes not to much from what I've seen in discussions on Discord.)

I've noticed those issues too, and I hope they get fixed sometime. Also there is the option to query for player information, but currently the only values I see are connected time which doesn't always seem accurate.
Still definitely better than nothing!

It seems like after 0.147.3 update A2S queries stopped working. Has anybody had any luck with querying server now? For me neither server browser, nor gamedig or custom queries work now. :(

it still works for me.
Le jeu. 4 mars 2021 à 12:50, mhwcat<EMAIL_ADDRESS>a écrit :

It seems like after 0.147.3 update A2S queries stopped working. Has
anybody had any luck with querying server now? For me neither server
browser, nor gamedig or custom queries work now. :(
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/mbround18/valheim-docker/issues/135#issuecomment-790557783,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAPUYMPFVMZIZ2UAJOMEC43TB5XZHANCNFSM4X2ALWPQ
.

--
Cordialement,
CLEMENT Giovanni.

For me, a gamedig query is still working fine:
{
	"name": "TheBigBadWolf",
	"map": "TheBigBadWolf",
	"password": true,
	"raw": {
		"protocol": 17,
		"folder": "valheim",
		"game": "",
		"appId": 892970,
		"numplayers": 1,
		"numbots": 0,
		"listentype": "d",
		"environment": "l",
		"secure": 0,
		"version": "<IP_ADDRESS>",
		"steamid": "92143726772729609",
		"tags": "0.147.3",
		"rules": {}
	},
	"maxplayers": 64,
	"players": [
		{
			"name": "",
			"raw": {}
		}
	],
	"bots": [],
	"connect": "valheim.example.com:2456",
	"ping": 39
}

I don't know if this can help but this is the Steam API I am using to know if my server's running/if players are online :
https://api.steampowered.com/IGameServersService/GetServerList/v1/ filter=\gamedir\valheim\name_match\*yout_server_name*&key=your_steam_api_key

@bforbis  I added this in #298 :) It has an output of something like:
{
  "name":"Fancy Name",
<EMAIL_ADDRESS>  "players":0,
  "max_players":10,
  "map":"Fancy Name",
  "online":true,
  "bepinex":{
    "enabled":true,
    "mods":[
      {
        "name":"ValheimPlus.dll",
        "location":"/home/steam/valheim/BepInEx/plugins/ValheimPlus.dll"
      }
    ]
  },
  "jobs":[
    {
      "name":"AUTO_UPDATE",
      "enabled":false,
      "schedule":"*/5 * * * *"
    },
    {
      "name":"AUTO_BACKUP",
      "enabled":true,
      "schedule":"*/5 * * * *"
    }
  ]
}

Very cool. I pulled down valheim:development-status and am playing around with it now. Some things I noticed:

running odin status in the container prints out incorrect info sometimes.

Not actually sure whats going on, but it is printing out an old server name. I pruned all my containers / volumes and restarted using a completely new "NAME" environment, however execing into the container and running odin status is somehow still tied to the old server (on steam) and is reporting the old server name and player count

Running with env HTTP_PORT=2559, I did not see the http server start.

Once the http server is running, are you planning on adding in prometheus metrics format, or I work on a sidecar to convert to prometheus?

@bforbis thats expected, I have not updated the image on dockerhub for a lil while. Ill push an updated image for ya to take a look at.

@bforbis pull down the image
mbround18/valheim:135-http-server-check

I can write a metrics http endpoint which uses status but I need help on formatting. Would something like this work:
valheim_online{name="Fancy Name"<EMAIL_ADDRESS>map="Fancy Name"} = 1
valheim_player_count{name="Fancy Name", max_players=10} = 0
valheim_bepinex_installed{name="Fancy Name"} 1
valheim_auto_update{name="Fancy Name", schedule="*/5 * * * *"} 1
valheim_auto_backup{name="Fancy Name", schedule="*/5 * * * *"} 1

@bforbis pull down the image
mbround18/valheim:135-http-server-check

I can write a metrics http endpoint which uses status but I need help on formatting. Would something like this work:
valheim_online{name="Fancy Name"<EMAIL_ADDRESS>map="Fancy Name"} = 1
valheim_player_count{name="Fancy Name", max_players=10} = 0
valheim_bepinex_installed{name="Fancy Name"} 1

Hi, I'm not @bforbis but I can offer some drive by Prometheus experience.
Prometheus labels aren't quite like key/value pairs, though that's what they appear at first glance. You want to keep the range of values fairly constrained and enumerable, each specific key/value combination is its own time series (the Prometheus docs will call this "cardinality"). A general rule of thumb is to avoid non-enum based interpolation.
So I would suggest something like:
valheim_online{name="Fancy Name"<EMAIL_ADDRESS>map="Fancy Name"} 1
valheim_current_player_count{name="Fancy Name"<EMAIL_ADDRESS>map="Fancy Name"} 0
valheim_max_player_count{name="Fancy Name"<EMAIL_ADDRESS>map="Fancy Name"} 10
valheim_bepinex_installed{name="Fancy Name"<EMAIL_ADDRESS>map="Fancy Name"} 1

valheim_auto_update{name="Fancy Name", schedule="*/5 * * * *"} 1
valheim_auto_backup{name="Fancy Name", schedule="*/5 * * * *"} 1

The schedule value is user provided and thus will create a new time series every time the user configures a different value.
If you wanted to go the extra mile, you could parse this out into an interval like:
valheim_auto_update_seconds{name="Fancy Name"<EMAIL_ADDRESS>map="Fancy Name"} 300
valheim_auto_backup_seconds{name="Fancy Name"<EMAIL_ADDRESS>map="Fancy Name"} 300

... but this assumes that the cron schedule is something convertable to a fixed number of seconds (which it is not guaranteed to be).
I'd suggest avoiding these metrics for now until there's a strong POV on what they should look like.

@rraval how is this?
When server is online,

When server is offline but docker is still running,

Here's the output from a prom express server I was testing against your http endpoint. This was generated using the nodeJS prom-client, so should be an accurate format. As a note, you could also pull down the official prometheus rust client to generate this too if you want
# HELP valheim_player_count The number of active players
# TYPE valheim_player_count gauge
valheim_player_count{name="ValheimPromTest",version="0.147.3"} 0

# HELP valheim_max_player_count The maximum number of players
# TYPE valheim_max_player_count gauge
valheim_max_player_count{name="ValheimPromTest",version="0.147.3"} 64

# HELP valheim_up Tracks if valheim server is online
# TYPE valheim_up gauge
valheim_up{name="ValheimPromTest",version="0.147.3"} 1
valheim_up 0

You should be able to test this locally with prometheus. Add this to your docker-compose:
  prometheus:
    image: prom/prometheus:v2.23.0
    container_name: prometheus
    restart: always
    command:
      - "--config.file=/etc/prometheus/prometheus.yml"
      - "--storage.tsdb.path=/prometheus"
      - "--storage.tsdb.retention.time=7d"
      - "--web.console.libraries=/usr/share/prometheus/console_libraries"
      - "--web.console.templates=/usr/share/prometheus/consoles"
    ports:
      - 9090:9090/tcp
    volumes:
    - prometheus:/prometheus:rw
    - ./configs/prometheus.yml:/etc/prometheus/prometheus.yml:ro

With this as your bind mounted prometheus.yml
global:
  scrape_interval: 15s
  scrape_timeout: 10s
  evaluation_interval: 1m
scrape_configs:
  - job_name: valheim
    static_configs:
       # Assuming docker bridge networking, replace "valheim:8080" with your valheim service name in docker-compose and whatever HTTP port you have set
      - targets: ['valheim:8080']

You can then graph the metrics in prometheus by navigating to http://localhost:9090/classic/graph

Alright tested with prometheus, this looks good from what I can tell:
valheim_online{name="Creative Update"<EMAIL_ADDRESS>map="Creative Update"} 1
valheim_current_player_count{name="Creative Update"<EMAIL_ADDRESS>map="Creative Update"} 0
valheim_max_player_count{name="Creative Update"<EMAIL_ADDRESS>map="Creative Update"} 10
valheim_bepinex_installed{name="Creative Update"<EMAIL_ADDRESS>map="Creative Update"} 1

@mbround18 I pulled down your new version and things look good. I uploaded a sample Grafana dashboard here:
https://grafana.com/grafana/dashboards/14123
This uses a combination of your new valheim metrics and some generic cpu/memory metrics from cAdvisor


[tern addon] Add caseInsensitive autocompletion option

allow caseInsensitive option eg.
 server = new CodeMirror.TernServer({caseInsensitive: true});

caseInsensitive is passed to tern request query which allow case insensitive autocomplete searches

relates to https://github.com/marijnh/tern/issues/163

Just a suggestion. Maybe caseSensitive true by default would be more clear.

hmm yes, double negatives... i used caseInsensitive following ternjs's change. let's see what @marijnh thinks :)

Please take a look at the attached patch instead. It adds a more general way to add options to queries. You'd pass it queryOptions: {completions: {caseInsensitive: true}} to get the effect you want. Does this work for you?
As for the double negative, I agree that's bad style, but that ship has sailed (it is part of the Tern API, and I don't consider it a big enough problem to change it now).

@marijnh hmm. let me test that.
but from my first looks it seems like queryOptions: {completions: {caseInsensitive: true}} might override any properties already in query.completions, unless usage of .request() ever pass in a completions property.
do you think it would be better to have a query hook filter option which takes a function instead (which might be even more flexible)?

quick test with queryOptions: {completions: {caseInsensitive: true}}  works.

I am not sure I understand your concern. Yes, queryOptions could override properties that the library itself added, and you could pass something like {type: "foo"} and break the library. But why would you?

maybe my concerns are unfounded, but looks like this issue has been resolve, so I'll close this. Thanks!


Various enhancements

Modifications:

eliminate unnecessary variables, functions
use ECMA_NUMBER macros where it is possible
simplify code
minor style fix (comments, increase-decrease operators)

JerryScript-DCO-1.0-Signed-off-by: Zsolt Borbély<EMAIL_ADDRESS>

RPi2 perf results
master (https://github.com/Samsung/jerryscript/commit/9d7978074f0ed301fb95a6faf79ee8b1a332ec9b)
                   Benchmark |                  Perf(+ is better)
                  3d-cube.js |  2.637s ->   2.618s :  +0.696%  (+-0.887%) : [≈]
      access-binary-trees.js |  1.410s ->   1.408s :  +0.118%  (+-0.477%) : [≈]
          access-fannkuch.js |  7.982s ->   7.973s :  +0.104%  (+-0.584%) : [≈]
             access-nbody.js |  3.083s ->   3.152s :  -2.216%  (+-0.596%) : [-]
 bitops-3bit-bits-in-byte.js |  1.733s ->   1.727s :  +0.384%  (+-0.692%) : [≈]
      bitops-bits-in-byte.js |  2.655s ->   2.657s :  -0.063%  (+-0.467%) : [≈]
       bitops-bitwise-and.js |  3.683s ->   3.738s :  -1.493%  (+-0.703%) : [-]
    controlflow-recursive.js |  0.890s ->   0.893s :  -0.374%  (+-0.955%) : [≈]
               crypto-aes.js |  4.720s ->   4.668s :  +1.095%  (+-0.675%) : [+]
               crypto-md5.js | 21.537s ->  21.497s :  +0.186%  (+-0.505%) : [≈]
              crypto-sha1.js |  9.725s ->   9.743s :  -0.188%  (+-0.825%) : [≈]
        date-format-tofte.js |  2.230s ->   2.210s :  +0.897%  (+-1.138%) : [≈]
        date-format-xparb.js |  1.080s ->   1.073s :  +0.618%  (+-0.787%) : [≈]
              math-cordic.js |  2.993s ->   3.000s :  -0.223%  (+-0.666%) : [≈]
        math-partial-sums.js |  1.665s ->   1.677s :  -0.701%  (+-0.747%) : [≈]
       math-spectral-norm.js |  1.513s ->   1.495s :  +1.211%  (+-1.061%) : [+]
             string-fasta.js |  4.555s ->   4.530s :  +0.549%  (+-1.263%) : [≈]
             Geometric mean: |                Speed up: 0.039% (+-0.194%) : [≈]

And in general, the file headers should be double-checked. The headers of some now-touched files bear a 2014 copyright.

@akiss77 @LaszloLango The PR is updated according the review.

Thanks. LGTM (FWIW)

It is strange that 'ecma_is_ex_string_magic' is not used in Jerry. May be removing it not the best solution.  Isn't this feature needed for IoT.js?

I didn't find any reference to this function in IoT.js, but there's a chance it will be used in the future, I agree with you, we should keep it. So I will update the PR.

@bzsolt, could you raise an issue for this?

After fix of https://github.com/Samsung/jerryscript/pull/914#discussion_r53811948 the patch LGTM.

@LaszloLango I've updated the PR. I've kept the external magic string related functions, and raised an issue for this topic: #922

I would call the patch "small refactorings".
LGTM


[Bug]:

Feature id and version
No response
Devcontainer base Image
No response
What happened?
A bug happened!
Relevant log output
No response

This pretty much looks like spam.
@seda1451 do you have any specific information about what you think is "bugged"? There are absolutely 0 details....

This project is not associated with the devcontainer project and appears to have been abandoned https://github.com/devcontainers-contrib/features/issues/628 . The official project has removed all devcontainer-contrib features from their listing. https://github.com/devcontainers/devcontainers.github.io/issues/451. There is a fork that is being worked on to update and maintain the features https://github.com/devcontainers-extra/features


build: lock execa to >=2.0.0

Before submitting this PR:
Proposing to upgrade   execa version to >=2.0.0  to mitigate this vulnerability  discovered via docker scout quickview nhost/hasura-auth:0.28.0 scan.
https://scout.docker.com/vulnerabilities/id/GMS-2020-2?s=gitlab&n=execa&t=npm&vr=<2.0.0
Checklist

[x] No breaking changes
[x] Tests pass
[ ] New features have new tests
[ ] Documentation is updated

Breaking changes
Avoid breaking changes and regressions. If you feel it is unavoidable, make it explicit in your PR comment so we can review it and see how to handle it.
Tests

please make sure your changes pass the current tests (Use the make test or the make watch command).
if you are introducing a new feature, please write as much tests as possible.

Documentation
Please make sure the documentation is updated accordingly, in particular:

Workflows. Workflows are Mermaid sequence diagrams
Schema. The schema in a Mermaid ER diagram
Environment variables. Please adjust the .env.example file accordingly
OpenApi specifications. We are using inline JSDoc annotations

Closing/Reopening as that should fix the permissions error.

I don't know why github "emptied" the commit when I rebased it... Anyway, opened #500 with your commit


No tsconfig support for strictNullChecks

TypeScript Version:
1.8.10
Code
{
    "compilerOptions": {
        "strictNullChecks": true
    }
}

Expected behavior:
compile with strictNullChecks.
Actual behavior:
This is a valid compiler option in https://www.typescriptlang.org/docs/handbook/compiler-options.html but does not exist in the schema http://json.schemastore.org/tsconfig and thus results in error TS5023: Unknown compiler option 'strictNullChecks'.

The handbook is jumping the gun, I think. That option is not in the current released version (1.8.10).

We should have the documentation covered by the time TS 2.0 ships out. see https://github.com/Microsoft/TypeScript-Handbook/issues/294

I will remove this option for now.

see https://github.com/Microsoft/TypeScript-Handbook/commit/86fa56d4f9fddb1be86e9d822d78f12c89eabe83

@mhegazy, https://www.typescriptlang.org/docs/handbook/compiler-options.html still have strictNullChecks option.

This is still an issue.

2.0 is still not released.

We update the sire in bulk. so we are waiting on the TS final release to move all docs to the 2.0 version (latest from https://github.com/Microsoft/TypeScript-Handbook). sorry for the delay.

I still get this error in version 2.0.3:
Error: Parse tsconfig error [{"messageText":"Unknown compiler option 'strictNullChecks'.","category":1,"code":5023}]
This is my tsconfig:
{
"compilerOptions": {
"module": "commonjs",
"target": "es5",
"sourceMap": true,
"strictNullChecks": true
},
"exclude": [
"node_modules"
],
"files": [
"test.ts"
]
}

should be updated now.


A flexible way to specify prefetch files to pull Nydus images

Users defined prefetch files list, which can be set to --prefetch files when starting the nydus daemon.
Nydus has implemented the addition of the -- prefetch-patterns parameter to the nydusify convert command. With this parameter, we can specify a prefetch list when converting nydus format images and pull files based on this prefetch list when starting a container. However, due to different services based on one image need to access different files, this method is not flexible enough. Here is another idea, users can customize prefetch lists and pull according to the required prefetch files when pulling images.
Requirements

NRI 2.0: Which has been integrated into containerd since v1.7.0-beta.1.

WorkFlow

Create a pod.yaml file. Define the user-defined prefetch list file path in the labels or annotation field.

sudo tee pod.yaml <<- EOF
metadata:
  name: busybox-sandbox
  namespace: default
  attempt: 1
  uid: hdishd83djaidwnduwk28bcsb
log_directory: /tmp
labels:
  prefetch_list: /path/to/prefetchList.txt
linux: {}
EOF

Pod creation
During the pod creation period, complete the following process.

Start an NRI plugin and subscribe to pod creation event. Note that NRI plugin can only be called from containerd/CRI. So creation a pod using crictl as below.

 crictl runp pod.yaml

NRI plugin invokes the RunPodSandbox method to obtain image name and prefetch list file path when monitoring the pod creation command.
NRI plugin sends image name + prefetch list file path as a massage to snapshotter through the specified socket API.
After receiving the massage sent by NRI, the snapshotter will add a handler to parse the message and temporarily store the result as a global variable.

Pull image
When pulling an image, containerd will notify snapshot to start a new daemon. When starting nydusd, obtain the prefetch list file path from the global variable and add it to the -- prefetch files parameter.
After completing the startup of nydusd, identify the corresponding image name when pulling the image, and then pull according to the prefetch list.

That's a reasonable extension, what do you think? cc @changweige

SGTM to me. My only suggestion is to add the designated KV to annotations rather than POD label since the label is used for node selection and affinity.

LGTM. I think there are three things we should consider.

How to implement a nydus snapshotter API to receive image+prepfetch list file path?
How to store the image+prepfetch list file path? In snapshotter database or memory cache?
May we also need API to UPDATE/REMOVE image+prepfetch list file path?

Another question is how do we upload the prefetch list file to every machine in the cluster?


Rename mocha desktop/app tests to use *.test extension

Follow up to https://github.com/artsy/force/pull/3877

Updates file names for all mocha tests in the src/desktop/app directory to use *.test.*
x's any broken tests

One more PR to come for the /mobile directory...

Thank you for starting this! Let me know if I can do anything to help on cleanup or reenabling tests.

I'm having some trouble getting the desktop/apps/auctions tests to run correctly, we can merge https://github.com/artsy/force/pull/3882 first and I will continue to debug.


Different keyboard layout?

I have a German keyboard layout. Would it be possible to change the layout so that the keys are arranged in the same way as my keyboard?

I've never seen a German keyboard, but I believe it shouldn't be too hard since I don't need to reposition the key buttons, only swap the buttons. Tell me, what keyboard layout your keyboard uses?

Here is the German keyboard layout I'm interested in right now.
On the Keyboard Layout Info page you will find many other layouts. Maybe you can think of way to support different keyboard layouts programmatically.

Interesting. So, the original Star Citizen's keyboard layout I'm replicating from seems to follow the ANSI layout. And there are ISO and JIS layout. It's not going to be a simple matter of swapping the keys from QWERTY layout to QWERTZ layout. I'll have to create new layout for ISO and JIS.
I guess those are good additions to the tool!


Fixes #2653

please update the title to something more specific, e.g. fix arithmeticShiftRight copy-paste issue: >> vs << when merging


Integration of WASI-NN

Initial integration of WASI-NN based on https://github.com/bytecodealliance/wasm-micro-runtime/pull/1225.

TODO:

[ ] Specify limitations: only support of tflite, etc.
[ ] Improve safety checks. For example, using invalid formats.

My team is planning to continuously support WASI-NN by adding support to GPU, etc.
@wenyongh how do we proceed with this? I have several questions:

Right now we haven't added this to the CI.
We are supporting only a subset of WASI-NN features.

My team is planning to continuously support WASI-NN by adding support to GPU, etc. @wenyongh how do we proceed with this? I have several questions:

Right now we haven't added this to the CI.
We are supporting only a subset of WASI-NN features.

@tonibofarull Sounds like it is not a short term work and may take some time to implement all the features, right? Normally we will develop the feature under a development branch (like dev/socket, dev/refactor_interp_module_inst), and periodically merge code from that branch into main branch when a milestone is reached. I have created a branch dev/wasi_nn, if there are more features to implement, how about submitting the PR to that branch? And if a milestone is reached and you want to merge it into main, please notify us.
And could you help file an issue to describe the feature we want to you implement, including motivation, changes, milestones and so on? Here are some samples:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1336
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1384

Thank you, I'll take a look!

And could you help file an issue to describe the feature you want to implement, including motivation, changes, milestones and so on? Here are several samples:

Let me address your comments before we discuss this :bow:

I've fixed all the previous comments!

And could you help file an issue to describe the feature you want to implement, including motivation, changes, milestones and so on?

I've created the document here.

I have created a branch dev/wasi_nn, if there are more features to implement, how about submitting the PR to that branch? And if a milestone is reached and you want to merge it into main, please notify us. (Or merge this PR to main, and the next PRs to dev/wasi_nn?)

As stated in the issue, the first milestone corresponds to support TensorFlow in CPU and FP32, which are the features implemented in this PR.
Having the dev branch makes sense for having the future intermediate results!

I've fixed all the previous comments!

And could you help file an issue to describe the feature you want to implement, including motivation, changes, milestones and so on?

I've created the issue here.

I have created a branch dev/wasi_nn, if there are more features to implement, how about submitting the PR to that branch? And if a milestone is reached and you want to merge it into main, please notify us. (Or merge this PR to main, and the next PRs to dev/wasi_nn?)

As stated in the issue, the first milestone corresponds to support TensorFlow in CPU and FP32, which are the features implemented in this PR.
Having the dev branch makes sense for having the future intermediate results!

Great! We will read the code again, many thanks!

@tonibofarull Many thanks for the fixing, I have merged the PR.
And I tried to compile the source code, and found that there are many compile warnings, could you help clear them?
And will you add a sample/wasi-nn in the next stage? Thanks.
[  8%] Building C object CMakeFiles/libiwasm.dir/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/random.c.o
In file included from /home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:16:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:77:5: warning: ‘export_module’ attribute directive ignored [-Wattributes]
77 |     attribute((import_module("wasi_nn")));
|     ^~~~~~~~~~~~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:77:5: warning: ‘import_module’ attribute directive ignored [-Wattributes]
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:89:5: warning: ‘export_module’ attribute directive ignored [-Wattributes]
89 |     attribute((import_module("wasi_nn")));
|     ^~~~~~~~~~~~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:89:5: warning: ‘import_module’ attribute directive ignored [-Wattributes]
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:102:5: warning: ‘export_module’ attribute directive ignored [-Wattributes]
102 |     attribute((import_module("wasi_nn")));
|     ^~~~~~~~~~~~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:102:5: warning: ‘import_module’ attribute directive ignored [-Wattributes]
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:112:1: warning: ‘export_module’ attribute directive ignored [-Wattributes]
112 | attribute((import_module("wasi_nn")));
| ^~~~~~~~~~~~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:112:1: warning: ‘import_module’ attribute directive ignored [-Wattributes]
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:130:5: warning: ‘export_module’ attribute directive ignored [-Wattributes]
130 |     attribute((import_module("wasi_nn")));
|     ^~~~~~~~~~~~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:130:5: warning: ‘import_module’ attribute directive ignored [-Wattributes]
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c: In function ‘wasi_nn_load’:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:68:71: warning: declaration of ‘graph’ shadows a global declaration [-Wshadow]
68 |              graph_encoding encoding, execution_target target, graph *graph)
|                                                                ~^
In file included from /home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:12:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_common.h:39:18: note: shadowed declaration is here
39 | typedef uint32_t graph;
|                  ^~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:95:23: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
95 |     for (int i = 0; i < builder->size; ++i) {
|                       ^
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c: In function ‘wasi_nn_init_execution_context’:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:140:64: warning: declaration of ‘graph’ shadows a global declaration [-Wshadow]
140 | wasi_nn_init_execution_context(wasm_exec_env_t exec_env, graph graph,
|                                                          ^
In file included from /home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:12:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_common.h:39:18: note: shadowed declaration is here
39 | typedef uint32_t graph;
|                  ^~~
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c: In function ‘wasi_nn_set_input’:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:194:23: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
194 |     for (int i = 0; i < dimensions.size; ++i) {
|                       ^
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:204:12: warning: declaration of ‘tensor’ shadows a global declaration [-Wshadow]
204 |     tensor tensor = { .type = input_tensor->type,
|            ^~~~~~
In file included from /home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn_native.c:16:
/home/agent/WASM/wamr-internal-test/wamr-ba/core/iwasm/libraries/wasi-nn/wasi_nn.h:62:3: note: shadowed declaration is here
62 | } tensor;


Rename helpers in account_testutil.go

Makes the naming more correct/consistent.

Codecov Report

Merging #617 (3e19ec6) into develop (620982f) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #617   +/-   ##
========================================
  Coverage    49.82%   49.82%
========================================
  Files           25       25
  Lines         3952     3952
========================================
  Hits          1969     1969
  Misses        1702     1702
  Partials       281      281

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 620982f...3e19ec6. Read the comment docs.