xgboost saving (into textfile and binary) and then loading from xgb and then saving to text) not identical

Hi,
I am trying to use xgboost text files to score my data in java. I save the trained model x in 2 type of files first dump_model and second using save_model. When i load the second type of file into python and save using dump_model again, i see there are differences in definition. I have attached 2 text files, xgb.model.orig.txt is the file saved using original model and xgb.model.txt is the file saved from binary loaded into python. There are minute differences in nodesplit values, ex. there is +-0.5 values appended to split value. Please let me know if it's a known bug, i have attached the code below and xgboost model definitions, for this issue.
Please find my code to train a sample model and then save it into binary and text formats,
import pandas as pd
import xgboost as xgb
import numpy as np
from time import gmtime, strftime

hadoop_xgboost_scoring_path="/Users/ydalal/hadoop_xgboost_scoring"
run_id=strftime("%Y%m%d-%H%M%S", gmtime())

#   GENERATE RANDOM TRAINING DATA
min_val=1
max_val=100
df = pd.DataFrame(np.random.randint(min_val,max_val,size=(1000, 10)),
                  columns=['fname1','fname2','fname3','fname4','fname5','fname6','fname7','fname8','fname9','fname10'])
dtrain = xgb.DMatrix( df.ix[1:500,:], label=pd.DataFrame(np.random.randint(2, size=500))) # First 500 records for training
dtest = xgb.DMatrix( df.ix[501:,:], label=pd.DataFrame(np.random.randint(2, size=499))) # Rest record for tuning
#   Train the xgb model
num_round = 40
max_depth=4
param = {'bst:max_depth':max_depth, 'bst:eta':1, 'silent':1, 'objective':'binary:logistic'}
param['nthread'] = 4
#param['eval_metric'] = 'logloss'
plst = param.items()
evallist  = [(dtest,'eval'), (dtrain,'train')]
bst = xgb.train( plst, dtrain, num_round, evallist)

#   Save the xgb model
model_output=hadoop_xgboost_scoring_path+"/resources/%s%s"%(run_id,".xgb.model.txt")
print(" Saving model to , %s"%(model_output))
bst.dump_model(model_output)
bst.save_model(model_output.replace("txt","xgb"))

#   Generate Validation data
dvalidation= pd.DataFrame(np.random.randint(0,100,size=(10000, 10)), columns=['fname1','fname2','fname3','fname4','fname5','fname6','fname7','fname8','fname9','fname10'])
dval_output=hadoop_xgboost_scoring_path+"/resources/%s%s"%(run_id,".xgb.valdata.txt")
print("Saving validation data to, %s"%(dval_output))
dvalidation.to_csv(dval_output,index=False)

#   Score Validation data
scores=pd.DataFrame(bst.predict(xgb.DMatrix(dvalidation),output_margin=False,ntree_limit=0))
dval_scores_output=hadoop_xgboost_scoring_path+"/resources/%s%s"%(run_id,".xgb.valdata.scores.txt")
print("Saving validation data scores to, %s"%(dval_scores_output))
scores.to_csv(dval_scores_output,index=False)

#   Information
print("\n\n\n\tThis run generated following files : \n\tRunId- %s\n\t%s\n\t%s\n\t%s\n\t%s"%(run_id,model_output,dval_output,dval_scores_output,model_output.replace("txt","xgb")))

20160517-022638.xgb.model.txt
20160517-022638.xgb.model.orig.txt

You see such differences b/c first you dump & save a Booster which contains non-float feature_types (you have int features), and then, I guess, you load it into a new Booster without making it aware of the feature types. However, the save_model/load_model methods don't serialize feature types. When feature types are known to a Booster, its dump_model method would result in integer features printed in a special way:
https://github.com/dmlc/xgboost/blob/51154f42fe0c09c9597b639d086f69cdaa5c6fa8/src/tree/tree_model.cc#L46
When no feature names/types are stored, dump_model would dump the splits as floats with the standard float printout precision.
A good and simple approach is to always use float features. Otherwise, if you save a model with different feature types, then it's better to make sure to restore the types too. You can do it either by passing a dataset of the same types via the cache= parameter to Booster() when loading a model, or by explicitly setting the feature_names and feature_types properties before running the dump_model again.

Thanks Vadim,
It works with float features.


clients/public: prevent org description from overflowing on public pages

Mind if we do a break-word here instead of break-all?


Have a template of grafana dashboard for monitoring emq？

I am using grafana + prometheus monitoring emq，but there is no avaiable template, any suggest?

Hi I'v created a dashboard with nuvo's great exporter.
Dashboard id 9963
Hope you like it

How to get emqx cluster other nodes


Add message to user when spinal cord is not detected

Lots of tests are crashing, because the spinal cord is not detected in sct_propseg.
It would be fairly straightforward to detect the crash (with a try/except) and exit the program with exit status 0 and a printout that tells the user that "spinal cord could not be detected, please do a manual initialization using -init-mask viewer, or -init-centerline viewer"?

@benjamindeleener: I cannot assume that if file does not exist it implies that detection failed, because an older segmentation can already be there (false negative).
instead, i am trying to use status from isct_propseg. With detection failing, i get status=-11: is that specific to failed detection? Is it cross-OS compatible?

When the spinal cord is not detected in isct_propseg, the following code is called:
cerr << "Error: Unable to detect the spinal cord. Please provide the initial position and orientation of the spinal cord (-init, -init-mask)" << endl;
return EXIT_FAILURE;

The EXIT_FAILURE should be cross-platform as it is the standard C++ failure termination code (http://www.cplusplus.com/reference/cstdlib/EXIT_FAILURE/) but I don't know if the status code is always the same. Moreover, the EXIT_FAILURE is also called in several other failure cases (e.g., wrong input parameters, invalid input contrast (t1 and t2), help, failure when reading the image, error during symmetry computation, wrong initialization). In all cases, the same status code is called and can be detected.

ok so i'll just exit as warning (instead of error) and display the output message

since we want to exit with an error anyway (we need to detect crash for testing), i will also modify isct_test_function to remove the verbose of crashed function (we can examine individual verbose in output.txt)

I fully agree with that.

OptiC always return a centerline (on all slices), even if it has failed. So this issue is not relevant anymore.


Crash

prim I
prim left : I
prim right : I
open struct Path (A : I -> Type) (a : A left) (b : A right) : Type
  | at (i : I) : A i {
    | left => a
    | right => b
  }
def path {A : I -> Type} (p : Pi (i : I) -> A i)
  => new Path A (p left) (p right) { | at i => p i }

variable A B : Type
def infix = (a b : A) => Path (\x => A) a b
def idp {a : A} : a = a => path (\x => a)
def funExt (f g : A -> B) (p : forall a -> f a = g a) : f = g

This code crashes the compiler. The code is wrong because funExt has nobody nobody but you!

Hmmmmmmmmmmmmmmmmmmmmmmmmmmm


chore(deps): upgrade runtime dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-runtime-dependencies-1.x" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 2795

Questions ?
Please refer to the Backport tool documentation


'abi.encodeWithSignature' shows issue in remix-tests plugin

Sample Contract can be:
contract test_1{
    bytes public by;
    function add(uint256 first, uint256 second) public {
        require(first == second, "First should be equal with second.");
  }
function check1() public{
    uint256 u = 3;
    bytes memory b = abi.encodeWithSignature("add(uint256,uint256)", u, u);
    by = b;
}
}

This compiles, deploys and shows correct output in Deploy & Run but doesn't run in Solidity Unit Testing plugin. I shows error in console:
Uncaught Error: Invalid number of parameters for "add". Got 0 expected 2!
    at Object.InvalidNumberOfParams (app.js:25609)
    at Object._createTxObject (app.js:26976)
    at app.js:19852
    at replenish (app.js:1037)
    at app.js:1039
    at Object.eachOfLimit (app.js:1040)
    at Object.runTest (app.js:19850)
    at app.js:19829
    at replenish (app.js:1037)
    at app.js:1039
    at Object.eachOfLimit (app.js:1040)
    at runTests (app.js:19829)
    at nextTask (app.js:1204)
    at next (app.js:1206)
    at app.js:1030
    at determineTestContractsToRun (app.js:19828)

This is not a problem with abi.encodeWithSignature. Actually remix-tests module expects that methods inside a test contract does not have any parameters.
See: https://github.com/ethereum/remix/blob/master/remix-tests/src/testRunner.ts#L172
I think that is fine as we don't have a functionality to pass parameters values in Solidity Unit Testing plugin. what do you think? @yann300

Yes that's ok, but I think we should document it and find a good way to not stop execution in that case or just forbid for the moment executing test if one function has parameter.


Multi Directional Death animation

Some enemies (Phatmaster, mighty kobra, and dombie) have death animations for only one directional, we should make them multi directional such as golden tiger

Got the art from emanual

Made into .pngs, just need to add in

Finished via 199eaf7e309d2de5356fad63fbddcf47608779c8


Feature/custom app

Refactored
Style: add background image and
color adjustments;
Build: update mini-css-extract-plugin with yarn;
Code: Condition in profile component to lack of information.

Refactored
Style: add background image and
color adjustments;
Build: update mini-css-extract-plugin with yarn;
Code: Condition in profile component to lack of information.


Comments not working

There should be an ability to comment out code. For me this is not working.

@datavistics. hello! Can you please give more complex explanation of your problem?
This plugin support highlighting of comments:

But if you mean auto comment and uncomment by Ctrl + / i try to add it in near update.

@datavistics, i don't know what do you mean but i add auto comment in 0.4 release.

Sorry for the delay I was on vacation :)
Id like to be able to use Cmd+/ to comment out code.
But it seems you are adding this in the 0.4 release!

Just tested this out, its great. Thanks so much!

@datavistics, thank you too for your feedback!


"Success" don't work in the response

Hello
when the bank confirm the payment, the gateway don't receive the response "success".
And my woocommerce keep in "waiting payment" for this customer.
However, the param "response" is ok...
An idea?
Thank you

Hi,
I've checked the plugin on a fresh install, and I think I know why it's not working for you.
As said in the doc:

Create a page that contains shortcode above and fill the automatic_response_url field in admin.
[woocommerce_atos_automatic_response]

Which means you have to:

create a new wordpress page
add the snippet above in the page content
retrieve the page url and copy it
go to the plugin settings and paste the page url in the automatic_response field

Hope it fixes your issue.
πR

Hello
It works!
Excuse me for forgetting.
Thanks for all!
Fred


added parseResource method to ConfigFile.

This will hopefully avoid duplicated code across plug-ins. It would be cleaner to read plug-in code that uses this method :).
Forum post: https://forums.spongepowered.org/t/help-me-understand-configuration/5582/3

I agree in adding this method, but i don't agree with the name. The name doesn't imply that it saves the configuration file to disk. Maybe something like parseAndSaveResource is better

It doesn't save. But indeed maybe a force-save method will be better. Than could avoid extra duplicated code.

Sorry. I first read the code and then the forum post so i accidentally thought this method saved too. Why not create such a method as mentioned in the forum post?

Idk, I was thinking. In case the dev wants to add something else before it saves the config. It would save it twice. And that not really efficient.

I just realise that this can't work without giving the plugin object as parameter. But wouldn't that look to messy?

@sk89q
Gonna let you make the call on this as you are adjusting the config API currently.

We are contemplating replacing the configuration code entirely with something new.
Something similiar to https://github.com/sk89q/Eduardo/tree/master/src/main/java/com/sk89q/eduardo/util/config but with support for serialization of arbitrary objects.

We've switched to configurate-hocon but thanks anyhow!


How to get the flags (key, value) pairs into a json object?

Hi dev team,
How do we get the (key, value) mapping from FLAGS if we want to save them?

There is some lightweight serialization / deserialization support:

To save, use FLAGS.append_flags_into_file() or FLAGS.flags_into_string() for serialization of all flags
To restore, specify --flagfile=<somefile> on the command line pointing to a file containing the above content, or call FLAGS. read_flags_from_files(['--flagfile=<somefile>'])

If you really need json, you'll need to do it manually. Take a look at https://github.com/abseil/abseil-py/blob/06edd9c20592cec39178b94240b5e86f32e19768/absl/flags/_flagvalues.py#L1183-L1190 how to get all flags and their serialized value.
Hope this helps.

Closing due to inactivity. Feel free to reopen if you still have questions.


run example app failed, AndroidX

[!] Your app isn't using AndroidX.
To avoid potential build failures, you can quickly migrate your app by following the
steps on https://goo.gl/CP92wY.
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:checkDebugDuplicateClasses'.

1 exception was raised by workers:
java.lang.RuntimeException: Duplicate class android.support.v4.app.INotificationSideChannel found in modules classes.jar (androidx.core:core:1.1.0) and classes.jar (com.android.support:support-compat:28.0.0)
Duplicate class android.support.v4.app.INotificationSideChannel$Stub found in modules classes.jar (androidx.core:core:1.1.0) and classes.jar (com.android.support:support-compat:28.0.0)
Duplicate class android.support.v4.app.INotificationSideChannel$Stub$Proxy found in modules classes.jar (androidx.core:core:1.1.0) and classes.jar (com.android.support:support-compat:28.0.0)
Duplicate class android.support.v4.graphics.drawable.IconCompatParcelizer found in modules classes.jar (androidx.core:core:1.1.0) and classes.jar (com.android.support:support-compat:28.0.0)

[!] Your app isn't using AndroidX.
To avoid potential build failures, you can quickly migrate your app by following the
steps on https://goo.gl/CP92wY.
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:checkDebugDuplicateClasses'.

1 exception was raised by workers:
java.lang.RuntimeException: Duplicate class android.support.v4.app.INotificationSideChannel found in modules classes.jar (androidx.core:core:1.1.0) and classes.jar (com.android.support:support-compat:28.0.0)
Duplicate class android.support.v4.app.INotificationSideChannel$Stub found in modules classes.jar (androidx.core:core:1.1.0) and classes.jar (com.android.support:support-compat:28.0.0)
Duplicate class android.support.v4.app.INotificationSideChannel$Stub$Proxy found in modules classes.jar (androidx.core:core:1.1.0) and classes.jar (com.android.support:support-compat:28.0.0)
Duplicate class android.support.v4.graphics.drawable.IconCompatParcelizer found in modules classes.jar (androidx.core:core:1.1.0) and classes.jar (com.android.support:support-compat:28.0.0)

Duplicate of #23


consul-svc-healthy: use system jq if available (master)

Duplicate of #7 but for master/Geneva

LGTM

@anonymouse64 Rebase required (option to update not available to me via GitHub UI).

Thanks, rebased

recheck

This PR is languishing from inattention and lack of repository-level committers.


Add Meta Tags for Enhanced SEO⭐

I've noticed that our web pages are missing essential meta tags which play a crucial role in optimizing our site for search engines. Implementing these tags will help improve our project's SEO, making it more discoverable and accessible to users via search engines.
Difference

Refence Link  : https://nextjs.org/docs/app/building-your-application/optimizing/metadata
Expected Benefits:

Enhanced search engine ranking, making our site more discoverable to users.
Improved visibility and presentation on social media platforms, leading to better user engagement.
Increased traffic through more effective and targeted SEO strategies.

Issue template was not used?
Please be specific, what tags?


About autoscaling of middle manager

In official doc(http://druid.io/docs/0.9.1/design/indexing-service.html), you mentioned:

The Autoscaling mechanisms currently in place are tightly coupled with our deployment infrastructure but the framework should be in place for other implementations. We are highly open to new implementations or extensions of the existing mechanisms. In our own deployments, middle manager nodes are Amazon AWS EC2 nodes and they are provisioned to register themselves in a galaxy environment.

So if I want to enable the autoscaling mechanism, should ”the Amazon AWS EC2“ and ”galaxy registration“ be two requirements?

Hey @billbargens, the current autoscaling implementation is EC2-based, but you don't need to use galaxy. You can set any userData you want and your AMI can interpret that however it wants.
fwiw "galaxy" refers to: https://github.com/ning/galaxy

So that still means I must deploy druid on Amazon EC2 nodes?

@gianm So that still means I must deploy druid on Amazon EC2 nodes?


Support PartiQL queries

Some AWS services, notably DynamoDB, Redshift and Glue, support  PartiQL as a query language. PartiQL support in the compiler plugin could be very useful.
It could look something like this:
val musicTable = new Table...;

musicTable
    .filter((x) => x.Artist === 'Artist' && x.SongTitle === 'SongTitle')
    .map(x => x.Album)

becomes
SELECT Album
FROM Music
WHERE Artist=? and SongTitle=?

Amazingly powerful feature! Simplifies and scales the compiler by acting as a common intermediate format.


wandb without a project configured throws an error

Weights and Biases version: 4.1
Python version: 2.7
Operating System: ubuntu

wandb sdf

l2k@l2k-Oryx-Pro:~$ wandb sdf
Traceback (most recent call last):
  File "/home/l2k/.local/bin/wandb", line 11, in <module>
    sys.exit(cli())
  File "/home/l2k/.local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/l2k/.local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/l2k/.local/lib/python2.7/site-packages/click/core.py", line 1061, in invoke
    cmd_name, cmd, args = self.resolve_command(ctx, args)
  File "/home/l2k/.local/lib/python2.7/site-packages/click/core.py", line 1100, in resolve_command
    cmd = self.get_command(ctx, cmd_name)
  File "/home/l2k/.local/lib/python2.7/site-packages/wandb/cli.py", line 60, in get_command
    project, bucket = api.parse_slug(cmd_name)
  File "/home/l2k/.local/lib/python2.7/site-packages/wandb/api.py", line 128, in parse_slug
    raise Error("No default project configured.")
wandb.api.Error: No default project configured.

Fixed in 0.4.2


Additional Discovery Events

Adding events to model various discoverable entities on Windows/Linux/MacOS:
Windows/Mac specific items were added as extensions for the OS.
Windows specific entities:
Registry Value
Registry Key
Prefetch
MacOS specific entities:
Startup Application
OS agnostic entities:
File
Folder
Admin Group
Job
Module
Network Connection
Networks
Peripheral Device
Process
Service
User Session
User Info
Other:
Unsuccessful Discovery : result document (event) that captures when a discovery attempt failed

Incorporated changes from the draft pr
Switching to pr request (non-draft)


remove fancy fonts from LaTeX docs

Nonstandard LaTeX fonts should be avoided in the docs. Looking at you passive_cable docs...
https://github.com/eth-cscs/nestmc-proto/blob/master/docs/passive_cable/cable_computation.tex#L11

TeX Gyre fonts 'should' be part of a full texlive install. They might be packaged differently under e.g. Arch Linux, or it could be a matter of them not being visible to fontconfig: see for example: (https://wiki.archlinux.org/index.php/TeX_Live#Fonts)

They aren't in my TeXlive install, and I have the "full install".

If it turns out this is a common TeX installation issue, I'll nuke the font. But: so pretty!

I was able to install them separately. I am not so comfortable with making LaTeX any harder than it is to set up. If we get any more issues, the fonts get nuked.


Create a docker package for flumotion

Currently we have 1 flumotion per stream. This currently uses 1 machine per stream as managing the configuration of multiple streams on one machine is to hard.
We should package flumotion up in a docker thingy - https://www.docker.io/
"Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more. "
Flumotion in the TimVideos streaming system
There are 3 types of flumotion configurations; they should share a base docker image but also be provides a separate docker image correctly configured.

Collector - runs on site at a conference and does light encode to compress data so you can send it over the internet to an encoder.
Requires dvswitch support.
Encoder - runs in the cloud and converts video from the light encode format to something suitable for people to view in their browser.
Repeater/Amplifier - runs in the cloud and streams a copy of the encoder output. Allows the system to scale past the limits of a single machine.

There are a couple of support applications we use with flumotion; these should be installed inside the docker image and started as part of the container.

watchdog - Restarts failed components and does a full system restart when things go totally wrong.
register - Registers with the tracker website (so the stream can be found) and sends statistics and logs for reporting and load balancing support.

Hi @maxstr - Can you dump a link to your final report into this bug? I believe your packages ended up at https://hub.docker.com/u/streamingsystem/ ?

Yep, available here:
http://mstritzi.nfshost.com/writeup/index.html
The docker image you'd want is streamingsystem/flumotion.


COSMOS: Uncaught ReferenceError: process is not defined in @azure/cosmos/dist-esm/common/logger.js:3

Package Name:  @azure/cosmos
Package Version: 3.2.0
Operating system: Windows 10 x64 build 17763.737
[ ] nodejs   node = 10.16.0, angular cli  = 7.3.9

version:

[ ] browser  Chrome v77

name/version:

[ ] typescript   3.2.2

version:

Is the bug related to documentation in

[ ] README.md
[ ] source code documentation
[ ] SDK API docs on https://docs.microsoft.com

Describe the bug
Uncaught ReferenceError: process is not defined
at<EMAIL_ADDRESS>(logger.js:3)
at webpack_require (bootstrap:78)
at<EMAIL_ADDRESS>(defaultQueryExecutionContext.js:1)
at webpack_require (bootstrap:78)
at<EMAIL_ADDRESS>(index.js:1)
at webpack_require (bootstrap:78)
at<EMAIL_ADDRESS>(queryIterator.js:1)
at webpack_require (bootstrap:78)
at<EMAIL_ADDRESS>(index.js:1)
at webpack_require (bootstrap:78)
To Reproduce
Steps to reproduce the behavior:

At initial runtime load (localhost:4200)  ng build/serve both no errors

Expected behavior
A clear and concise description of what you expected to happen.
package.json.txt
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
Sorry, not exactly where I wanted to drop my package.json.  I'm trying to convert the QueryThroughput.js sample into actual Typescript.  with the classes and the imports, services, components, html.  What dependency am I missing?
P.S.  github link on npmjs.org for @azure/cosmos still points to the old repo.

Looking at the code:  azure-sdk-for-js/sdk/cosmosdb/cosmos/src/common/logger.ts   line 4  is this:
const cosmosLevelFilter = process.env.COSMOS_LOG_LEVEL || "warn|error";
The ReferenceError cannot find where 'process' is defined.  Is there a peer dependency that's missing?

@BulldogCTISolutions Which version of webpack are you using?

Steve,
Not sure I'm using any webpack.  Just plain old Angular.  Started from ng new.  I do an npm start.  I'm attaching my package.json.
package.json.txt

It looks like this was a change in Angular CLI. They stopped polyfilling the node builtins https://github.com/angular/angular-cli/issues/9827#issuecomment-369578814
I think we can handle this better on our side too. I'll investigate a fix.

https://github.com/Azure/azure-sdk-for-js/pull/5223

Thanks Steve!  I thought I might be missing an import.  Is there a specific one I need?  npm package to add?

I don't know of one. Try running this in your global scope before any other code or importing anything:
var process = { env: { } }

This has been fixed in 3.3.0


make nixos.stateVersion mandatory

Motivation for this change
In #61321 a necessary migration was only run if the stateVersion is older than 19.09. The default value of cfg.release (= 19.09) prevented it from being run on systems without explicit stateVersion.
In this particular case, maybe using stateVersion could have been avoided, but in general, it is not safe to have a system without explicitly set stateVersion.
All systems installed in the last few years should have it set automatically, but there are surely still a lot of configs out there with it not set.
(While stateVersion as it is currently designed has other problems, like mentioned in #50112, the goal here is to just do the minimal change to get systems safe.)
Things done

[X] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[X] NixOS
[N/A] macOS
[N/A] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

CC @edolstra @oxij @Ekleog @srhb @samueldr

@Infinisil Regarding the assertion: Maybe replace the last paragraph. We want to avoid a situation where a user accidentally deletes the setting, gets that assertion message, then puts it back with a release possibly several versions ahead of their system's compatibility at the suggestion of the text.

@srhb Yeah, but I'm not sure what would be a good replacement for it. Ideally I'd like to say "Set it to the NixOS version you initially installed, but only if you recently updated and notice problems. If you're sure that you don't need an older stateVersion by checking all things you would have to manually upgrade, you can set it to 19.09". Not sure how this can be phrased better..

@Infinisil Punting a bit, but how about referring to the option documentation and we can try to improve on that? Better than having to cook up a great explanation in multiple places. :)

I took the liberty to push a better stateVersion description to this PR in https://github.com/NixOS/nixpkgs/pull/65314/commits/548d4273e419d30577f75b423c0322b055bc73ba, how does it sound?

But I expect @edolstra to revert this.
See #39447 and
https://github.com/NixOS/nixpkgs/commit/1f0b6922d3c2de3da235a7075d0d3cb9255b7cd7#commitcomment-28997477
and 4d1332e507fe55103bc5ddd0a384447dc7833e12.

Actually, it's kind of bad UX if we now make stateVersion mandatory, but at some later point deprecate that option in favor of per-module stateVersion's as described in https://github.com/NixOS/nixpkgs/pull/50112#issuecomment-438769388
So it's arguably better if we first figure out the per-module stateVersion, and only after that make it mandatory. This also means a lot of users won't ever see a warning for it being unset, because they're not using any modules that need it.
But then again, figuring out per-module stateVersion will take some time, likely with an RFC, and until that point users who don't have the global stateVersion set are at risk of breakage

But then again, figuring out per-module stateVersion will take some time, likely with an RFC, and until that point users who don't have the global stateVersion set are at risk of breakage

Yes. Which is why I think for now we should just copy-paste the warning from 1f0b6922d3c2de3da235a7075d0d3cb9255b7cd7 (maybe with minor modifications describing what will break if they won't set it) to all the modules that actually depend on stateVersion. Users that don't have those modules enabled won't feel a thing, the rest could be affected by the issue, so, I say, let them see it.
Thoughts?

That sure would be the least-invasive solution for now, but copy-pasting it around doesn't sound very nice. Maybe an internal option system.warnStateVersionUnset which modules like postgres can set to true would be better. I like this as a temporary solution

To be more end-user friendly, the internal setting could be a list, system.modulesRequiringState, and modules requiring state append to it. This would allow printing "The modules xxx, yyy and zzz require stateVersion to be set`. An empty list would mean nothing requires a stateful version to be set...

Though, this is all well, but with this coming update, most, if not all, systems will require stateVersion to be set because of how it ends up being used for systemd-timesyncd. (#64922) So, in a way, it's now de facto required. Unless a solution is figured out before the 19.09 fork/release, to handle the issue in another way.

I see. So it might be best to just go ahead with the original idea of this PR and make stateVersion mandatory. With a good assertion error this shouldn't be a big problem.

One can use NixOS without systemd-timesyncd. There are, at the very least, ntpd and chrony modules doing the same thing (the latter one is much simpler source-vise, yet also much more advanced, with automatic clock drift calculations and etc, btw).
Thus, I still vote for per-module warning (regardless of the actual implementation, the list proposal above sounds nice, though).

I'm contra to adding mandatory options. I'm also against of having asserts about filesystems."/" and boot.loader. We should remove those out for
$ nix-build ./nixos -A system --arg configuration '{ services.postgresql.enable = true; }'

to just build.
It should be nixos-install or nixos-rebuild who should check whether stateVersion is configured correctly.

@oxij Always requiring stateVersion has the benefit that a module that initially didn't need it and was enabled by the user will continue evaluating successfully even if it eventually does need it. If this was done with the module list thing, people could suddenly have to set stateVersion when they update their NixOS.
I feel like this is a band-aid like situation, we should just rip it off in one swoop by making stateVersion mandatory, which it should've been from the beginning. Once everybody has it set, we won't have to worry it again, no maintenance cost either.
@danbst That would be neat yeah, but I think that's something for the future. For most users whether the stateVersion check is done in the module or in nixos-rebuild doesn't make a difference, so there's no big cost to going with the former for now.

@Infinisil
Firstly, I think everybody here agrees than a global stateVersion is a hack which should be eventually removed.
Secondly, modules using stateVersion should check it explicitly, if they don't I would consider it a bug.
Hence

If this was done with the module list thing,

(or any other listed non-mandatory method)

people could suddenly have to set stateVersion when they update their NixOS.

but only if they use something that depends on it, which is the point.

we should just rip it off in one swoop by making stateVersion mandatory

Firstly, concerning systemd-timesync use of stateVersion in nixos/modules/system/boot/timesyncd.nix, I don't see why that piece of code can't be made unconditional. It looks like a migration script that will work for both old and new version of the package to me. (Since the old version of systemd-timesyncd can just regenerate the old directory, there's nothing of essence stored there anyway. Though, if you boot into an older release it might create some junk on the second migration, but that seems easy to fix with one more conditional around the line doing mv.)
Which brings us back to the point that most systems should not, and with systemd-timesyncd fixed would not, ever use stateVersion. Currently stateVersion is used in conditional expressions in just a handful of services. Most uses of stateVersion in nixos tree are assignments to it to "avoid warnings during eval" and similar reasons, which is rather meh, IMHO.
In short, I think that making it mandatory is counterproductive. By making it mandatory we will just make it harder to remove.
After some more thinking about this I reverted to the opinion that copy-pasting edited warnings from 1f0b6922d3c2de3da235a7075d0d3cb9255b7cd7 to modules using it should be better than any new infrastructure. Those copy-pasted pieces will just eventually turn into warnings/assertions for modularized stateVersion.
I.e. I vote for simple copy-paste.
@danbst
Well, adding some runtime checks that would check that on-disk stateVersion is equal to stateVersion of configuration.nix would be useful, but I don't see how we could ever remove it from configuration.nix eval altogether.
Because, in general, we can't just migrate on-disk service state a-la database schema migration because users can boot into older generations of their NixOS systems and booting into a newer one without explicit stateVersion could cause backward-incompatible migration for the older ones. (Which, I claim, is not the case for systemd-timesyncd issue above, especially since it does not store anything of essence in its state anyway.)

On second thought, yeah actually that doesn't sound half bad @oxij, having a warning copy-pasted to every module that uses stateVersion until the per-module version is implemented. It's the only solution that doesn't need new infrastructure (which would eventually have to be removed again).

To make the switch less painful, I think we should handle this like a deprecation (basically deprecating omittance of stateVersion).
So IMO, another PR (for 19.09) which emits a warning would be more appropriate. This PR could go into 20.03, then.

still important to me

Is this still relevant?

Duplicate of #149877


RIP Unity

Thanks for killing Unity MS

Hi guys I'm new to github and I was looking for a project to contribute. This looks like a nice one as I like the way Unity container works and I'm experienced with c#.
How can I start? I just grab an issue and try to fix?

@hangarter yeah in the open source community it usually works like that. You check out an issue, ask if you can fix it, fix it, then create a pull request. I'm not so sure however if this project is actually being maintained, so before starting to fix something I would ask around.

Hi guys I'm new to github and I was looking for a project to contribute. This looks like a nice one as I like the way Unity container works and I'm experienced with c#.
How can I start? I just grab an issue and try to fix?

Well, given that the maintainers have never merged a single pull request, I think you'd be losing your time. Looks like MS gave ownership of this project to someone who doesn't care...

The King is dead, long live the King!
I will be helping to keep this lib afloat for a while.
I welcome any help and contributions!

Great news, thanks @ENikS for taking over this project!
I was really sad to see Unity being abandoned so soon after going open source.

A little shout out to @ENikS here:
https://blogs.msdn.microsoft.com/dotnet/2015/08/21/the-future-of-unity/#comment-545865
It would seem to me that The King is just getting started. 😄 👍


[util] Enable cachedDynamicBuffers for Steel Soldiers and FIFA 2003

Steel Soldiers does some "classic" CPU read-backs from DYNAMIC buffers. So does FIFA Football 2003. Plus some minor config file alignment, as it was a bit all over the place for the last two entries.

Have the demo been checked for if the exe name is the same?

Have the demo been checked for if the exe name is the same?

I've included it in the regex for FIFA 2003, but the demo for Steel Soldiers uses "demo.exe", so I'll let it rest.


relocation group_coordinator_impl_test

Codecov Report

Merging #11 (cc8d598) into main (71f9bdf) will decrease coverage by 23.68%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             main      #11       +/-   ##
===========================================
- Coverage   61.70%   38.01%   -23.69%
===========================================
  Files           4        7        +3
  Lines         141      484      +343
===========================================
+ Hits           87      184       +97
- Misses         43      281      +238
- Partials       11       19        +8

Impacted Files
Coverage Δ

pkg/kafsar/kafka_impl.go
0.00% <0.00%> (ø)

pkg/kafsar/kafsar.go
0.00% <0.00%> (ø)

pkg/kafsar/group_coordinator_impl.go
68.79% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 71f9bdf...cc8d598. Read the comment docs.


Null check on video player orientation setting

Orientation setting logic was moved above the visibility check here https://github.com/edx/edx-app-android/pull/449#discussion_r46538847, which caused the allowSensorOrientation to be called with getActivity being null causing the Null Pointer and resultantly a crash.
https://openedx.atlassian.net/browse/MA-1852
@1zaman plz review again
NOTE: Due to some rebasing havoc this PR https://github.com/edx/edx-app-android/pull/486 went useless

:+1:


Dropdowns are unreadable

Dropdowns have a white background and a very light text color which makes it unreadable, luckily hovering over the items gives the hovered item a blue background so this doesn't really means it's unusable, just though you should know though you probably already knew.
Note that I'm using OneDark which comes with Atom so not a weird syntax theme and the same thing happens with all the other preinstalled syntax themes like Atom Dark and Solarized Dark.
I'm using Atom 1.0.0 in Windows 8.1.

It look worse in monokai syntax theme, which makes the text color white so you can't see the text at all, but I understand it's a third party plugin, just letting you know.

Yeah, you're right. This came up a few days ago. Though I had fixed it but looks like I didn't.
It's absolutely important to me that the UI theme works with other syntax themes. Even light variants.
I just pushed an update to fix this. Update and let me know if it's better for you too.

Much better, thank you! It looks perfect now :smiley:


Fix SW-20939

1. Why is this change necessary?
The alert modal for comparison view (e.g. too many products) does not close as expected.
The overlay still is visible after the modal is closed.
2. What does this change do, exactly?
Remove additional overlay, that's not closing.
3. Describe each step to reproduce the issue or behaviour.

Goto Listing
Try to compare more products as allowed
Close alert modal

4. Please link to the relevant issues (if any).
https://issues.shopware.com/issues/SW-20939
5. Which documentation changes (if any) need to be made because of this PR?
6. Checklist

[ ] I have written tests and verified that they fail without my change
[x] I have squashed any insignificant commits
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/SW-21588
Please use this issue to track the state of your pull request.

Hi @felixgeyer,
thank you for your contribution and please excuse the late response.
Actually this issue has been resolved with PR 1819.


No support for badges / labels

Badges are very common UI elements in docs. They can be used for many purposes:
Showing relevant tags

Showing version numbers

Showing attributes of something in an easy-to-recognize way

Showing the status of an element

I think this would be a great candidate for a third party extension. There is no need for this to be included with the Markdown library directly. You may want to check the list of known extensions to be sure one doesn't already exist.


Removing usages of jQuery from jolokia.js and jolokia-simple.js

Relates #575. Cleaning up as many usages of jQuery functions as possible from those JS scripts.
What remain to be done are:

$.extend
$.ajax

@grgrzybek thanks!


Cant make it  work on Win11 22H2 22622.590

When trying to use this I cannot find any way to make it work - e.g. hold down 'a' and it keeps repeating ad infinitum, press space bar to get the accents and nothing happens other than space bar keeps repeating. The same happens when trying to use left & right arrow keys .
Great pity as it would have been useful if it worked.

Hello, it seems you are in the Windows Preview version, so it can be the reason.
On my side it is working well on Stable release 21H2 22000.856.
May be do you have a an error report in the Event Viewer of Windows? (Event Viewer -> Windows logs -> Application)
Anyway, I readed the 22H2 will be released soon so will be able to check with this release.

No answer so I close the issue.


decision on VERIFIER role

See https://github.com/sigstore/root-signing/blob/main/VERIFIER.md
I'm trying to establish how important the process described here is

I don't remember seeing this verification being done visibly before  -- but maybe it was?
In my opinion the hardware key authenticity part is not critical

tuf-on-ci already makes it impossible to "accidentally" use a non-hw key
if keyholder is malicious enough to secretly use a non-hw key (by modifying the signing software), I think we have far bigger problems and concentrating on the key being in a yubikey is not useful

I am not sure how relevant the ceremony verification parts are now -- should we salvage something or no?

Comments @kommendorkapten or @haydentherapper ?

see also #518

I think this is only relevant during the initial step for a new key holder, i.e when a key is added to root.json. Once it's there, it would be almost impossible to fake it, as the public key is kept in the signed metadata.

I think this is only relevant during the initial step for a new key holder

Can you be a bit more specific, do you consider the hardware key authenticity check relevant or some other part of the script?
If it's the hw authenticity check, do you think my arguments against it are reasonable?

Can you be a bit more specific, do you consider the hardware key authenticity check

Yeah, so when a new key-holder is introduced, say a planned rotation, at that time, I think it's relevant to verify the authenticity of they public key added to root.json, by getting a device attestation from the Yubikey and making sure that the public key can be verified up to Yubico's CA.
Once that is verified (and the proof recorded, e.g. as a comment in a PR), it's not as useful anymore, as the public key is now in the signed root.json, and so, the key can not be changed to a new one that is not belonging to a HSM.

I have to admit I don't quite understand what we are protecting against here

If the issue is malicious keyholder, the key location seems  irrelevant
if the issue is trusted keyholder being somehow tricked into using a tampered yubikey, could we just add a step to keyholder documentation  that directs them to https://www.yubico.com/genuine/ before accepting an invite?

Or maybe I'm missing some advantage?
For context, I'm trying to understand if I can remove the code, scripts and documentation related to this VERIFIER role in some way

if the issue is trusted keyholder being somehow tricked into using a tampered yubikey, could we just add a step to keyholder documentation that directs them to https://www.yubico.com/genuine/ before accepting an invite?

As this is an open source project, I think transparency is important. How can I as a community member trust that they key is a legitimately Yubikey? Just having the key-holder verify it is not enough IMHO. Trust but verify.
Cryptographic evidence that all keys used are authentic Yubikeys removes any such questions. As this is the trust anchor for Sigstore's trust root, I think we should be as transparent as possible about the security posture we are using.

If this is critical then it should be a part of the signing tools or at least the signing process, not some optional verifier role, right?
And the verification should actually check the keys used in the metadata, not what happens to be in a separate keys/ directory. I've not reviewed all of the code yet but so far I've not seen anything that verifies that the signing keys have a valid certificate in keys/. Is that actually done  somewhere?

The tooling today only prints the key id (by recomputing it over the public key), and verifies the certificate chain. So the complete verification does not appear to be automated.
Yes agreed, having device attestation verification built into the signing tool/process is preferable IMO. (This is how we are doing it at GitHub, each signature over root and targets are always verified against the public key coming from the device attestation, which is verified against Yubico's root CA.)

For the verification of signed metadata, I don't think this role is necessary anymore as we have comprehensive automated testing.
As for verifying attestations on the Yubikey, your suggestion of "could we just add a step to keyholder documentation that directs them to https://www.yubico.com/genuine/ before accepting an invite?" seems sufficient to me. We should encourage keyholders to verify their Yubikeys, but I don't think we need to maintain our own scripts to do that if Yubikey provides such a utility. Maybe we can see if Yubikey has an open source utility rather than a website, to Fredrik's point about transparency?

There is a tool ykman https://developers.yubico.com/yubikey-manager/
some useful commands:
ykman piv keys attest 9c -

and
ykman piv keys export 9c -

I can look more into this tomorrow, and see if there is simple way to verify using this.

Okay, I think I'll do this:

remove VERIFIER as a specific role and the related tools
file a new issue about integrating "key authenticity check" into signing events, either as tuf-on-ci feature, a separate root-signing workflow or a just command that signers can run

Sounds good!


coffeelint --version still reports 1.9.3

According to the package.json I do have version 1.10.1 installed. Yet coffeelint --version yields 1.9.3

I can't reproduce the issue. Can you provide more details?
Are you talking about the package.json in your project, or the one in node_modules/coffeelint/package.json? I'm not sure what other details I need.

I apologize. I haven't figured out what is going on, but it must be my local system that is to blame, somehow...
Depending on the terminal window used I get different results for the exact same command:
/usr/local/bin/coffeelint --version

In the one it's 1.9.4, in the other it's the 1.10.1 that I was expecting to see. The two terminal sessions are on the same system, just in different directories.

...
And as it turns out, in one session the current working directory happens to be that of a project that has a non-global install of coffeelint, which, surely by no coincidence, happens to be version 1.9.4.
How can it be that whilst explicitly invoking the global version, the local one gets run instead?
At least, I suspect that that must be what is happening here but it sure is freaking me out.

The file that executes when you run coffeelint is very small. It's only purpose is to find and execute a version of lib/commandline.js. If you're in a project that has a local coffeelint then it uses ./node_modules/coffeelint/lib/commandline.js, otherwise it uses the one that it ships with.
Lines 29 and 30 look for a local version to use.

Thanks for the enlightenment. Understanding the mechanics makes the behaviour a lot less scary, albeit counterintuitive (still). Perhaps it could be documented?

It definitely could be. I recommend opening a pull request to add it to
wherever you would have looked. The website is in the gh-pages branch and
there is a docs folder.
On Jul 21, 2015 4:35 PM, "David Bouman"<EMAIL_ADDRESS>wrote:

Thanks for the enlightenment. Understanding the mechanics makes the
behaviour a lot less scary, albeit counterintuitive (still). Perhaps it
could be documented?
—
Reply to this email directly or view it on GitHub
https://github.com/clutchski/coffeelint/issues/477#issuecomment-123470098
.

Thanks. I'll look into it. But it may take a while...

That's totally fine. With the current site design I'm not sure where to put this info that people will find it. If the struggle is explaining with it I'll definitely help with that.
Even though it's unusual my hope is that it's simply doing the right thing. If your project is setup to use 1.9.3 on your CI server, then I'm sure that's the version you should be running when you lint your code while you're working. It handles upgrades too, so maybe you have 1.9.3 globally, but your project/CI is setup for 1.10.1. In that case you don't get mysterious errors where your CI server complains about something but you can't get coffeelint to error locally.


Refactor context: tests on ibm pipelines

Unit Tests finished.
View logs: TXT HTML

Kubernetes Tests finished.
View logs: TXT HTML

OpenShift Tests finished.
View logs: TXT HTML

Kubernetes Tests finished.
View logs: TXT HTML

Unit Tests finished.
View logs: TXT HTML

OpenShift Tests finished.
View logs: TXT HTML


Add License

Your public repo and PyPI submission indicate that you intend for this to be broadly usable, but unless I've failed to look in the right place, you haven't provided any license with the code. If you don't have an opinion on such things, may I suggest using the MIT license?
https://opensource.org/licenses/MIT
Thanks for sharing your code!

in the MIT license it’s inconvenient to edit the year and the author, it’s easier for me to copy a whole file without editing ))))))0
i will use UNLICENSE, ok?

Thank you!


Search field Icon

It looks like the searchfield-current-engine-icon has been implemented in Vivaldi.

What?

Sorry, I just realized that I got confused by the wrong window... :) It is not implemented in Vivaldi!


Rename ReadMe file so GitHub renders it correctly

Completely aesthetic change but makes the project much more approachable on Github. Other source control interfaces should also render it more pleasingly if they know it contains MarkDown.

I think that this was done.

Sure.


SPI read_passive_target timeout

Modified the reverse_bit function to be more efficient, roughly 40% faster on a Raspberry Pi 3b+.
To address the issue described in https://github.com/adafruit/Adafruit_CircuitPython_PN532/issues/19:
Modified the _wait_ready function to by avoiding re-declaring the with statement each iteration of the while loop.

nice, did you try this on raspi only or also on some circuitpy hardware?

Unfortunately the only things I have that will run circuitpython are Raspberry Pis.

@caternuson wanna review by checking on a samd?

Thanks. Looks good. I recreated what I think is the original issue by doing this:
>>> pn532.read_passive_target(timeout=5)
bytearray(b'&)3~')
>>> pn532.read_passive_target(timeout=5)
>>>

where for the first read, the card was sitting on the reader, so could be read right away. For the second read, I put the card on the read after ~2 secs, and got nothing. With the updated code, both seem to work:
>>> pn532.read_passive_target(timeout=5)
bytearray(b'&)3~')
>>> pn532.read_passive_target(timeout=5)
bytearray(b'&)3~')
>>>

Yes, that's basically how I encountered the error as well, setting the timeout to a long value and trying to scan a tag somewhere after the initial call.
Without looking harder into the adafruit_bus_device.spi_device module and possibly one of it's imports, I assume something is being reset in the __exit__ call with makes.


BAH-2426 Configured Admission Letter

Co-authored-by: manimaarans<EMAIL_ADDRESS>

Can you also attach a screenshot of how it looks now?


Windows randomly stop accepting keyboard input

Describe the bug
Open windows will stop accepting keyboard input at random, typically just after interacting with the window by mouse. It will remain impossible to send any keypresses to the window (typed input, hotkeys, etc.) until focus is shifted to another window or workspace, then back to the problem window.
This most commonly occurs in browsers (brave) and terminal windows (kitty, foot). I cannot reliably recreate this issue, but it pops up every minute or two in Brave, typically while interacting with a browser extension.
Images/videos/anything that would help
https://user-images.githubusercontent.com/36456160/163271008-cc8d1dd8-d1c8-4289-bee5-51ce82674eda.mp4
EDIT Replaced original video with  new copy in a less problematic codec.
You'll have to take my work that I was trying to type in vim before shifting focus :)
Please attach a log (it's in /tmp/hypr/hyprland.log) and, if crashing, a crashdump (coredumpctl, then coredumpctl info )
hyprland.log

I am aware and investigating tha issu

should be fixed in ebfbd4f7c302f80032476e65937575a2d6ff0882
please check it, and if it's resolved, close =)

This happens to me again. If I use mouse to switch focus to another window and switch back again, the original window won't accept keyboard input

fixed in git


Boost RuntimeLibrary missmatch

Hi
I am trying to build 64bit dll in my project and using boost static lib to my project using CMake , it builds project fine but compilers gives me this error.
1>boost_thread-vc140-mt-gd.lib(tss_pe.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MTd_StaticDebug' doesn't match value 'MDd_DynamicDebug' in 1obj
1>boost_thread-vc140-mt-gd.lib(thread.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MTd_StaticDebug' doesn't match value 'MDd_DynamicDebug' in 2.obj
My Dll project is created with Runtime Library is Mutli-threaded DLL (/MDd) and using boost static lib.
I had install vcpkg using this commands

vcpkg install boost:x64-windows-static
CMake -G "Visual Studio 14 2015 Win64" -H./Software/ -B./Software/Build64 -DVCPKG_TARGET_TRIPLET=x64-windows-static -DCMAKE_TOOLCHAIN_FILE=D:/Work/vcpkg/scripts/buildsystems/vcpkg.cmake

Environment

OS: Windows 10 64bit
Compiler: Visual Studio 2015 Update 3

Is this bug or am i doing anything wrong ? vcpkg generate using wrong type of Runtime library ?
If i link the pre-compiled boost library with static version it works fine.
Naresh

Try the x64-windows-static-md triplet when installing the boost libs

Try the x64-windows-static-md triplet when installing the boost libs

Thanks .,. it working now.

But when I use vcpkg install boost:x64-windows-static-md (or vcpkg install boost  --triplet=x64-windows-static-md), it only changes the output directory. The .lib file still is *-mt.lib


Ludusavi tries to back up my entire %userprofile%

Using the GUI, Ludusavi tries to backup many GB worth of files it identifies as "Battlefield II"'s save data (a game I've never had on my PC). The GUI crashes after it tries to display the list of those files. After following through with the backup, it turns out the files are actually the entirety of my %userprofile%.
This doesn't happen if the CLI is used instead of the GUI.

Thank you for reporting this. The save path for Battlefield 2 is malformed and I could see that causing an issue (I'll try to get it fixed on PCGamingWiki), but I'm not able to reproduce this issue using the latest version of Ludusavi. Some questions for you:

What version of Ludusavi are you using?
Are you on Windows 10?
Just to be sure, does it really say Battlefield II or is it Battlefield 2?

v0.7.0
Yes.
It's actually Battlefield 2, sorry.
I used a freshly installed Ludusavi. Here's the whole file:

---
manifest:
  url: "https://raw.githubusercontent.com/mtkennerly/ludusavi-manifest/master/data/manifest.yaml"
  etag: "\"d012ef512df925a0c3f71a1abbdb85d84c656bc5750c3748b75fa0c52fef6fd3\""
roots:
  - path: "C:\\Program Files (x86)/Steam"
    store: steam
backup:
  path: "F:\\Portable Software\\ludusavi-win64\\ludusavi-backup"
  ignoredGames: []
  merge: true
  filter: {}
restore:
  path: "F:\\Portable Software\\ludusavi-win64\\ludusavi-backup"
  ignoredGames: []
  redirects: []
customGames: []

The wiki was updated to fix the bad path, so I've updated Ludusavi's data set as well. Could you reopen Ludusavi and see if the issue is fixed?
For some reason, even with your exact config, I couldn't reproduce the issue, so I'll still need to try to figure out what the difference is there.

The issue is still present.

The issue is no longer present after editing the manifest.yaml file in %userprofile%\.config\ludusavi\ manually.

Ah, sorry, it looks like I hit an error when trying to push the data update, and it didn't actually go through. It should update properly the next time you open Ludusavi now.

Yup, Ludusavi recreated the file now (it didn't even launch previously). Issue no longer present.

Great! I'll try to add some more safeguards in the importer script to handle bad paths like that before they make it into the manifest.
As far as why it behaved differently on our systems and why you got different results from the GUI and CLI, would you be willing to try a custom version of Ludusavi with some diagnostic info? (No worries if you'd rather not.)

Sure.

Thanks! Here it is (Windows 64-bit):
ludusavi-issue-36-test-1.zip
This build will create a %USERPROFILE%\.config\ludusavi\log folder with some log files (it adds a new one each time you run it). It's set up to only log activity for Battlefield 2 in particular.
For the test, please also replace the customGames: [] line in config.yaml with this, in order to override the path for Battlefield 2 back to the bad version:
customGames:
  - name: Battlefield 2
    files:
      - "<winDocuments>/Battlefield 2/Profiles/<home>"
    registry: []

Then just try the same things you did in the GUI and CLI before and post the logs here.

Here are the logs (CLI backup went well, GUI backup detected "Battlefield 2" and froze after ~11 GB of backup:
log.zip

Got it; could you check if this version solves the issue? I believe I have a fix that will prevent this particular problem from happening again:
ludusavi-issue-36-test-2.zip
Basically, when C: was in the middle of a path, it caused all of the path beforehand to get truncated. I thought an invalid path like <winDocuments>/<home> would be handled as C:\Users\Admin\Documents\C:\Users\Admin and just not find anything, but it was really being handled as C:\Users\Admin. The fix I made is to treat it like C:\Users\Admin\Documents\C_\Users\Admin (replace : with _), which avoids the truncation issue and should just not find any matching files or folders.
Now, I'm still not sure why that didn't happen on my system or why the GUI and CLI were different (my code at fault here is the same for both). My only guess is that it's somehow a quirk in the dependency I'm using to handle globs.

This one works properly.

Awesome! Thanks again for testing it. I'll include the fix in the next release.


Updates to flesh, ashib, mtc.

Updates to flesh, ashib, mtc.

removing since it doesn't like adding fleshFetcher to index.ts


Allow setting scheduleDelay for BatchSpanProcessor

BatchSpanProcessor seems to have option to configure exporting frequency, defaulting to 5s.
SplunkRum.initialize always uses the defaults, hence it's impossible to change that parameter - and 5s might be too often for us.
It seems that adding a simple configuration param should solve this issue.

Release 0.11.1 adds bspScheduleDelay to address this.


Questions about multi-gradient steps

Hi, thank you for providing great implementations! I've learned a lot from this repo, which is pretty easy-understanding and fast. My question is that for the 2d-navigation task, I trained num_steps=5 and tested it, but the results are quite different from those in the original paper.
I edited the test.py code like the followings:
# test.py
...
for batch in trange(args.num_batches):
        tasks = sampler.sample_tasks(num_tasks=args.meta_batch_size)
        train_episodes, valid_episodes = sampler.sample(tasks,
                                                        num_steps=config['num-steps'], # num_steps=5
                                                        fast_lr=config['fast-lr'],
                                                        gamma=config['gamma'],
                                                        gae_lambda=config['gae-lambda'],
                                                        device=args.device)

        logs['tasks'].extend(tasks)
        grad0_returns.append(get_returns(train_episodes[0]))
        grad1_returns.append(get_returns(train_episodes[1]))
        grad2_returns.append(get_returns(train_episodes[2]))
        grad3_returns.append(get_returns(train_episodes[3]))
        grad4_returns.append(get_returns(train_episodes[4]))
...
logs['grad0_returns'] = np.concatenate(grad0_returns, axis=0)
logs['grad1_returns'] = np.concatenate(grad1_returns, axis=0)
...

after saw https://github.com/tristandeleu/pytorch-maml-rl/issues/26#issuecomment-573679714.
To see the results, I did something like this.
...
data = np.load('path-to-results')
grad0_returns = data['grad0_returns']
grad1_returns = data['grad1_returns']
...
val0 = grad0_returns.mean()
val1 = grad1_returns.mean()
...

However, as the figure shows, the result values are far from what we want.

Figure from the paper.
I also tested just 1 gradient step, which shows about -10, similar to the original paper. Only more gradient steps are the problem.
And one more thing, the paper says for evaluation, they used fast learning rate=0.1 for 1 gradient step, then halved it to 0.05 for all future. But in this implementation, I can't find out that. Isn't this a critical thing? Since I didn't check this, so now I'm struggling to modify the codes to follow the original paper.
Thank you very much in advance!

Thank you for the kind words!
It looks like in the MAML paper they only used a single step of gradient descent during training for all experiments in RL (so I guess including the one for 2D-Navigation). From Appendix A.2:

In all reinforcement learning experiments, the MAML policy was trained using a single gradient step with α = 0.1.

That seems to be consistent with what you find (a return of -10 after 1 step). The results with multiple gradient steps must then be limited to evaluation, and you're right there is a special evaluation scheme. What you found is very interesting though: it seems that training directly with 5 steps of adaptation is not performing as well as using only 1 single step (there is some kind of "overfitting" happening where the performance decreases on the 4 gradient step).
Now for using different learning rates for each step, I don't think there's an easy way to do that with the current code unfortunately. However it shouldn't be too hard to modify the fast_lr argument to accept a list of learning rates. I can take a look at that!

Thank you for your quick reply and consideration!
Yeah, I also considered the overfitting problem. But in that case, the result value after 1 gradient step would be around -10, not -25 as in the resulting graph because when 1 gradient step is done, the average return should be at least -10. This discrepancy causes some curiosity so I run some experiments with different codes. If I find something new, I will let you know.
Thanks again!