Inconsistent behavior with compound configurations depending on both settings.json and launch.json configurations

VSCode Version: 1.36.1
OS Version: MacOS 10.14.5

Scenario
Extension user reported a problem with a compound configuration not working correctly with a custom extension using vscode.debug.startDebugging. The extension creates a command  that sets up the debugging environment (e.g. starts services) before launching a given debug configuration by name.
Steps to Reproduce

Create a new workspace (any target should do).

Define a configuration structure similar to this:

settings.json
...
"launch": {
  "configurations": [
    { "name": "Config_A", ... }
  ],
  "compounds": [
     { name: "Compound", "configurations": [ "Config_A", "Config_B" ] }
  ]
}
...

launch.json
...
"configurations": [
  { name: "Config_B" , ...  }
]
...

From the Debug drop down, select "Compound" and run.
This works.

Create an extension that runs this line (a workspace must be loaded): vscode.debug.startDebugging(vscode.workspace.workspaceFolders[0], 'Compound')
This fails with "Configuration 'Compound' is missing in 'launch.json'".

Questions

Should step 2 (run from drop down) work? It seems to me it should not:

According to this page: https://code.visualstudio.com/Docs/editor/debugging, Tip: If a workspace contains a "launch.json", the global launch configuration is ignored.

According to WorkspaceConfiguration.inspect documentation,  local values (launch.json) should override global (settings.json). This works strictly as documented. For the provided repro: Workspace.getConfiguration('launch').get('configuration') only returns configurations in launch.json (i.e. arrays are not merged),  Workspace.getConfiguration('launch').get('compounds') only returns compound configurations in settings.json (i.e. because launch.json does not define a compounds value). Using WorkspaceConfiguration.inspect  I can verify all configuration values exists and are overriden as intended.

If Step 2 should work, how to run step 3? i.e. how to use vscode.debug.startDebugging to call 'compound' successfully?

If step 3 should not work, neither should step 2. Correct?

The idea would be for vscode.debug.startDebugging to have behavior parity with the Debug  dropdown, whatever the proper behavior happens to be. Extension users see it as an extension bug when behavior is not equivalent.
Having said that, it is not a P1 bug for sure but would it be possible to update documentation to call out this difference?

I agree that there should be parity. And we will aim for this.
I think updating docs makes sense and we accept PRs in our vscode-docs repository https://github.com/microsoft/vscode-docs


Document ability to configure cargo-new.vcs = "hg"

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @steveklabnik (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

@bors: r+
On Monday, August 15, 2016, Rust highfive robot<EMAIL_ADDRESS>wrote:

Thanks for the pull request, and welcome! The Rust team is excited to
review your changes, and you should hear from @steveklabnik
https://github.com/steveklabnik (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra
commits. This ensures that the reviewer can see what has changed since they
last reviewed the code. Due to the way GitHub handles out-of-date commits,
this should also make it reasonably obvious what issues have or haven't
been addressed. Large or tricky changes may require several passes of
review and changes.
Please see the contribution instructions
https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md for more
information.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/rust-lang/cargo/pull/2994#issuecomment-239889155, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAD95HiT1JtUtLAi8q5Q64wxI88ssBm4ks5qgLJqgaJpZM4Jkpb0
.

:pushpin: Commit 5152d37 has been approved by alexcrichton

:hourglass: Testing commit 5152d37 with merge 2cda339...

:sunny: Test successful - cargo-cross-linux, cargo-linux-32, cargo-linux-64, cargo-mac-32, cargo-mac-64, cargo-win-gnu-32, cargo-win-gnu-64, cargo-win-msvc-32, cargo-win-msvc-64
Approved by: alexcrichton
Pushing 2cda339cc23dda36ac247a34d28adcd30beead00 to master...


First full draft of tutorial scripting

Known issues:
Camera triggers a little far away but also doesn't trigger when super close. This is based on the way the must_see trigger functionality is implimented.
When the mission is complete the loadenviroment function needs to be called.
The player can double back through the dark cave after finding the robot. Added some dialog to deter them doing this but it is possible. Consider another gate.

Are we happy to merge?


Moniker/ProductCode: OpenJS.NodeJS version 12.17.0

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

What does this error means, I have not seen it before...

"Validation-Domain" has to do with one or more of the domains for one or more of the URLs in the manifest. Now that we have waivers in place, we can be more transparent about policy checks and exceptions. You can see any granted waivers in the .validation files for packages with waivers. Over the coming weeks there will likely be a few more "new" labels. We're also working with some of our upstream dependencies to get better logging and output for validation as well.

/azp run


VIVI-13774 Remove Clock Shadows

The abundance of complex shadows on the clock widget was causing memory to be over-allocated on the new bromite webview (introduced in 3.8.4). This created a memory leak which would result in boxes eventually crashing, often during exams.
If we're to deliver a solution without a box update, a new widgets bundle is desirable. In turn, shadows have been removed, and the clock background has been replaced with an svg to emulate the original look. It's worth noting this change will be reverted whenever the webview is upgraded to a safer version again. Comparison is provided below:
Before:

After:

Such a weird bug, well done tracking this down! Keen to hear the story of how you ended up debugging it


Broken again!

This is happening when starting into SE itself.  I don't have any mods loaded except for most of the plugins (building a custom world).
XYR29Z2F.log

Please note, I really love this plugin, and already have builds with it enabled, in order to be able to play first-person-only!  I'd be really sad if this went away for good!

The log file you attached doesn't look like it has anything to do with SE. Where did you get it? What is happening when the game starts?


Preview Revision page

Card: https://github.com/bcgov/cas-cif/issues/1582

Work moved to new PR https://github.com/bcgov/cas-cif/pull/1650 , rebase got too messy


Referencing audiojs objects from other scripts

I may just be dense.  I've created a bunch of audiojs objects on a page, and they work great.   However, I'd like to also set up links that make the player jump to a specific point... I just don't know how to reference the created objects.
I've got the js and swfs working in the header, and in the page, I have...
<audio src="x.mp3" id="x1" />
<audio src="y.mp3" id="y1" />
What I'd like to add is
start x at 50%
start y at 50%
Is that even possible?
Thanks!

You can do:
var as = audiojs.createAll();

Then, as will be an array of audiojs objects which can be accessed through as[0], as[1], etc.


Can support all APIs.

This library does not support collection API and video upload.
Exposing this method allows you to be more flexible with various APIs and new APIs.

Apologies for the late review. The direct implementation of getJSON and postJSON is hidden as an implementation detail. Any additions to the API should be introduced directly instead of using this. Please file a PR on the actual missing APIs instead.


Error: "Could not find a declaration file for module 'vue-scrollto'."

I am using Vue with Typescript and keep getting this error. I've tried running npm install @types/vue-scrollto but that didn't fix the problem. This is how my main.ts file looks like:
import Vue from 'vue';
import App from './App.vue';
import router from './router';
import VueScrollTo from 'vue-scrollto';

Vue.config.productionTip = false;

Vue.use(VueScrollTo, {
   container: '#document',
   duration: 500,
   easing: 'ease',
   offset: 0,
   force: true,
   cancelable: true,
   onStart: false,
   onDone: false,
   onCancel: false,
   x: false,
   y: true,
});

new Vue({
   router,
   render: (h) => h(App),
}).$mount('#app');

Any help would be appreciated.

I'm not familiar with how the @types/ packages are supposed to work, and I'm not sure what's missing. From their docs seems like it should be working unless you are using an older version of typescript.

You can use require instead of import, it's working for me:
// tslint:disable-next-line:no-var-requires
const VueScrollTo = require('vue-scrollto');

Vue.use(VueScrollTo);

@Bruno7kp Thanks for the tipp!
But somehow the error was gone when I started my computer the next. I have not the slightest clue what happened. I did not change anything.


[FTL-1589] Remove tsdx from core SDK

Re-send of #175 from this repository.

Should be all addressed now


Attack success rate evaluation

Hello, Thanks so much for your open source benchmark. I appreciate your work.
I want to clarify something. How do you calculate the success rate? How did you get those success rate percentages?
Another related question is, I understood that the training set contains 25 poisoned samples. How about the test set? How many poisoned examples are they tested during the evaluation of the attack?
Thanks a lot

How do you calculate success rate?

For each test there is a single target image. If a model trained on poisoned data classifies that target image into the desired class (the label the attacker wishes to give the target image, and not its ground truth class) then the attack is a success. This measurement is binary, but over a batch of tests (each including a round of training and evaluation on a new target image) the portion of success is called the "success rate." Is this explanation clear?

How about the test set? How many poisoned examples are they tested during the evaluation of the attack?

If I'm understanding this question correctly to mean "How many images in the test set are targets of the attack?" then the answer is 1. For each test a single target image is chosen. By 2023 standards in the poisoning and backdoor literature this is a relatively simple setting for the attacker. At the time of publication, most poisoning methods had been developed or tested in this setting.
Let me know if these answers are satisfactory.

Yes thanks a lot!


Add capability using the Yarn PnP

Yarn PnP uses custom machinery for storing module. This breaks the TestCafe.

The possible fix is here

We should add tests for this scenario


Install fails 2.14.x

The pypi page is not loading as well:
https://pypi.python.org/pypi/requests/2.14.1
Logs:
07:58:51 Processing requests-2.14.1.tar.gz
07:58:51 Writing /tmp/easy_install-WACg1D/requests-2.14.1/setup.cfg
07:58:51 Running requests-2.14.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-WACg1D/requests-2.14.1/egg-dist-tmp-MeAs3M
07:58:51
07:58:51 stderr: no previously-included directories found matching 'docs/_build'
07:58:51 warning: no previously-included files matching '.py[cdo]' found anywhere in distribution
07:58:51 warning: no previously-included files matching 'pycache' found anywhere in distribution
07:58:51 warning: no previously-included files matching '.so' found anywhere in distribution
07:58:51 warning: no previously-included files matching '*.pyd' found anywhere in distribution
07:58:51 zip_safe flag not set; analyzing archive contents...
07:58:51 error: Setup script exited with error in requests setup command: Invalid environment marker: sys_platform == "win32" and python_version<"3.3"

Thanks, I think this is an issue with your system. What OS are you running, and can you print out the version of pip you have installed please? This is almost certainly related to #4006.

Requests does not support being installed via easy_install. Have you tried running pip install cloudmonkey?

@Lukasa okay will amend changes, looks like the failure happens when sys_platform is not defined,  pip install --upgrade distribute may fix it.

Thanks @Lukasa for your support, using pip fixed the issue for me. Though bear in mind, our scripts used to work well with requests 1.13.0, so something has changed.
I'll close the ticket now.

See the discussion in #4006 for an idea of what has changed.


How to correctly return null values from a resolver?

I have a resolver that may return a null value, and the corresponding schema definition allows to return null values, but my client receives an error showing the following message: "Object reference not set to an instance of an object".
The data field contains the right data, but the error stills comes, even if null is a allowed response.
Is there a correct way to setup de server to not throw this error?

can you paste you code?

I will try to replicate it in a small sample and post here.

I could not replicate the error in a small sample, but I can paste the stack trace here:
The response I receive is this one:
{
"errors": [
{
"message": "Unexpected Execution Error",
"locations": [
{
"line": 10,
"column": 6
}
],
"path": [
"estimativa",
"cliente"
],
"extensions": {
"message": "Object reference not set to an instance of an object.",
"stackTrace": "   at HotChocolate.Resolvers.CodeGeneration.___CompiledResolvers__9163361aef8544be859a761587d56374.<>c.<<-cctor>b__60_21>d.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at HotChocolate.Configuration.ResolverRegistry.<>c__DisplayClass21_0.<b__0>d.MoveNext() in C:\hc\src\Core\Types\Configuration\ResolverRegistry.cs:line 228\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at HotChocolate.Execution.ExecutionStrategyBase.ExecuteFieldMiddlewareAsync(ResolverTask resolverTask) in C:\hc\src\Core\Core\Execution\ExecutionStrategyBase.Resolver.cs:line 97\r\n   at HotChocolate.Execution.ExecutionStrategyBase.ExecuteMiddlewareAsync(ResolverTask resolverTask, IErrorHandler errorHandler) in C:\hc\src\Core\Core\Execution\ExecutionStrategyBase.Resolver.cs:line 54"
}
}
],
"data": {
"estimativa": {
"id": "xxxxx",
"codigo": "xxxxxx",
"responsavel": {
"nome": "xxxxxx",
},
"cliente": null
}
}
}

Oh, I found the problem :)
This is the method:
        public Task<IEmpregado> GetCliente(IEstimativa source, [Service] IForcaTrabalhoService forcaTrabalhoService)
        {
            if (source.Cliente != null)
            {
                return forcaTrabalhoService.ObterPorChaveAsync(source.Cliente);
            }
            return null;
        }

I should be returning Task.FromResult(null), because my method is not declared as async.
Lesson learned :)
Closing the issue. Unless you think the error message should be better here.

I will look into that since others could fall into that trap.

I might create a custom error for that

A fix for this one will be included into 0.8.0-preview.11

This one is now fixed. Since tasks will in the end be awaited by the query engine we are now returning
null if the resolver task is null.


Arrival day

Is your feature request related to a problem? Please describe.
For same date ranges the arrival day should be enforced. The user shouldn't select any other days of the week than those allowed (e.g. only Mondays), but can still freely select the departure date (e.g. being a Friday).
The bookedDays looks like as it does exactly what I described. However, it is not possible to select a start date in front of a "booked day" and then select an end date after the "booked day" as it should be possible, as that option is described to lock the start selection, not the end selection and the disallowLockDaysInRange option is for lockedDays, not bookedDays!
You can reproduce this issue in this pen: https://codepen.io/strarsis/pen/ZEQzreb
One booked day is set up, 10.06.2020. It is not possible to select a start day before it and and an end day after it.
Describe the solution you'd like
A more flexible mechanism like a callback for allowing the selectability of days in calendar or a dedicated option (which belongs to hotel mode).
Describe alternatives you've considered
Disallowing those dates, though this is also a recurring day in the week, so this can be quite wasteful/complicated. Another issue is that those disallowed days of week look like unavailable days of the week, which is confusing. It would be better if those days that are not allowed as a day of the week are a bit greyed out, but not just locked.
Additional context
Basically I want to select + style (and change state (selectable to unselectable)) of each calendar item, some functions, and a callback would offer the most flexible way doing so.

Addendum: Being able to specify a date/time period and then apply rules, e.g. "locked" or custom classes to the days/blocks that match would make all the things like arrival days possible.

possibly related to #72


Application feature

[x] 以网页形式进行预览和选择

[x] https://github.com/zoo-js/zoo-issue-helper  预览：https://zoo-js.github.io/zoo-issue-helper/
[ ] 网站优化

[ ] 组织自动邀请

https://github.com/thundergolfer/automated-github-organization-invites

https://docs.github.com/en/free-pro-team@latest/rest/reference/orgs#create-an-organization-invitation


add assay codes to assay abbr

Add "epigen-atac-seq" and "epigen-rrbs" to names of "assay_abbr"

@biodavidjm can you please review and merge PR? it looks like the "add reviewer" function is disabled


As a Backstage user, there's no visual difference between experimental and production entities

Currently, experimental and production entities have the same banner styles; the only difference between the two is under "Lifecycle" which a user can easily miss:
experimental entity

production entity

Feature Suggestion
Differentiate styles for experimental vs. production entities. Possible suggestions:

Different banner colour for experimental entites?
Use different coloured tags for Lifecycle?

Context
I was working with an engineer to improve their information in Backstage and they were surprised how there's no visual difference between experimental and production entities.

It makes sense to me to have a more obvious visual pattern for indicating the lifecycle. However, one thing I'll point out is that lifecycle can be any value and an organization can make up their own list. So experimental is only a suggestion. However, it's likely this is commonly used by most organizations I would guess. So, it might be that the theme would need to accommodate this.

I’m interested in exploring some options here if no one else is working on it.

Please do! I'll assign you @jkjetty

Thanks! Didn't make a ton of progress yet, but I'm liking highlighting the label for any non-prod entity like this so far. Thoughts?

Thanks! Didn't make a ton of progress yet, but I'm liking highlighting the label for any non-prod entity like this so far. Thoughts?

Love this @jkjetty!! I think any difference between experimental and production is better than nothing. Or perhaps making the highlight for production entities only? Kinda like how required fields are usually in bold or with an asterisk (*).

I'd suggest a small icon to the left/right of the word instead of a border. Perhaps even leveraging the builtin <StatusOK /> and <StatusWarning /> or similar for it? Or leaving it unmodified for the production lifecycle.


Add health check endpoint and tests

Issue Description
healthcheck endpoint for discord service. its going to be a get request
Expected Behavior

Current Behavior
none
Screenshots

Reproducibility

[ ] This issue is reproducible
[x] This issue is not reproducible

Steps to Reproduce

Severity/Priority

[ ] Critical
[ ] High
[ ] Medium
[x] Low

Additional Information

Checklist

[x] I have read and followed the project's code of conduct.
[x] I have searched for similar issues before creating this one.
[x] I have provided all the necessary information to understand and reproduce the issue.
[x] I am willing to contribute to the resolution of this issue.

Add test coverage as well

Add test coverage as well

i had already write tests for it


fix: automatically determine the number of role policy arns

Description

Changed to automatically calculate number_of_role_policy_arns even if you omit this.
Added backward compatibility to #116
Motivation and Context

By default the number_of_role_policy_arns should be calculated automatically, but it can be also specified explicitly as variable.
Breaking Changes

Probably not
How Has This Been Tested?

Use new example.

Thanks @yukin01 !
v3.5.0 has been just released.


Shifter: Quadruped template: Metacarpals are non-existant.

This issue was moved to mgear-dev/shifter#10


Unclear which column of the table command should be used as [NAME OF DATASET] in download command

When I run the help for the dataset download command, I get
% readii-datasets radiomicsets download -h
Usage: readii-datasets radiomicsets download [OPTIONS] [NAME OF DATASET]

  Download a file for this dataset.

and then the dataset table command has these headings:
% readii-datasets radiomicsets table
                                      Radiomicsets
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Name                             ┃ Dataset Name            ┃ Date Created ┃ Datatypes ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ HNSCC_Features                   │ HNSCC                   │ 2024-11-20   │ radiomics │
│ RADCURE_Features                 │ RADCURE                 │ 2024-11-20   │ radiomics │
│ HEAD-NECK-RADIOMICS-HN1_Features │ HEAD-NECK-RADIOMICS-HN1 │ 2024-11-20   │ radiomics │
└──────────────────────────────────┴─────────────────────────┴──────────────┴───────────┘

so it's not immediately clear what [NAME OF DATASET] is referring to.
I know the list function shows what's in the Name column, but even that outputs
radiomicsets:
  - HNSCC_Features
  - RADCURE_Features
  - HEAD-NECK-RADIOMICS-HN1_Features

which doesn't have [NAME OF DATASET] either.
Suggestion

Change the heading names in the table command to make Name and Dataset Name more distinguishable. Maybe

Name --> radiomicset Name or Orcestra Dataset Name
Dataset Name --> Original Dataset Name or Processed Dataset Name

Update the download help message to better reflect the table output

@strixy16 greeeat point,
honestly this is definitely confusing, its not intuitive at all, and I think your naming conventions are way better.
➜ orcestra radiomicsets table
                                      Radiomicsets
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Orcestra Dataset Name            ┃ Original Dataset Name   ┃ Date Created ┃ Datatypes ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ HNSCC_Features                   │ HNSCC                   │ 2024-11-20   │ radiomics │
│ RADCURE_Features                 │ RADCURE                 │ 2024-11-20   │ radiomics │
│ HEAD-NECK-RADIOMICS-HN1_Features │ HEAD-NECK-RADIOMICS-HN1 │ 2024-11-20   │ radiomics │
└──────────────────────────────────┴─────────────────────────┴──────────────┴───────────┘

and
➜ orcestra radiomicsets download --help
Usage: orcestra radiomicsets download [OPTIONS] [ORCESTRA DATASET NAME]

  Download a file for this dataset.


Support parsing categories from setup.cfg

The categories provided in the metadata will be parsed as a string, but a list is required.
It is now allowed to provide a single category that is parsed as a list.
Or the list separated by newlines will be parsed as a list.

Is the categories field defined in PEP, or is it an AiiDAlab specific thing? If it is the former, we should adhere to how it is defined in the "standard" I guess (e.g. if it is defined as a list, I would probably not support a single string here).

Is the categories field defined in PEP

No, it is not in PEP at least not in pep426 used in this repo. I think the format read from setup.cfg conform with https://setuptools.pypa.io/en/latest/userguide/declarative_config.html. We need to update it after moving to pyproject.toml. But thanks to this issue I have a close look at the code base, and I'll work on it after we have pyproject.toml supported.

@unkcpz can you verify that this fixes the issues described in #257?

@unkcpz can you verify that this fixes the issues described in https://github.com/aiidalab/aiidalab/issues/257?

Yes, this is exactly the issue I gonna fixed.


Add upload permissions for hyper-build-step-plugin

Corresponding repository / POM:
https://github.com/jenkinsci/hyper-build-step-plugin/blob/master/pom.xml#L11...L12
@Jimmy-Xu is one of the maintainers and has commit access to that repo.

@Jimmy-Xu Permissions should be active within 5 minutes.


(ORCHESTRATE-2667) Integrate with SAS Viya Orchestration tooling for SAS Viya install and uninstall

This one is also closed given the nature of the branch scheme. This is the work done for SAS Orchestration tooling integration.

This will be easier to track if this change is not coming from a fork, going to merge this into IAC-552 and continue work from there. Will comment the new PR number once this is complete.

See: https://github.com/sassoftware/viya4-deployment/pull/276 for the new PR


Test how sdbth runs with projects other than well

Still trying to dockerize CoderDojo zen platform: https://github.com/CoderDojo/cp-local-development
Will continue on the next day

Tried a lot, but won't be able to connect sdbth with zen without reliable internet connection


Level 5 Gyatt Rizz Copypasta or Level 5 Gyatt Rizz Livvy Dunne Rizzing Up Baby Gronk

Sometimes I wake up and think about skibidi toilet.  I build up with ecstasy when I hear that first note of a skibidi toilet video. God, I wish skibidi toilet was real.  But alas we live in this cruel cruel world without skibidi toilet. Words cannot describe how much I love it, My family has tried to make me stop watching them but when they do I bark and bite and claw.    Skibidi toilet is the only thing that brings me true joy anymore. I watch skibidi toilet any time I can. One time I saw Donald Trump and wondered what he'd look like as a skibidi toilet guy. The lore of skibidi toilet is so deep and complex and it turns me on so much. Imagine if skibidi toilet had breasts 🤤 . God, even the idea of it gets me hard. Please don't judge me, Skibidi toilet is really sexy and it's not my fault it decided to be that f-ing hot. If you disrespect skibidi toilet in my presence you're going to get a knuckle toilet pal 😡. I have the skibidi song playing 24/7 in my headphones to the point if even see/hear the letters s-k-i-b-i-d-i-t-o-i-l-e-t I start acting out. Skibidi toilet makes me foam at the mouth. God, sometimes I wish skibidi toilet would pop out my toilet one day. I would happily let it turn me into one of it's kind. I wonder in the afterlife if I'll see skibidi toilet.

erm. what the sigma?

you fell off + ratio


[bug]

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
When installing Talos Linux arm64 virtualised on macOS Apple Silicon, using QEMU via UTM which "employs Apple's Hypervisor virtualization framework"
The node registers, mislabelled, with Omni as platform:metal
otherwise.. works a treat! thankyou
Expected Behavior
The node registers, labelled; platform:QEMU or somesuch
Steps To Reproduce
On M1 or M2 Apple Mac, utilise Apple's Hypervisor virtualization framework, with https://github.com/utmapp/UTM
Provision new virtualised QEMU instance using Omni downloaded arm64.iso
What browsers are you seeing the problem on?
Safari
Anything else?
No response

The metal platform is a Talos way to call where it runs. Other platforms are aws, gcp, etc. So it's expected that bare-metal, VMs use a generic metal platform. It defines more the way the node boots up, acquires the machine config, rather than the hardware it runs on. Hardware is described via SMBIOS data, which might or might not be available in your case.

understood.
thanks
On 28 May 2023 at 19:40 +1000, Andrey Smirnov @.***>, wrote:

The metal platform is a Talos way to call where it runs. Other platforms are aws, gcp, etc. So it's expected that bare-metal, VMs use a generic metal platform. It defines more the way the node boots up, acquires the machine config, rather than the hardware it runs on. Hardware is described via SMBIOS data, which might or might not be available in your case.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>


Embedded bitmaps on HTML5 should be preloaded

If bitmap is embedded with :bitmap metadata, then it's loaded in runtime using this method: https://github.com/openfl/openfl/blob/master/openfl/display/BitmapData.hx#L1586
But this method loads bitmapdata asynchronously.
So following code silently fails:
@:bitmap('mybitmap.png') class MyBitmap extends BitmapData { }

var bmp = new MyBitmap(0, 0);
graphics.beginBitmapFill(bmp);
//this line breaks execution because at this point bmp.width & bmp.height is still not set
graphics.drawRect(0, 0, bmp.width,  bmp.height);
graphics.endFill();

Since there is no way to load images synchronously in javascript, i think they should be preloaded during standart assets preloading.

This is a tricky one.
You should be able to pass an "onload" handler in the constructor in order to receive when it is ready. If you want multiple instances of the same data, you need to be able to instantiate yourself. Also, for performance reasons, it may be preferable not to make them all instantiate at the beginning.
Maybe you're right. I think that @:bitmap is designed to cache the result the first time it is loaded. Perhaps all we need, then, is to instantiate each one with a load handler as part of the preload process

With previous versions of OpenFL (i think prior to 2.x) it was loaded synchronously. I was able to execute that code without errors.  Also on other targets bitmap data is immediately available right after instantiation.
So, i think, to make :bitmap consistent across all targets, preloading it during app start is the only way.

I'm going to mark this as WONTFIX for now, if anyone is interested in coming up with a solution to make these preload on HTML5, would be open to taking a look, but otherwise I would not recommend embedded image assets on HTML5


Performance issue on load with onBlockCountChange

Hello,
I'm loading SirTrevor with a saved state and ~60 blocks. Init was taking ~10s, most of it spent in multiple onBlockCountChange calls.
I disabled the function when ST isn't loaded yet and called it just one time after the initial load, and init time went back to an acceptable ~500ms.
Thanks!

We've also seen this problem on articles with large number of blocks - from around 40 to 140. The slowest area is onBlockCountChange, which could probably be deferred and run after a period of time (or when Sir Trevor is fairly certain no new blocks will be added).
The next slowest hotspot is focussing each block after it is rendered, on line 185 of block.js:
/* Generic implementation to tell us when the block is active */
focus: function() {
this.getTextBlock().focus();
},
After that, it's calculating the width or height of a block - it's unclear exactly where that is occurring though.
Is it worth making a pull request for this? Are there any plans for optimisation and batching when loading in large amounts of blocks (either 'manually' through the block manager function calls or by initialising/reinitialising sir trevor on a large dataset), so decisions such as focussing on each block and recalculating the block count can be ignored or deferred?

Thanks for your observations @williamheslamfuture. The focus behaviour is clearly wrong and we've encountered related undesirable behaviour because of this in the past. We'll be addressing this definitely.
We'll also be looking at other performance related issues — if you encounter any more, please do report them!


are there checksums (or other validation recomendation) ?

hello,
github runner images team here.
we are looking into improving supply chain security.
are there checksums available (or maybe other integrity validation) ?
cheers

Can you clarify where you are expecting checksums? Is it the bicep binaries on releases? Generated JSON? Artifacts we push and pull to/from ACR?

Closing as no response, please reopen with more information

sorry, I missed this issue.
we are looking for checksums (or other integrity validation) for bicep binaries


CLIP embedding generation

Closing because we already have this: https://github.com/ml6team/fondant/tree/main/components/embed_images


better problems

better randomized problems

.?

This doesn't really add any new problems, just rewrites old ones. Plus, there are merge conflicts, so I don't really see a point in merging. New problems have been added since this commit. I'll wait for gamester to make a decision before doing anything though.

two days ago there was no merge conflict and then someone decided to indent a paragraph or something. also, these problems don't just rewrite the random computational ones, they require more reading and a little thoughtpower? idk i just wanted something different but i guess i'll just wait until you get actual problems. and oops i need to fix the last problem


Add ability to skip run when using git configuration and git not present

In cases where a brand new project not yet committed to git or where dev/ops might run build outside of git control, it would be nice to have a setting to skip throwing errors when git variables cannot be calculated.  In this case, check early for presence of git and if not present, skip run writing output once that run was skipped.  Is this possible or will some assistance be needed to possibly bring this in?

Can't you use the license.skip flag and set it to the maven command-line or elsewhere when you detect that there is no .git folder ?


cmd/compile: function ABIs broke TestNexting

CL 147160 broke TestNesting. For example, https://build.golang.org/log/32264b507240ee2701ccd300c01463e1968f98bf
@dr2chase, is there anything obvious that would have caused this failure?

It happens somewhere after the compiler, so officially not obvious.

My working hypothesis is that something goes wrong here:
		pciterinit(ctxt, &pcstmt, &sym.Pcdata{P: s.FuncInfo.IsStmtSym.P})

		if pcstmt.done != 0 {
			// Assembly files lack a pcstmt section, we assume that every instruction
			// is a valid statement.
			pcstmt.value = 1
		}

in cmd/link/internal/ld/dwarf.go
The iterator is empty, so is_stmt is always true.

Thanks. I suspect where that's going wrong is
		s.FuncInfo.IsStmtSym = r.syms.Lookup(dwarf.IsStmtPrefix+s.Name, int(s.Version))

in objReader.readSym. The IsStmt symbol is probably always declared as version 0, but s.Version may be 1 now for an ABIInternal symbol.

This appears to be failing again for linux-amd64-longtest.
Latest failure: https://build.golang.org/log/ad327f4df87dc312a4840aa82dccbaf832822120

@dr2chase Can you look into this and let us know if it should still be a release blocker?


Quick fix to Travis e2e testing issues (running against a Docker backend)

Recently, all PRs began failing e2e tests (which run against a Docker-based REST API in Travis).  See for example the latest master build: https://travis-ci.com/github/dspace/dspace-angular/builds/167632878
This began when https://github.com/DSpace/DSpace/pull/2735 was merged, and I've tracked it down to a configuration problem for the Docker REST API backend.  Simply put, in https://github.com/DSpace/DSpace/pull/2735, the REST API CORS headers were updated to limit access to only URLs defined in rest.cors.allowed-origins, which only includes dspace.ui.url by default.
Unfortunately, in the REST API, currently dspace.ui.url still defaults to http://localhost:3000, while the frontend now defaults to port 4000 (as of #625)
So, this PR simply changes our Travis Docker REST API configuration to set dspace.ui.url=http://localhost:4000, which ensures the front end in Travis CI (running on port 4000) is allowed to access the backend running in Docker.
I'll be submitting a separate PR to the REST API to correct this port in the default dspace.cfg (now submitted, see https://github.com/DSpace/DSpace/pull/2772).  But this PR fixes the Travis build errors for the front end and ensures the dspace.ui.url is explicitly defined (rather than relying on defaults).
This PR only affects Travis CI and therefore will be merged immediately if Travis CI succeeds.

Merging immediately as Travis builds succeed now. So, this fixes the immediate build problem & any PRs seeing build issues can now be rebased & should succeed.
A corresponding REST API/backend PR has been created to correct the default settings there (along with Docker settings), see https://github.com/DSpace/DSpace/pull/2772


Add Color to the Triangle

Add color to the triangle. Implement the functionality to get source code from a glsl shader file.

git fetch origin
git checkout 1-add-color-to-the-triangle


Add support for PHP 8

I was just about to create a PR for this exact same change. If this could be merged in, that would be great!


can not sort by multiple columns

can we sort the grid by multiple columns, when I click the column title, it only sort by the clicked column and the other sorted column will be cleared, what should I do if I want to sort the grid data by multiple columns

Ok, I got it  To sort by multiple columns, you can use Shift+Click.


Fixes logging

🚀 🚀 Pull Request
Checklist:

[x]  My code follows the style guidelines of this project and the Contributing document
[x]  I have commented my code, particularly in hard-to-understand areas
[x]  I have kept the coverage-rate up
[x]  I have performed a self-review of my own code and resolved any problems
[x]  I have checked to ensure there aren't any other open Pull Requests for the same change
[x]  I have described and made corresponding changes to the relevant documentation
[x]  New and existing unit tests pass locally with my changes

Changes

@AbhinavTuli, here are a few linked PR's about this issue.
I had initally introduced the logger code to log when a dataset is created/loaded, in this PR (https://github.com/activeloopai/Hub/pull/993)
However, that produced a lot of redundant output while running tests. Hence, another PR was made to attempt to solve this (https://github.com/activeloopai/Hub/pull/997)
Finally, this PR (https://github.com/activeloopai/Hub/pull/1097) removes the default logging code as there were some issues with it. The above PR removes the same code introduced in this one. Is there any other workaround to solve this issue?


build: Initial spell check setup

Can one of the admins verify this patch?

Automation for azure-sdk-for-java

Automation for azure-sdk-for-js

Automation for azure-sdk-for-ruby

Automation for azure-sdk-for-go

Automation for azure-sdk-for-python

Assigning @sergey-shandar for CI related PRs.


[WIP]: dependencies: pin webargs to lower than 6.0.0

Lack of pinning causes:
[2020-01-07 19:54:01,089] ERROR in app: Exception on /records/ [GET]
Traceback (most recent call last):
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask_restful/__init__.py", line 269, in error_router
    return original_handler(e)
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_rest/views.py", line 240, in dispatch_request
    *args, **kwargs
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/flask/views.py", line 163, in dispatch_request
    return meth(*args, **kwargs)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_records_rest/views.py", line 429, in need_record_permission_decorator
    return f(self, record=record, *args, **kwargs)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_records_rest/views.py", line 471, in inner
    error_status_code=400,
TypeError: parse() got an unexpected keyword argument 'locations'
[pid: 257|app: 1|req: 1/4] <IP_ADDRESS> () {58 vars in 954 bytes} [Tue Jan  7 19:54:01 2020] GET /api/records/?page=1&size=20&q= => generated 182 bytes in 86 msecs (HTTP/1.1 500) 15 headers in 709 bytes (1 switches on core 0)

The upper limit is not set in invenio-rest. I'd propose to NOT merge this until the next release is imminent and see whats the way to go on invenio-rest.
Reported by Dan and Kai.

Closed via this: https://github.com/inveniosoftware/invenio-rest/pull/103


[computer use] Discussion about token use and potential for reducing unnecessary use of them

I don't know if there is a better place to have this discussion, so I apologize in advance for abusing this one, but after a bit more than a week of intensive use of computer use (which I think is an absolutely fantastic tool) I am hoping to find some ways to put Claude on some form of a token diet, since it is insatiable in gobbling up tokens during computer use and I am surely not the only one who is getting rate limited every day now after just a few hours of using it.
My token usage since I started computer use paints a pretty clear picture:

My first surprise since obviously I didn't read the fine menu is that cached prompts seem to account the same way that regular prompts do - so my first questions is: what is the benefit of prompt caching? Is it just to save a few KB of data upload? Is it to speed up the response time? What it does not seem to do is to reduce one's token use count.
My second question is: does computer use always send the entire (growing) communication thread to the API? Or do earlier user prompts or machine replies that get marked as "ephemeral" actually vanish from the input after a while? Could some intelligent (or manual) culling of older messages within a thread reduce the total token count?
A simple example would be one of the things that often happen, where Claude tries some bash tool and needs 3 attempts until it has figured out the right parameter scheme. Or it searches for a string in some files and finds it in the 3rd one. It is my impression that it could easily remove the failed attempts from the thread without a negative effect on the context.
So are there any ideas how to save on tokens in a typical workflow (apart from removing screenshots)?

I guess it helps to read the documentation (https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching)
Interesting:

Cache write tokens are 25% more expensive than base input tokens
Cache read tokens are 90% cheaper than base input tokens
The cache has a 5 minute time to live (TTL).
Changes that can break the cache include modifying any content, changing whether there are any images (anywhere in the prompt), and altering tool_choice.type. Any of these changes will require creating a new cache entry.

Do I interpret this correctly, that one one receives a "rate-limit" timeout for being over the token limit, the next API call to the API makes it actually worse (and more expensive) since by the the previously cached tokens have expired and will get cached again?
Also it seems that the strategy to only keep the last N screenshots might also interfere with the caching.

def _inject_prompt_caching(
    messages: list[BetaMessageParam],
):
    """
    Set cache breakpoints for the 3 most recent turns
    one cache breakpoint is left for tools/system prompt, to be shared across sessions
    """

    breakpoints_remaining = 3
    for message in reversed(messages):
        if message["role"] == "user" and isinstance(
            content := message["content"], list
        ):
            if breakpoints_remaining:
                breakpoints_remaining -= 1
                content[-1]["cache_control"] = BetaCacheControlEphemeralParam(
                    {"type": "ephemeral"}
                )
            else:
                content[-1].pop("cache_control", None)
                # we'll only every have one extra turn per loop
                break

Maybe I am missing something important, but the current caching strategy seems to be sub-optimal since first of all it will only cache user messages. The problem I see here is that a lot user messages are rather short - often they are just a confirmation or correction. But according to the documentation the minimum cacheable prompt length is 1024 tokens for Claude 3.5 Sonnet. The current algorithm does not take into account the message length at all meaning that it will blindly just cache the last 3 user messages even though several of them might not even be cacheable, dropping older messages that might be long enough to justify caching (since there are only 3 slots plus the one for the system prompt which is probably the most useful one)
Wouldn't it make more sense to cache the 3 longest messages, independent of whether they are user or machine prompts?
Of course in my ideal scenario Claude could determine itself which of the messages in the current thread are the most valuable ones to cache. But I guess that is wishful thinking.

I had to contact Claude sales after burning my daily within 4h of cloning the repo. My limit was magically lifted the next morning! YMMV. You'll need a non-gmail address to be able to get through to them.
AFAICS as the conversation grows, the prior convo is cached serverside, so the LLM can just load up the prior state and throw in the new tokens. And if you take > 5min in replying in your convo it's lost and the whole thing gets thrown through cold. That way multiple (rapid) agent-cycles are cheap but (slow) human cycles are still costly.

Are you sure that the entire conversation up to a breakpoint is cached? Maybe I misunderstand the cached query docs, but it is my impression that only those messages that are marked as such are cached. And only those that use at least 1024 tokens. If it was the whole convoy up to the breakpoint, what would be the point of having more than one breakpoint?

The new token counting endpoint seems to come at the right time for making the coaching strategy at least a little bit smarter: https://docs.anthropic.com/en/docs/build-with-claude/token-counting


Review new example (and skim old examples)

There were a few typos and formatting errors in the newest example. While editing it, I also did a quick review of the older examples--because there is always something that can be better!

@giovannipappalardiansys or @waltersma Would someone please review and accept this PR? Thanks!


HADOOP-17507. Add install notes GCC9, CMake 3.19

Added instructions for Ubuntu 14.04 and
Centos 8 regarding installation of GCC 9.3.0
and CMake 3.19.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 21s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

_ trunk Compile Tests _

+1 :green_heart:
shadedclient
23m 24s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
15m 31s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
asflicense
0m 29s

The patch does not generate ASF License warnings.

42m 28s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2663/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2663

Optional Tests
dupname asflicense

uname
Linux 89ce270fbbf1 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / ad483fd66e8

Max. process+thread count
572 (vs. ulimit of 5500)

modules
C: . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2663/1/console

versions
git=2.25.1 maven=3.6.3

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 21s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

_ trunk Compile Tests _

+1 :green_heart:
shadedclient
23m 24s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
15m 31s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
asflicense
0m 29s

The patch does not generate ASF License warnings.

42m 28s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2663/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2663

Optional Tests
dupname asflicense

uname
Linux 89ce270fbbf1 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / ad483fd66e8

Max. process+thread count
572 (vs. ulimit of 5500)

modules
C: . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2663/1/console

versions
git=2.25.1 maven=3.6.3

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  5s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

_ trunk Compile Tests _

+1 :green_heart:
shadedclient
23m  2s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
15m 39s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
asflicense
0m 31s

The patch does not generate ASF License warnings.

41m 59s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2663/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2663

Optional Tests
dupname asflicense

uname
Linux 96dfcbd4bb09 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 9801b78edf2

Max. process+thread count
584 (vs. ulimit of 5500)

modules
C: . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2663/2/console

versions
git=2.25.1 maven=3.6.3

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  5s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

_ trunk Compile Tests _

+1 :green_heart:
shadedclient
23m  2s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
15m 39s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
asflicense
0m 31s

The patch does not generate ASF License warnings.

41m 59s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2663/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2663

Optional Tests
dupname asflicense

uname
Linux 96dfcbd4bb09 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 9801b78edf2

Max. process+thread count
584 (vs. ulimit of 5500)

modules
C: . U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2663/2/console

versions
git=2.25.1 maven=3.6.3

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.


Nullable Parameters

I have found myself in need of writing a sql-query that had variable declared and assigned as NULL. However, the library implicitly does NOT assign NULL when a null is actually passed.
For example:
DECLARE @MyNumber SMALLINT = @pMyNumber

becomes
DECLARE @MyNumber SMALLINT = @pMyNumber

while we need it to be
DECLARE @MyNumber SMALLINT = NULL

We can achieve this via the following snippet. ( Not completely refactored and polished yet )
private static DatabaseCommand AddNullableParameter( this DatabaseCommand command, string parameterName, object parameterValue, DbType dbType )
        {
            object paramValue = parameterValue;

            if ( paramValue != null
                && parameterValue is string
                && string.IsNullOrWhiteSpace( parameterValue.ToString() ) )
            {
                paramValue = null;
            }

            command = paramValue == null
                ? command.AddParameter( parameterName, DBNull.Value )
                : command.AddParameter( parameterName, parameterValue, dbType );

            return command;
        }

Here, the usage of string.IsNullOrWhiteSpace can be argued against string.IsNullOrEmpty.
When AddNullableParameter is called, the caller's explicit intention of handling 'nullables' can be interpreted into the fact that when a parameter is a string, unless a valid value is supplied, it could be a null. In this case, whitespaces assigned to a 'nullable' , would be a NULL. Most variables could possibly get serialized from empty/default/null values to an empty or white-space string.
It can also be argued that in the context of NULLABLE Parameter in this library, when a white-spaced empty string should always be interpreted as a valid user input. Which would mean that the code should handle 'string' parameters' validation with string.IsNullOrEmpty.

Question - not quite sure how this would affect or even be compatible with the other DB technologies this lib support like ORACLE, PostGreSQL etc.
Any thoughts on this?

As noted in the following pull request comments one of the unit tests needs to be moved to a different project in order to get the continuous integration build to pass for this pull request.

I've merged the pull request and moved the unit tests to the correct project and labeled this the v0.5 release in the following commit: 4c970f75a6dd194cd9e1131a892b4e353648753d


RDF ACL for node

Along with authentication, RDF ACL authorization is an important feature for this server.  Then we will be able to run it facing the open net rather than as a protected proxy.
The goal would be to make the file structure used on the server  be compatible with gold, so that one could run node-ldp-httpd or gold on the same files and have redundancy of project, and no 'project lock-in'.
Also of course made sure node-rdf-acl works with any other express systems.

There's a limited ACL that already works (see acl.js). I wrote it using the acl node module but it's quite limited. For example, setting permissions for all users cannot be done with a wildcard. I will look at gold to see how ACL is implemented and write a compatible solution.

what is the status of this?

I am working on writing a test suite. The basic tests work.
On Jul 7, 2015 5:26 AM, "Nicola Greco"<EMAIL_ADDRESS>wrote:

what is the status of this?
—
Reply to this email directly or view it on GitHub
https://github.com/linkeddata/ldnode/issues/17#issuecomment-119135745.

Can we consider this close?
Or otherwise, can we list in this format all the features missing?
- [ ] not done
- [x] done

It's almost done but all the features involving fetching a graph from a URL currently don't work.

From any URL, or just from URLs hosted on servers having self-signed certs?

Actually, it's just https requests that don't work.
There's another bug in the WebID delegation. The test is supposed to add a delegates triple to the user's profile via a POST request. The requests return a 200 status code but the triple is not being added in the databox server so the next test fails but it correctly retrieves the profile (just without the triple that was meant to be added). Not sure how to solve this bug since I am not in control of the databox server.

@deiu: I found out what's causing the bug. I found a node library that can do sync requests and the code is correct but the test is hosting the group file in the same folder. Since the acl code is blocking waiting for the response, the server deadlocks when it gets the request for the group file.
I tested the file by spawning another ldnode server in the same folder but with a different port and retrieving the file from the second server. The acl module correctly retrieves the group file and passes all the tests. However, it's impossible to host the group file in the same server without rewriting the whole module and that would break other features (glob support for example).
I've thought of a workaround: before sending the request, the acl module checks if the domain of the server is the same as its own, and if so reads the file from the filesystem instead of doing a request over the network. I think that would require ldnode to know it's own domain/base URI though so I am not sure how it ties on what we talked about at the last meeting..

Closing for now. PR #87 passes all the tests except for one of the delegation tests which is commented out for now.
However the code for delegation is correct but it fails because my test users are hosted in databox.me and the server fails to add a delegate triple despite returning a 200 status code. Therefore, this issue is beyond us to fix unless I talk to @deiu or @sandhawke (who I think run the server) or ldnode implements an identity provider.
Everything else works as it should.


Metadata doesn't save

Hi, I am trying to upload a new dataset to GBIF. However, whenever I fill out the metadata section and save it, when I go back to the resource page, nothing is saved and I fill the info out all over again. I've done it about 5 times now...what am i doing wrong? Chris

@AMNHcjohnson Thank you for contacting us.
I need some more information please - what IPT version do you? Do you have any exceptions displayed?

Hi Mikhail,
It's Integrated Publishing Toolkit (IPT) Version 2.5.5-ra872e56
Probably out of date - it's also not showing the managed resource file I had uploaded. I didn't see any exception or anything that would indicate the information wouldn't save.
Thanks.
Chris
Chris Johnson, Ph.D.https://www.amnh.org/research/staff-directory/christine-johnson | Curatorial Associate | Division of Invertebrate Zoologyhttps://www.amnh.org/research/invertebrate-zoology | American Museum of Natural Historyhttps://www.amnh.org/research/invertebrate-zoology| 200 Central Park West | NYC NY 10024 | Tel<PHONE_NUMBER> |<EMAIL_ADDRESS>Looking to see if IZ has a specimen? Search first on our IZ online databasehttps://emu-prod.amnh.org/imulive/iz.html!
Digitizing Terrestrial Polyneoptera at AMNHhttps://www.amnh.org/research/invertebrate-zoology/collections/other-insect-orders/digitizing-terrestrial-crickets-earwigs-grasshoppers--roaches (NSF-PEN grant)
Digitizing Marine Invertebrates at AMNHhttps://www.amnh.org/research/invertebrate-zoology/collections/other-invertebrate-phyla/databasing-imaging-marine-invertebrates-collections-natural-history (NSF-TCN grant)
Managing Editor, Entomologica Americanahttps://nyentsocjournal.org/
Associate Editor, Journal of Negative Results - EEB
From: Mikhail Podolskiy @.>
Sent: Saturday, January 7, 2023 12:45 PM
To: gbif/ipt @.>
Cc: Christine Johnson @.>; Mention @.>
Subject: Re: [gbif/ipt] Metadata doesn't save (Issue #1925)
EXTERNAL SENDER
@AMNHcjohnsonhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAMNHcjohnson&data=05|01||ef45934b00364a70af0a08daf0d6e2f1|be0003e8c6b9496883aeb34586974b76|0|0|638087103010451693|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=JfZ75L4QTKfOq1jw7vRCTeAciMB%2FUMeGhP2PUk6mniE%3D&reserved=0 Thank you for contacting us.
I need some more information please - what IPT version do you? Do you have any exceptions displayed?

Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgbif%2Fipt%2Fissues%2F1925%23issuecomment-1374548441&data=05|01||ef45934b00364a70af0a08daf0d6e2f1|be0003e8c6b9496883aeb34586974b76|0|0|638087103010451693|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=DOSTWmwvvYNemSyK56%2BCW5o8TKf5yutkazPVlnAdRNY%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAH4PLA3DYHS7ZMGELVLUV4LWRGTRLANCNFSM6AAAAAATUBUELU&data=05|01||ef45934b00364a70af0a08daf0d6e2f1|be0003e8c6b9496883aeb34586974b76|0|0|638087103010451693|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=jyoTFv4oT5RJCfn5sDXiYaCpuPtrWif3fEbpP1bwOWU%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

I don't remember anything like that. Could you send me your IPT logs please? Or provide me with administrator rights for your IPT?
And, if possible, I would recommend you to update your IPT to the most recent version (2.6.3 currently)

Hi,
I will have to ask our IT department to upgrade to the new version. I am not sure where to go to get the log, but if you can tell below, the AMNH-Crustacea is not showing up in my managed resources, yet, I cannot "create a new resource" with the same
Name because it says it exists.
Where would I find the logs for this?
Thank.
Chris
@.***
@.***
Chris Johnson, Ph.D.https://www.amnh.org/research/staff-directory/christine-johnson | Curatorial Associate | Division of Invertebrate Zoologyhttps://www.amnh.org/research/invertebrate-zoology | American Museum of Natural Historyhttps://www.amnh.org/research/invertebrate-zoology| 200 Central Park West | NYC NY 10024 | Tel<PHONE_NUMBER> |<EMAIL_ADDRESS>Looking to see if IZ has a specimen? Search first on our IZ online databasehttps://emu-prod.amnh.org/imulive/iz.html!
Digitizing Terrestrial Polyneoptera at AMNHhttps://www.amnh.org/research/invertebrate-zoology/collections/other-insect-orders/digitizing-terrestrial-crickets-earwigs-grasshoppers--roaches (NSF-PEN grant)
Digitizing Marine Invertebrates at AMNHhttps://www.amnh.org/research/invertebrate-zoology/collections/other-invertebrate-phyla/databasing-imaging-marine-invertebrates-collections-natural-history (NSF-TCN grant)
Managing Editor, Entomologica Americanahttps://nyentsocjournal.org/
Associate Editor, Journal of Negative Results - EEB
From: Mikhail Podolskiy @.>
Sent: Saturday, January 7, 2023 1:10 PM
To: gbif/ipt @.>
Cc: Christine Johnson @.>; Mention @.>
Subject: Re: [gbif/ipt] Metadata doesn't save (Issue #1925)
EXTERNAL SENDER
I don't remember anything like that. Could you send me your IPT logs please? Or provide me with administrator rights for your IPT?
And, if possible, I would recommend you to update your IPT to the most recent version (2.6.3 currently)

Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgbif%2Fipt%2Fissues%2F1925%23issuecomment-1374557539&data=05|01||4a25e8aa5d3142ab38ef08daf0da6c06|be0003e8c6b9496883aeb34586974b76|0|0|638087118167534981|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=mC7Q1RAv5%2FefroiyqwqQ6%2BjrrElM3XjRJ8X%2ByUKlYWc%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAH4PLA6BLKTGRWL3BDCUTPDWRGWQJANCNFSM6AAAAAATUBUELU&data=05|01||4a25e8aa5d3142ab38ef08daf0da6c06|be0003e8c6b9496883aeb34586974b76|0|0|638087118167544936|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=ITWP0KC2U%2B9QaqiPwOCRulbZ5ump%2BbZWVFVljyrc2vQ%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

I'm sorry I don't quite understand, you can't see the resource now? Have you deleted it or it just disappeared?
Log file is available for the admin users in the Administration -> Logs, or you can download them directly from the server: IPT data dir ->  logs

Correct I don’t see it. I never deleted it.
Christine Johnson, Curatorial Associate
American Museum of Natural History
@.***

From: Mikhail Podolskiy @.>
Sent: Saturday, January 7, 2023 1:53:54 PM
To: gbif/ipt @.>
Cc: Christine Johnson @.>; Mention @.>
Subject: Re: [gbif/ipt] Metadata doesn't save (Issue #1925)
EXTERNAL SENDER
I'm sorry I don't quite understand, you can't see the resource now? Have you deleted it or it just disappeared?
Log file is available for the admin users in the Administration -> Logs, or you can download them directly from the server: IPT data dir -> logs
—
Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgbif%2Fipt%2Fissues%2F1925%23issuecomment-1374573726&data=05|01||014bb0b4b6eb4c12baa708daf0e08756|be0003e8c6b9496883aeb34586974b76|0|0|638087144441069866|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=WVc%2BArSE4JOux4lr6XbfwGh71L6N07pLhjhWkJTheDQ%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAH4PLA7ISRFX5CEK3PHQUSLWRG3UFANCNFSM6AAAAAATUBUELU&data=05|01||014bb0b4b6eb4c12baa708daf0e08756|be0003e8c6b9496883aeb34586974b76|0|0|638087144441069866|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=dgaVtdqSIAIgTPVgJ1lZ5Ek3sUeELUQy6BObWwfuof0%3D&reserved=0.
You are receiving this because you were mentioned.Message ID: @.***>

Hi Mikhail,
This morning when I logged in, I could see my Crustacea resource.
I went ahead and published it (or set it from private to public). How long does is take to determine whether everything is correct? In the log there are a few question marks – does that mean these are incorrect? Or is it still determining whether all is true?
In addition, I am a big confused as to why I have datasets visible on GBIF, but when I search under our institution code, the records don’t appear although the institution code field is populated in these files.
Here is the log:
Archive generation started for version #1.0
Start writing data file for Darwin Core Occurrence
No lines were skipped due to errors for mapping Darwin Core Occurrence in source amnhcrustaceacollection202317b
No lines were skipped due to errors for mapping Darwin Core Occurrence in source amnhcrustaceacollection202317b
No lines with fewer columns than mapped for mapping Darwin Core Occurrence in source amnhcrustaceacollection202317b
All lines match the filter criteria for mapping Darwin Core Occurrence in source amnhcrustaceacollection202317b
Data file written for Darwin Core Occurrence with 15285 records and 53 columns
All data files completed
EML file added
meta.xml archive descriptor written
Validating the core file: occurrence.txt. Depending on the number of records, this can take a while.
? Validating the core basisOfRecord is always present and its value matches the Darwin Core Type Vocabulary.
? Validating the core ID field occurrenceID is always present and unique.
No lines are missing occurrenceID
No lines have duplicate occurrenceID
✓ Validated each line has a occurrenceID, and each occurrenceID is unique
No lines are missing a basisOfRecord
All lines have basisOfRecord that matches the Darwin Core Type Vocabulary
No lines have ambiguous basisOfRecord 'occurrence'.
✓ Validated each line has a basisOfRecord, and each basisOfRecord matches the Darwin Core Type Vocabulary
Archive validated
Archive has been compressed
Archive version #1.0 generated successfully!
Chris Johnson, Ph.D.https://www.amnh.org/research/staff-directory/christine-johnson | Curatorial Associate | Division of Invertebrate Zoologyhttps://www.amnh.org/research/invertebrate-zoology | American Museum of Natural Historyhttps://www.amnh.org/research/invertebrate-zoology| 200 Central Park West | NYC NY 10024 | Tel<PHONE_NUMBER> |<EMAIL_ADDRESS>Looking to see if IZ has a specimen? Search first on our IZ online databasehttps://emu-prod.amnh.org/imulive/iz.html!
Digitizing Terrestrial Polyneoptera at AMNHhttps://www.amnh.org/research/invertebrate-zoology/collections/other-insect-orders/digitizing-terrestrial-crickets-earwigs-grasshoppers--roaches (NSF-PEN grant)
Digitizing Marine Invertebrates at AMNHhttps://www.amnh.org/research/invertebrate-zoology/collections/other-invertebrate-phyla/databasing-imaging-marine-invertebrates-collections-natural-history (NSF-TCN grant)
Managing Editor, Entomologica Americanahttps://nyentsocjournal.org/
Associate Editor, Journal of Negative Results – EEB
From: Christine Johnson @.>
Sent: Saturday, January 7, 2023 1:55 PM
To: gbif/ipt @.>; gbif/ipt @.>
Cc: Mention @.>
Subject: Re: [gbif/ipt] Metadata doesn't save (Issue #1925)
Correct I don’t see it. I never deleted it.
Christine Johnson, Curatorial Associate
American Museum of Natural History
<EMAIL_ADDRESS>
From: Mikhail Podolskiy<EMAIL_ADDRESS>Sent: Saturday, January 7, 2023 1:53:54 PM
To: gbif/ipt<EMAIL_ADDRESS>Cc: Christine Johnson<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [gbif/ipt] Metadata doesn't save (Issue #1925)
EXTERNAL SENDER
I'm sorry I don't quite understand, you can't see the resource now? Have you deleted it or it just disappeared?
Log file is available for the admin users in the Administration -> Logs, or you can download them directly from the server: IPT data dir -> logs
—
Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgbif%2Fipt%2Fissues%2F1925%23issuecomment-1374573726&data=05|01||014bb0b4b6eb4c12baa708daf0e08756|be0003e8c6b9496883aeb34586974b76|0|0|638087144441069866|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=WVc%2BArSE4JOux4lr6XbfwGh71L6N07pLhjhWkJTheDQ%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAH4PLA7ISRFX5CEK3PHQUSLWRG3UFANCNFSM6AAAAAATUBUELU&data=05|01||014bb0b4b6eb4c12baa708daf0e08756|be0003e8c6b9496883aeb34586974b76|0|0|638087144441069866|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=dgaVtdqSIAIgTPVgJ1lZ5Ek3sUeELUQy6BObWwfuof0%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

I'm glad to hear you managed to publish your resource. Question mark in the publication log simply indicates that the validation process was started. As you can see further in the log the IPT reported all went successfully.
What is your dataset please? After publishing in the IPT it might take some time for the dataset to be indexed by GBIF.

Hi Mikhail,
Sorry for the bother again, but something still isn't working correctly. Although it shows there is a dataset, the dataset search comes up with 0 occurrences. Can someone please help me determine, why this is so?
Chris
@.***
From: Mikhail Podolskiy @.>
Sent: Monday, January 9, 2023 2:30 PM
To: gbif/ipt @.>
Cc: Christine Johnson @.>; Mention @.>
Subject: Re: [gbif/ipt] Metadata doesn't save (Issue #1925)
EXTERNAL SENDER
I'm glad to hear you managed to publish your resource. Question mark in the publication log simply indicates that the validation process was started. As you can see further in the log the IPT reported all went successfully.
What is your dataset please? After publishing in the IPT it might take some time for the dataset to be indexed by GBIF.

Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgbif%2Fipt%2Fissues%2F1925%23issuecomment-1376172135&data=05|01||9af04c246d8644f3868a08daf277e9dd|be0003e8c6b9496883aeb34586974b76|0|0|638088894098718683|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=taost6uE3cwUxRKgqt2akBphUwkkEFhOOtD5QLxfAqQ%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAH4PLA2JIATNSMNS3JSKUM3WRRRL3ANCNFSM6AAAAAATUBUELU&data=05|01||9af04c246d8644f3868a08daf277e9dd|be0003e8c6b9496883aeb34586974b76|0|0|638088894098718683|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=o9OPQ40wZXdML8WTkGGVnUSo4Pnbib5EaLngSIIrrZ4%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

Hi Again,
In my search for my Crustacea records, I came across this "finding" - something seems very off here. Crustace in GBIF backbone is a genus under the bee family Apidae.
It looks like the Benthic Baseline Biodiversity Survey has the wrong taxon string.
https://www.gbif.org/dataset/36449c1f-679d-4235-b34e-1c275ebcd968
Chris
Chris Johnson, Ph.D.https://www.amnh.org/research/staff-directory/christine-johnson | Curatorial Associate | Division of Invertebrate Zoologyhttps://www.amnh.org/research/invertebrate-zoology | American Museum of Natural Historyhttps://www.amnh.org/research/invertebrate-zoology| 200 Central Park West | NYC NY 20024 | Tel<PHONE_NUMBER> |<EMAIL_ADDRESS>Looking to see if IZ has a specimen? Search first on our IZ online databasehttps://emu-prod.amnh.org/imulive/iz.html!
Digitizing Terrestrial Polyneoptera at AMNHhttps://www.amnh.org/research/invertebrate-zoology/collections/other-insect-orders/digitizing-terrestrial-crickets-earwigs-grasshoppers--roaches (NSF-PEN grant)
Digitizing Marine Invertebrates at AMNHhttps://www.amnh.org/research/invertebrate-zoology/collections/other-invertebrate-phyla/databasing-imaging-marine-invertebrates-collections-natural-history (NSF-TCN grant)
Managing Editor, Entomologica Americanahttps://nyentsocjournal.org/
Associate Editor, Journal of Negative Results - EEB
@.***
From: Mikhail Podolskiy @.>
Sent: Monday, January 9, 2023 2:30 PM
To: gbif/ipt @.>
Cc: Christine Johnson @.>; Mention @.>
Subject: Re: [gbif/ipt] Metadata doesn't save (Issue #1925)
EXTERNAL SENDER
I'm glad to hear you managed to publish your resource. Question mark in the publication log simply indicates that the validation process was started. As you can see further in the log the IPT reported all went successfully.
What is your dataset please? After publishing in the IPT it might take some time for the dataset to be indexed by GBIF.

Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgbif%2Fipt%2Fissues%2F1925%23issuecomment-1376172135&data=05|01||9af04c246d8644f3868a08daf277e9dd|be0003e8c6b9496883aeb34586974b76|0|0|638088894098718683|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=taost6uE3cwUxRKgqt2akBphUwkkEFhOOtD5QLxfAqQ%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAH4PLA2JIATNSMNS3JSKUM3WRRRL3ANCNFSM6AAAAAATUBUELU&data=05|01||9af04c246d8644f3868a08daf277e9dd|be0003e8c6b9496883aeb34586974b76|0|0|638088894098718683|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=o9OPQ40wZXdML8WTkGGVnUSo4Pnbib5EaLngSIIrrZ4%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

@AMNHcjohnson I'm glad to help, but I don't know what dataset we're talking about. Could you send me the link please?
And, if possible, create an admin account in your IPT, that would help to diagnose what's going on.

Thanks Mikhail, @ManonGros has access - I need to request IT to add you as well. I already asked them to install the updated release but that hasn't happened yet.
What email should I give our IT for you?
I realize there is something wrong with my file - when I try to keep the dates from turning into text in excel, I think something else went awry with my file.
I'm always so close but never can get over this hurdle and I have 750K records I would like to share.
Thanks.
Chris
From: Mikhail Podolskiy @.>
Sent: Thursday, January 12, 2023 1:12 PM
To: gbif/ipt @.>
Cc: Christine Johnson @.>; Mention @.>
Subject: Re: [gbif/ipt] Metadata doesn't save (Issue #1925)
EXTERNAL SENDER
@ManonGroshttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FManonGros&data=05|01||e121a5140822451ac67708daf4c877a8|be0003e8c6b9496883aeb34586974b76|0|0|638091439147107511|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=RO0gJi6JxyzAUxv87kP8WGqhLf5uDxJBje2A55%2Bo55I%3D&reserved=0 Could you assist with this please?

Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgbif%2Fipt%2Fissues%2F1925%23issuecomment-1380804419&data=05|01||e121a5140822451ac67708daf4c877a8|be0003e8c6b9496883aeb34586974b76|0|0|638091439147107511|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=fHR2jNIvLbDpMF9DfSXx2p%2F%2BX8W2eTDFBe5ADNXiYXQ%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAH4PLA5W25RRSYKB3QDOG7TWSBCODANCNFSM6AAAAAATUBUELU&data=05|01||e121a5140822451ac67708daf4c877a8|be0003e8c6b9496883aeb34586974b76|0|0|638091439147107511|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=Cp748EJ87tCVLmjevNbbhdG1QXSTUSy72thY0uk9yJA%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

<EMAIL_ADDRESS>

Hi Mikhail,
Okay - our IT department has updated the IPT version & added you as a managed user (you should have received an email from them). I detected some errors in my upload file, which I fixed, and published a new dataset. It looks like everything is fine, however, I still see 0 occurrences for this dataset.
https://www.gbif.org/dataset/a8035a1d-e674-4d2a-bb59-b476af6a3d6d
Any help you can provide to identify the misstep would be appreciated (so I can go forward with our remaining datasets.
Best,
chris
From: Mikhail Podolskiy @.>
Sent: Thursday, January 12, 2023 1:18 PM
To: gbif/ipt @.>
Cc: Christine Johnson @.>; Mention @.>
Subject: Re: [gbif/ipt] Metadata doesn't save (Issue #1925)
EXTERNAL SENDER
<EMAIL_ADDRESS>

Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgbif%2Fipt%2Fissues%2F1925%23issuecomment-1380812487&data=05|01||5ec5c0e2bb2f4dbf0ae808daf4c9678d|be0003e8c6b9496883aeb34586974b76|0|0|638091443125325275|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=jNeYBc7xoT1rkcDKISIWHhqyr7DjB9HGV1sQy8etr9g%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAH4PLA4OBU4BEPXPXDMNY3DWSBDHJANCNFSM6AAAAAATUBUELU&data=05|01||5ec5c0e2bb2f4dbf0ae808daf4c9678d|be0003e8c6b9496883aeb34586974b76|0|0|638091443125325275|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=6mkFwJdFmn4npLvYX7mtdMgDD%2Fta2G%2FlE4hWltHL2Lo%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

Hi again Mikhail,
I really need help – I have tried to publish this dataset many, many times – the log says successful, the ingestion history say finishReason: ABORT.
I don’t understand what is wrong with the file that it can’t be publish.
The dataset is American Museum of Natural History (AMNH) Crustacea Collection.
Here are some links:
https://www.gbif.org/dataset/a8035a1d-e674-4d2a-bb59-b476af6a3d6d
https://registry.gbif.org/dataset/a8035a1d-e674-4d2a-bb59-b476af6a3d6d/ingestion-history
Thanks.
Chris
Current version
Pending version
Version
1.9
1.10
DOI

Published on
17 January 2023 10:19
Publication log
Download https://ipt.amnh.org/publicationlog.do?r=amnh_crustaceacollection
Publication report
Hidehttps://ipt.amnh.org/manage/resource?r=amnh_crustaceacollection#anchor-publish

10:19:37 Archive generation started for version #1.9
10:19:37 Start writing data file for Darwin Core Occurrence
10:19:38 Data file written for Darwin Core Occurrence with 15285 records and 55 columns
10:19:38 All data files completed
10:19:38 EML file added
10:19:38 meta.xml archive descriptor written
10:19:38 Validating the core file: occurrence.txt. Depending on the number of records, this can take a while.
10:19:38 ? Validating the core basisOfRecord is always present and its value matches the Darwin Core Type Vocabulary.
10:19:38 ? Validating the core ID field occurrenceID is always present and unique.
10:19:38 ✓ Validated each line has a occurrenceID, and each occurrenceID is unique
10:19:38 ✓ Validated each line has a basisOfRecord, and each basisOfRecord matches the Darwin Core Type Vocabulary
10:19:38 Archive validated
10:19:38 Archive has been compressed
10:19:38 Archive version #1.9 generated successfully!

@.***
@.***
Chris Johnson, Ph.D.https://www.amnh.org/research/staff-directory/christine-johnson | Curatorial Associate | Division of Invertebrate Zoologyhttps://www.amnh.org/research/invertebrate-zoology | American Museum of Natural Historyhttps://www.amnh.org/research/invertebrate-zoology| 200 Central Park West | NYC NY 10024 | Tel<PHONE_NUMBER> |<EMAIL_ADDRESS>Looking to see if IZ has a specimen? Search first on our IZ online databasehttps://emu-prod.amnh.org/imulive/iz.html!
Digitizing Terrestrial Polyneoptera at AMNHhttps://www.amnh.org/research/invertebrate-zoology/collections/other-insect-orders/digitizing-terrestrial-crickets-earwigs-grasshoppers--roaches (NSF-PEN grant)
Digitizing Marine Invertebrates at AMNHhttps://www.amnh.org/research/invertebrate-zoology/collections/other-invertebrate-phyla/databasing-imaging-marine-invertebrates-collections-natural-history (NSF-TCN grant)
Managing Editor, Entomologica Americanahttps://nyentsocjournal.org/
Associate Editor, Journal of Negative Results – EEB
From: Mikhail Podolskiy @.>
Sent: Thursday, January 12, 2023 1:05 PM
To: gbif/ipt @.>
Cc: Christine Johnson @.>; Mention @.>
Subject: Re: [gbif/ipt] Metadata doesn't save (Issue #1925)
EXTERNAL SENDER
@AMNHcjohnsonhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAMNHcjohnson&data=05|01||b6dd76ee85e2410fb6b708daf4c78c64|be0003e8c6b9496883aeb34586974b76|0|0|638091435157135460|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=ub1x%2FXPooniHj8hvyPpQ8aIStjKeJCqerCVCVnq2TvI%3D&reserved=0 I'm glad to help, but I don't know what dataset we're talking about. Could you send me the link please?
And, if possible, create an admin account in your IPT, that would help to diagnose what's going on.
—
Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgbif%2Fipt%2Fissues%2F1925%23issuecomment-1380798105&data=05|01||b6dd76ee85e2410fb6b708daf4c78c64|be0003e8c6b9496883aeb34586974b76|0|0|638091435157135460|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=LqUV07rR%2BjU3BnQw0m29sax%2FrPQr%2F2Q4AZLZI86YJGM%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAH4PLA7PYPMZIIP6KTIHELTWSBBVNANCNFSM6AAAAAATUBUELU&data=05|01||b6dd76ee85e2410fb6b708daf4c78c64|be0003e8c6b9496883aeb34586974b76|0|0|638091435157135460|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=Utb%2BPzTBpFHeMjS8dL11VB51LRBvA0BzPt3GRQyV38M%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

Hi @AMNHcjohnson I will take a look today

@AMNHcjohnson it looks like we are unable to access the archives from your IPT.
This could be due to some firewall settings. It looks like it isn't just this dataset, for example the last time we were able to access the archive from this dataset (https://www.gbif.org/dataset/a8035a1d-e674-4d2a-bb59-b476af6a3d6d) was in July 2021.
You can find more information in our IPT manual here: https://ipt.gbif.org/manual/en/ipt/latest/installation#opening-the-ipt-to-the-internet
I will close this issue as I don't think this is a problem with the IPT software. Please follow up with us at<EMAIL_ADDRESS>thanks!

@AMNHcjohnson One of my colleagues noticed that your IPT is behind Cloudflare, which is blocking machine access from our servers. You will need to configure Cloudflare to permit access to at least GBIF's servers, <IP_ADDRESS>/25.

Thank Marie!!!
I will forward this to our IT department.
Chris
Chris Johnson, Ph.D.https://www.amnh.org/research/staff-directory/christine-johnson | Curatorial Associate | Division of Invertebrate Zoologyhttps://www.amnh.org/research/invertebrate-zoology | American Museum of Natural Historyhttps://www.amnh.org/research/invertebrate-zoology| 200 Central Park West | NYC NY 10024 | Tel<PHONE_NUMBER> |<EMAIL_ADDRESS>Looking to see if IZ has a specimen? Search first on our IZ online databasehttps://emu-prod.amnh.org/imulive/iz.html!
Digitizing Terrestrial Polyneoptera at AMNHhttps://www.amnh.org/research/invertebrate-zoology/collections/other-insect-orders/digitizing-terrestrial-crickets-earwigs-grasshoppers--roaches (NSF-PEN grant)
Digitizing Marine Invertebrates at AMNHhttps://www.amnh.org/research/invertebrate-zoology/collections/other-invertebrate-phyla/databasing-imaging-marine-invertebrates-collections-natural-history (NSF-TCN grant)
Managing Editor, Entomologica Americanahttps://nyentsocjournal.org/
Associate Editor, Journal of Negative Results - EEB
From: Marie Grosjean @.>
Sent: Wednesday, January 18, 2023 8:34 AM
To: gbif/ipt @.>
Cc: Christine Johnson @.>; Mention @.>
Subject: Re: [gbif/ipt] Metadata doesn't save (Issue #1925)
EXTERNAL SENDER
@AMNHcjohnsonhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAMNHcjohnson&data=05|01||7c35f678319d440908c708daf9589b16|be0003e8c6b9496883aeb34586974b76|0|0|638096456213471909|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=urFk9oz5w66SZC2kr%2BAwXXQfzWeO3C0HyQD5j%2F%2FZNXE%3D&reserved=0 One of my colleagues noticed that your IPT is behind Cloudflare, which is blocking machine access from our servers. You will need to configure Cloudflare to permit access to at least GBIF's servers, <IP_ADDRESS>/25.

Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgbif%2Fipt%2Fissues%2F1925%23issuecomment-1387082430&data=05|01||7c35f678319d440908c708daf9589b16|be0003e8c6b9496883aeb34586974b76|0|0|638096456213471909|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=IPjDJDy8gs%2Bdk8xtzFshFcyHDOpFDp9py%2F6YLMWehAM%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAH4PLA7P27HHWIPR6X5PENDWS7WLDANCNFSM6AAAAAATUBUELU&data=05|01||7c35f678319d440908c708daf9589b16|be0003e8c6b9496883aeb34586974b76|0|0|638096456213471909|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=Yu7NmFlkN3eb1lVwBE00%2BRdfGJHu7mdfEC88SLdRw5s%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

Hey All! AMNH IT Here :)
I'll dig into the logs on our side of things, but my guess is that we're blocking it because it is automated/bot traffic. While we most certainly can add the range to our allow list it isn't the preferred solution as it does negate some security controls. We heavily leverage Cloudflare's Bot Management solution to help mitigate aggressive crawlers and data scrapers, unfortunately some legitimate solutions do run afoul of this. Coincidentally July 2021 is when we enabled this service within Cloudflare, so that adds up nicely.
Do the GBIF servers make requests to servers that include a specific user agent (e.g. GBIF Metadata Bot v1.0) instead of a generic user agent (e.g. Curl, Python Requests, etc)? If not, that'd be the first step. And then from there you can request that Cloudflare marks the bot as verified. We're happy to leverage our account and support team at Cloudflare to help assist with this if necessary.
https://developers.cloudflare.com/bots/reference/verified-bots-policy/
You can submit the bot verification on their Google Form:
https://forms.gle/pWVxfCj6cQgWGxDp9
Source documentation for the Google Form link (because why is Cloudflare using Google Forms for this? I'm not entirely sure...)
https://blog.cloudflare.com/friendly-bots/
-Ben

Hi Ben,
The IPT tool provides a managed data repository; the purpose is to allow  programmatic access to the published data, with GBIF as the primary user.
I have completed the form, though I doubt we meet the scale Cloudflare requires. I think there are only 4 IPT installations behind Cloudflare, and yours is the only one with these tightened security settings.  For https://ipt.amnh.org/ we would normally make 8 HTTP requests per week.
Our user agents include COLServer (COLServer/24a3ae9 2022-12-20), org.GBIF.utils/1.16 (Java/11.0.17; M-1800000-25-2; +https://www.gbif.org/), GBIF-Url-Validator and Thumbor/6.7.0.  As far as I know, no-one is currently using user agents to allow/block access to an IPT, so we have not made any particular effort to align or maintain these.  A few publishers do limit access to <IP_ADDRESS>/24.
Other biodiversity systems or researchers also access IPTs using various tools or scripting languages.  In the last week, I can see two researchers/groups have used Python and RStudio to query IPTs at https://cloud.gbif.org/.  Blocking Python, Curl etc will block these users.
Matt


time series line chart has a date shift

I used time series line chart to display the data trending, but the value was shifted by 1 day, e.g. the value on 2016-05-01 was 23801, but actually the value was plotted on 2016-04-30. Will you help to investigate it or give some suggestions. Thanks a lot.
the data exported by caravel:

the plot actually made by caravel:

This looks like a duplicate of #763 , right?

I have the same problem.How can  I make up it with out add "at time zone" to the query every time #763

Closing as duplicate of #763

Seems like this problem still persists. How to disable the UTC conversion?

I had the same problem. Solved it with setting offset value in the datasource settings!


[BUG] Hitting  superfluous response.WriteHeader  during stress test

I tried this code:
Symfony 5 out of the box (then add roadrunner integration) or a simple php router (I can provide access to my repo). Running stress test with autocannon
autocannon -c 100 -d 40 -p 10 -t 120 http://<IP_ADDRESS>:8080
Seems to be in line with issue #301
I expected to see this happen:
no logs
Instead, this happened:
2020/05/28 16:32:33 http: superfluous response.WriteHeader call from github.com/spiral/roadrunner/service/http.(*Handler).handleError (handler.go:135)
2020/05/28 16:32:33 http: superfluous response.WriteHeader call from github.com/spiral/roadrunner/service/http.(*Handler).handleError (handler.go:135)

The version of RR used: 1.8.1

@andrei-dascalu Thanks for the report. Seems, that I need to deeper investigate this bug :)
Linked: #301

Hi,
I've made some other tests using RR and might be useful info:

tried a basic setup with Ubiquity framework (using their integration, though their README is outdated, their CLI tool can do biquity serve -t=roadrunner and will setup some sane defaults). They don't seem to setup a relay via socket and using the same stress test I got a much weaker general result, but no such errors across a dozen or so tests.
Symfony (and my setup) use sockets and got to a much higher request count, always getting some errors)

@andrei-dascalu Thank you for the information. In high overview I know the root cause of such errors, just need to allocate some time to dig into :)


Change default log level to logging.INFO

MOTIVATION:

For the time being, during development, avoid the need to use the --log-level=info CLI arg (or setting the PUP_LOG_LEVEL environment variable).
Should simplify initial testing/exploration by others and gathering their feedback.

THOUGHT:

Maybe later consider changing it back to logging.WARNING.

Closing after having merged #59.


All values or query

Is there a way to include all results from an or query? Something like:
jsonQuery("[*element|sibling.child]", {
  data: {
    "element": "wanted element",
    "exclude": "exclude",
    "sibling": {
      "child": "wanted sibling child",
      "etc": "etc"
    }
 }
})
// => ["wanted element", "wanted sibling child"]

No, I don't think so.
But you could do it with a helper function. Here's one way (with the most minimal helper possible):
var helpers = {
  values: function (input, ...values) {
     return values
   }
}
jsonQuery(":values({element}, {sibling.child})", {
  locals: helpers,
  data: {
    "element": "wanted element",
    "exclude": "exclude",
    "sibling": {
      "child": "wanted sibling child",
      "etc": "etc"
    }
 }
})

All that is happening here is we use {...} sub queries, and then a helper to join them all into a single array.
I do like your syntax proposed above though. It would fit quite nicely with the rest of the module.

ah, that's pretty elegant. I ended up baking my own "querying" solution because of the amount of custom functionality I wanted, but I'll keep this module in mind :-)


Allow passing Saver write_version to 'evaluation_once' and 'evaluatio…

…n_loop'
Thanks @tedhtchang for the commit https://github.com/tensorflow/tensorflow/commit/2a16133061ba3f8fa60c0338cd629f2211f9b17d to add checkpoint file prefix check with default to SaverDef.V2.
However there are some pre-trained slim models which stored in SaverDef.V1 format. This will broken eval_image_classifier.py for these models since there is no way to choose in slim/evaluation.py.
So a parameter write_version is added for evaluation_once and evaluation_loop which allows programs like eval_image_classifier.py could tell which format the checkpoint uses.

Nagging Assignee @sguada: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Thanks for the suggestion, I'll update the branch with evalutation_test.py.

In current master branch, there is no prefix check as described in saver.py, so it's OK now to eval pre-trained slim models.