Implement padding option

Hi,
PR looks good overall. Please make a few things, and I'll merge it:

Move the padding code right before the extending code.
Remove the color argument. I think it's better to use background color as extending does.
Make the other arguments more CSS-like. The first one sets all the dimensions, the second one sets left and right, the third one sets bottom, the 4th one sets left.
Add documentation for the new option.

Move the padding code right before the extending code.

Not quite understand. You mean code in newProcessingOptions and applyPaddingOption or actual embed func call?

Remove the color argument. I think it's better to use background color as extending does.

Was thinking about it while writing code but decided not to clash them at first iteration.

Make the other arguments more CSS-like. The first one sets all the dimensions, the second one sets left and right, the third one sets bottom, the 4th one sets left.

Was thinking about it too :)

Add documentation for the new option.

Will do

Not quite understand. You mean code in newProcessingOptions and applyPaddingOption or actual embed func call?

Sorry for puzzling. I mean put this => here.

Was thinking about it while writing code but decided not to clash them at first iteration.

Let's maintain consistency and use background here :)

Done, but there're a few cases to discuss.

I've added docs, but probably padding could be also mentioned in "About the processing pipeline".

Coz we need to use background option now implicit flatten could be applied. Default bg color is white, so we can get different results for calls without bg at all and with smth like /bg:fff/. Not sure if it's bad. Just to consider.

We need to decide what should happen if padding that is more than image size is set.

I've adding this one (not commited) to get rid of strange image transformations.
	if po.Padding.Enabled {
		dstW -= float64(po.Padding.Left + po.Padding.Right)
		dstH -= float64(po.Padding.Top + po.Padding.Bottom)
		if dstW < 1 {
			dstW = 1
		}
		if dstH < 1 {
			dstH = 1
		}
	}

But still in this case resulting image size grows coz of this part (image is resized to 1x1px + surrounded with passed padding):
		img.Embed(
			img.Width()+po.Padding.Left+po.Padding.Right,
			img.Height()+po.Padding.Top+po.Padding.Bottom,
			po.Padding.Left,
			po.Padding.Top,
			po.Background,
		)

We can make code more complex and prevent image growing or leave it for end users.

Related to previous. Extend goes after padding. Enlarge goes before. Some results can be not obvious. Not sure if we should cover in docs (I doubt that it's normal usage).

I've added docs, but probably padding could be also mentioned in "About the processing pipeline".

"About the processing pipeline" needs updating anyway, I'll do this

Coz we need to use background option now implicit flatten could be applied. Default bg color is white, so we can get different results for calls without bg at all and with smth like /bg:fff/. Not sure if it's bad. Just to consider.

vipsImage.Extend works in a hacky way: if the image has alpha-channel, it extends it with fully transparent color, otherwise, it uses the provided color. In case you don't need to add a border with a color that differs from the background, it's ok.

We need to decide what should happen if padding that is more than image size is set

Good catch. I wanted to point on this myself. I think, increasing srcW and srcH instead of decreasing dstW and dstH should solve the problem. IMHO if the user sets the padding larger than the requested size, they should expect some unexpected behavior :)
The other solution is making padding outbound and moving padding after extending. This would solve 4 too.

I think outbound padding would be a better solution since it solves most of the problems and has a more predictable behavior.

I've tried to move it under extend like that:
if po.Extend.Enabled && (po.Width > img.Width() || po.Height > img.Height()) {
	offX, offY := calcPosition(po.Width, po.Height, img.Width(), img.Height(), &po.Extend.Gravity, false)
	if err = img.Embed(po.Width, po.Height, offX, offY, po.Background); err != nil {
		return err
	}
}

if po.Padding.Enabled {
	if err = img.Embed(
		img.Width()+po.Padding.Left+po.Padding.Right,
		img.Height()+po.Padding.Top+po.Padding.Bottom,
		po.Padding.Left,
		po.Padding.Top,
		po.Background,
	); err != nil {
		return err
	}
}

checkTimeout(ctx)

if po.Watermark.Enabled && watermark != nil {
	if err = applyWatermark(img, watermark, &po.Watermark, 1); err != nil {
		return err
	}
}

return img.RgbColourspace()

...and it works well. Also this allowed to remove padding calc in calcScale.
But now we have 2 new questions:

Do we need to apply dpr for padding?

May padding embed should go after watermark too?

Do we need to apply dpr for padding?

Damn, that's a good question. Seems like we should.

May padding embed should go after watermark too?

I don't think so:

extend is applied before watermark, let's maintain consistency.
There is an optimization for applying watermarks on animated images that wouldn't work if watermarks are applied not at the last step.

To summarize:

We leave padding code exactly like in snippet above.
Do apply dpr for padding (in simple terms enlarge it).
Mention in docs that padding is applied after all operation except watermark and encounters dpr.

Yup, that's right. Sorry for the delay

Now I'm sorry for the delay :) All that stuff happening made my working computer unavailable for some time.
I've updated PR, seems like now it's all like we discussed.

Looks good! Thanks for your contribution!


KafkaProducer with sendAsync method

Motivation
Enable users to send messages in a non-blocking way with an optional callback.
Modifications

specified minimum OS versions in Package.swift to support new Swift
concurrency
made it possible to test the KafkaProducer inside of Docker
KafkaProducerMessage
KafkaConsumerMessage
KafkaProducer with sendAsync method
initialization of KafkaError from rd_kafka_resp_err_t

Hey @Lukasa,
First of all, thanks a lot for your review! I have applied your changes and replied to your conversations.
Please let me know what you think and if there is something I misunderstood.

Hey @Lukasa ,
thanks again for re-reviewing my code! I hope that this new implementation is what you have thought of in the first place!
I have also answered all your comments in this PR. Thanks a lot!

Hey @Lukasa, Hey @FranzBusch,
I have implemented additional changes that try to obscure the entire Unmanaged logic inside the KafkaConfig.
Please take a look at the individual commits:
9ce49e2 is an implementation I worked out together with @FranzBusch and your ideas, @Lukasa
834f243 is an experimental simplification of what we worked out yesterday
Please let me know what you think!


fix fix resource is not reopen when rename catalog

Proposed changes
Issue Number: close #xxx
pick  (#33432)
During the renaming of JdbcCatalog, I noticed that the jdbcClient was being closed, resulting in exceptions during subsequent queries. This happens because the removeCatalog method is invoked when changing the name, which in turn calls the onClose method of the catalog. Ideally, the client should not be closed when renaming the catalog. However, to avoid extra checks in the removeCatalog method, we can simply execute onRefresh in the addCatalog method to address this issue.
Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run feut


47 garden ai pipeline create

closes #47

:books: Documentation preview :books:: https://garden-ai--66.org.readthedocs.build/en/66/

Codecov Report

Merging #66 (3bfbb8f) into main (40599e4) will decrease coverage by 0.88%.
The diff coverage is 83.09%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             main      #66      +/-   ##
==========================================
- Coverage   88.49%   87.62%   -0.88%
==========================================
  Files          14       15       +1
  Lines         965     1139     +174
==========================================
+ Hits          854      998     +144
- Misses        111      141      +30

Impacted Files
Coverage Δ

tests/conftest.py
100.00% <ø> (ø)

garden_ai/steps.py
94.11% <40.00%> (-4.32%)
:arrow_down:

garden_ai/client.py
70.41% <72.34%> (-1.13%)
:arrow_down:

garden_ai/app/pipeline.py
76.05% <76.05%> (ø)

garden_ai/app/garden.py
58.62% <77.77%> (ø)

garden_ai/gardens.py
92.53% <88.88%> (+0.47%)
:arrow_up:

garden_ai/app/main.py
100.00% <100.00%> (ø)

garden_ai/pipelines.py
91.66% <100.00%> (+2.01%)
:arrow_up:

garden_ai/utils.py
77.55% <100.00%> (+4.82%)
:arrow_up:

tests/test_cli.py
100.00% <100.00%> (ø)

... and 3 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


ibm-openbmc: move CI scripts over to ibm-openbmc repos

This is to get CI working with ibm-openbmc forked repos
Signed-off-by: Andrew Geissler<EMAIL_ADDRESS>

Verified that this fixed the issue with PLDM CI and it's dependency on ibm-openbmc/PDI


[Feature] Add base parameters to context

When creating a trait, it seems we currently need to define base parameters again, because they don't seem to be in the context. Take this example
name: my-app
services:
  app:
    type: webservice
    image: crccheck/hello-world
    port: 8000
    myTrait:
      port: 8000

In myTrait we need to redefine port, in order to use it in the trait to create a service resource. Please correct me if I'm wrong and it is to be found somewhere.
It would be good, if those base parameters would be available. Maybe in context.port?

@joscha-alisch Hey! The current design is we recommend to package Service and Deployment in the same template module of a WorkloadDefinition, i.e. a Web Service = Deployment + Service.
The related PR is WIP here: https://github.com/oam-dev/kubevela/pull/937/files#diff-9afacce3fa2a274c3add4aebf38abfdc7cd7fc8d25d2e63ab8a6fcf7601f70e8

@joscha-alisch Hey! The current design is we recommend to package Service and Deployment in the same template module of a WorkloadDefinition, i.e. a Web Service = Deployment + Service.
The related PR is WIP here: https://github.com/oam-dev/kubevela/pull/937/files#diff-9afacce3fa2a274c3add4aebf38abfdc7cd7fc8d25d2e63ab8a6fcf7601f70e8

Ah thanks for that @resouer , that looks like the better approach :+1:

Ah thanks for that @resouer , that looks like the better approach :+1:


Closing this issue.


cant click on categories

just picks up head when clicking on the categories

please provide the following information:

version of your server
version of DecoHeads
full console log


gobrew use mod

Not exactly fixing #99 but a step in this direction
Usage
gobrew use mod // reads version from go.mod file and use the latest version of that

Codecov Report

Merging #101 (c2386f7) into master (f39ad94) will decrease coverage by 2.27%.
The diff coverage is 0.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##           master     #101      +/-   ##
==========================================
- Coverage   52.55%   50.29%   -2.27%
==========================================
  Files           1        1
  Lines         489      511      +22
==========================================
  Hits          257      257
- Misses        195      216      +21
- Partials       37       38       +1

Impacted Files
Coverage Δ

gobrew.go
50.29% <0.00%> (-2.27%)
:arrow_down:


pull request for e2e test at 01/16/2017 14:26:15

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


feat: panel daynames component

Please check if the PR fulfills these requirements

[x] It's submitted to right branch according to our branching model
[x] It's right issue type on title
[ ] When resolving a specific issue, it's referenced in the PR's title (e.g. fix #xxx[,#xxx], where "xxx" is the issue number)
[x] The commit message follows our guidelines
[ ] Tests for the changes have been added (for bug fixes/features)
[ ] Docs have been added/updated (for bug fixes/features)
[ ] It does not introduce a breaking change or has description for the breaking change

Description

add panel daynames component & story

Thank you for your contribution to TOAST UI product. 🎉 😘 ✨

Can one of the admins verify this patch?

아마 PR올릴때마다 이게 계속 나올텐데 jenkins 설정에서 관리자로 등록하거나 해당 브랜치를 젠킨스에서 제거하는게 좋지 않을까요?


Implement local and remote super-linter for linting project

Description
We need to ensure that we're linting locally (pre-commit checks) and linting in our GitHub Actions pipeline (which we do today).
Lets try out super-linter
docker run --platform linux/x86_64
-e ACTIONS_RUNNER_DEBUG=true
-e VALIDATE_ALL_CODEBASE=false
-e VALIDATE_ANSIBLE=false
-e DEFAULT_BRANCH=main
-e RUN_LOCAL=true
-v ${PWD}:/tmp/lint
ghcr.io/super-linter/super-linter:latest
For GitHub, use the github actions described here
---
name: Lint

on:  # yamllint disable-line rule:truthy
  push: null
  pull_request: null

jobs:
  build:
    name: Lint
    runs-on: ubuntu-latest

    permissions:
      contents: read
      packages: read
      # To report GitHub Actions status checks
      statuses: write

    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          # super-linter needs the full git history to get the
          # list of files that changed across commits
          fetch-depth: 0

      - name: Super-linter
        uses<EMAIL_ADDRESS># x-release-please-version
        env:
          DEFAULT_BRANCH: main
          # To report GitHub Actions status checks
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Is your feature request related to a problem? Please describe.
Need linting locally before a push
Describe the solution you'd like
Use super-linter (or pep8 or similar) with a pre-commit check
Describe alternatives you've considered
Not doing it but that means we still are pushing with bad lints and have to wait for the pipeline to tell us we done bad.

https://github.com/Raptacon/Robot-2024/pull/38


(Performance) issue with github_outside_collaborators table

Describe the bug
This is a follow-up of https://github.com/turbot/steampipe-plugin-github/issues/404 as discussed with @ParthaI. In addition to the comments in that issue, I've provided more context and code below.
Additional context

Are you aware of the rough number of collaborators within your organization?

The organization where this issue appears, has ~550 collaborators (both users and outside collaborators).

Could you describe how the query SELECT * FROM github_organization_collaborator WHERE organization = '...' and affiliation = 'OUTSIDE'  behaves, both with and without the limit clause?

Actually, this specific query works in all cases; with and without LIMIT. I took the opportunity to run a COUNT to give an idea of the size:
> SELECT COUNT(*) FROM github_organization_collaborator WHERE organization = '...' AND affiliation = 'OUTSIDE';
+-------+
| count |
+-------+
| 635   |
+-------+

I notice that in the case with this query, progress is shown:

Without the additional AND affiliation = 'OUTSIDE', the behavior as described in https://github.com/turbot/steampipe-plugin-github/issues/404 is observed.

Hello @olafz, after further investigation into the problem, I find myself unable to reproduce the error when working with a smaller dataset. It appears that reaching the secondary rate limit is less likely with a limited number of results. To accurately reproduce the issue, accessing real-time data is essential, as recursive API calls have proven to be of limited assistance.
From what I've observed:

The error stems from reaching the secondary rate limit, not the primary one.
On the server side, GraphQL enforces a 10-second execution time limit, which we can't alter.
Queries exceeding this timeframe result in a 502 response.
There's also a restriction where individual calls can't request over 500,000 total nodes.

By the way, I've made an update to reduce the query's page size in the issue-413 branch. Could you test it out in the PR branch and share if it makes any difference?
Thank you!

By the way, I've made an update to reduce the query's page size in the issue-413 branch. Could you test it out in the PR branch and share if it makes any difference?

It does! The query
> SELECT * FROM github_organization_collaborator WHERE organization = '...' and affiliation = 'OUTSIDE';

Now runs fine without LIMIT clause 🎉


Added Online Playground for C and Python

What does this PR do?
Add resource(s)
For resources
Description
Add Online C Playground
This commit introduces an Online Python Playground and C Playground that allows users to write, edit, and run code directly in their browsers. This feature supports real-time coding and debugging, enhancing learning and development flexibility.
Why is this valuable (or not)?
This allows users to write, edit, and run code directly in their browsers without the need of downloading or installation of other software. This saves time and helps you focus on learning.
How do we know it's really free?
Anybody can access it. It is an absolutely free resource.
For book lists, is it a book? For course lists, is it a course? etc.
Checklist:

[x] Read our contributing guidelines.
[x] Search for duplicates.
[x] Include author(s) and platform where appropriate.
[x] Put lists in alphabetical order, correct spacing.
[x] Add needed indications (PDF, access notes, under construction).
[x] Used an informative name for this pull request.

Follow-up

Check the status of GitHub Actions and resolve any reported warnings!

please refrain from duplicate PRs


Please clarify which policies/settings can block or impact the Edge WebDriver

We are facing a frustrating issue, with a customer in a secure environment who is unable to use our product which uses the Edge WebDriver, because for some reason the WebDriver cannot communicate with Edge. So far, we found these 3 settings which could impact or block the WebDriver from functioning. The frustrating part is how hard it is to find this information, we found some on random forums, and after a lot of searching and googling on this site.

Application Guard with untrusted sites
The DeveloperToolsAvailability policy being set to 2
The RemoteDebuggingAllowed policy being disabled

We don't know if there are any others that could prevent the WebDriver from functioning properly.
Could you please clarify this information in the documentation? Thank you.

@chibicitiberiu Hello - Contact information for the WebDriver team and link to their GitHub can be found here: https://docs.microsoft.com/en-us/microsoft-edge/webdriver-chromium/contact
Please try and contact them directly.  Thanks!


fix: replaced getImageProperties with dart buffer and descriptor

What this PR does:

Adds missing alert config in test/live_activities_test.dart
fixes https://github.com/istornz/flutter_live_activities/issues/46 by replacing getImageProperties with dart buffer and descriptor

Hi @aumb,
Thanks for your PR. Can you resolve conflicts in order to merge it?
Thanks :)

Hey @istornz conflicts have been resolved!


Price rounding for "cost depends on usage" is sometimes misleading

In the following example, the price is $0.000025/transition, or $0.25 per 1K transitions. The costs are shown correctly but the "cost depends on usage" line is rounding to 2 decimal places when unitMultiplier is set to larger numbers. Should we round to 4 decimal places to handle this case?
INFRACOST_LOG_LEVEL=info make -s run ARGS="breakdown --path internal/providers/terraform/aws/testdata/step_function_test --usage-file internal/providers/terraform/aws/testdata/step_function_test/step_function_test.usage.yml"

Project: internal/providers/terraform/aws/testdata/step_function_test

 Name                                              Monthly Qty  Unit                    Monthly Cost

 aws_sfn_state_machine.standard
 └─ Transitions                                             10  1K transitions                 $0.25

 aws_sfn_state_machine.standardWithoutUsage
 └─ Transitions                              Monthly cost depends on usage: $0.03 per 1K transitions

 PROJECT TOTAL                                                                                $0.25

Currently we round to 2dp if price < 0.01. We could change that to 0.10 maybe?
Code is here: https://github.com/infracost/infracost/blob/master/internal/output/format.go#L45

@alikhajeh1 hey, I want to work on it!


Added the schema doc

Description of what I changed

This is the first step for addressing #1025.
E2E test

TESTED:
NA
Checklist: I completed these to help reviewers :)

[x] I have read and will follow the review process.

[x] I am familiar with Google Style Guides for the language I have coded in.
No? Please take some time and review Java and Python style guides.

[x] My IDE is configured to follow the Google code styles.
No? Unsure? -> configure your IDE.

[ ] I have added tests to cover my changes. (If you refactored existing code that was well tested you do not have to add tests)

[x] I ran mvn clean package right before creating this pull request and added all formatting changes to my commit.

[x] All new and existing tests passed.

[x] My pull request is based on the latest changes of the master branch.
No? Unsure? -> execute command git pull --rebase upstream master

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 50.49%. Comparing base (9bfd5d9) to head (049cb5a).

Additional details and impacted files
@@            Coverage Diff            @@
##             master    #1087   +/-   ##
=========================================
  Coverage     50.49%   50.49%
  Complexity      674      674
=========================================
  Files            91       91
  Lines          5519     5519
  Branches        709      709
=========================================
  Hits           2787     2787
  Misses         2470     2470
  Partials        262      262

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


too slow in startup

This starts considerably more slowly than Neovim(qt). I have been using this for months, yet I have decided to switch back to neovim for now.
BTW, I use AutoHotkey to start this. It does not show in FG, but starts as blocked by other FG windows. Neovim works fine. Windows 10.

hi
Thanks for the issue report.
Yes, I am aware of this problem as I also use Windows at work, and the improvement in startup speed is a TODO.
I would like to check, how long are the startup speeds of neovim-qt and goneovim respectively? (even if you can feel it)

I just recently started experimenting with goneovim and nvim-qt seems to start 5 or 6 times faster -- especially when starting fullscreen. I routinely use a dark theme for vim and neovim so the other really annoying thing about goneovim startup is the white canvas being displayed prior to the theme being applied. It's very jarring...

@gwojan
Thanks for your opinion.
What your OS?

I routinely use a dark theme for vim and neovim so the other really annoying thing about goneovim startup is the white canvas being displayed prior to the theme being applied. It's very jarring...

Hmmm, I know. Actually, in the previous version of goneovim, the specification was that the window would not be displayed until the entire application color was determined. I changed this specification to show the window first to make it look more sensible and quicker to launch, since the color information is provided by neovim and it takes a lot of time to apply the color to the entire application.
This is entirely a matter of taste, but I'm going to reconsider this specification for a moment.

@akiyosi I'm currently using Microsoft Windows 10 Enterprise, Version 1909 (OS Build 18363.1198). I apologize, I should have included that in the first place.
I also use the nightly Windows builds of Neovim 0.5 if that makes any difference at all.

@bon-ami
hi
Sorry for late reply.
I have been working on solving the boot time issue in Windows. If you have time, I would appreciate it if you could check if the problem has improved somewhat. You can get the binary below.
https://github.com/akiyosi/goneovim/suites/1733424569/artifacts/32846679

Thank you Aki Yosi. It runs fine to me.

Thank you Aki Yosi. It runs fine to me.

Closed.


Not Working in Oppo Devices, when force quit the application

Using 1.3.0-alpha-3.

https://github.com/evernote/android-job/wiki/FAQ#what-happens-with-jobs-after-the-app-was-forced-killed


Universal PRON tag.

Hiya,
I was just wondering if there was any reason you deviated from the Universal Tagset by changing PRP and other pronoun tags to NOUN rather than PRON.
In fact I noticed some other changes that are different from Petrov et al.'s original proposal in lang_data/en/tag_map.json too.
Is this intended?
Thanks,
Chris

Hey,
I first used the Google Universal Tags, and then updated to the Stanford Universal ones. I guess this slipped through, I didn't notice this tag was written differently. I'll fix it, thanks.

Ok np.
I'd not looked at the Stanford one before, but read it's no longer just a 1:1 mapping from old to new POS tags, but rather relies on additional syntactic context: (https://mailman.stanford.edu/pipermail/java-nlp-user/2014-September/006335.html).
Currently I am just using your POS-tagger, but does this mean I need to parse too to get the syntactic information for the more accurate POS tags?
I see you also put some of the Universal Features in the tag_map.json too. Is there any way to access these for a particular token?
Thanks again and keep up the good work!
Chris

Bear in mind that you can always use the PTB tags, via the token.tag and token.tag_ attribute.
The lack of 1-to-1 mapping is sort of troubling =/. The thing is, the tagger should really be trained to predict fine-grained, language-specific tags. They can be leaned more accurately and they make better features for downstream tasks. I like the idea of supporting the coarse-grained representation. But I don't want to learn a model just to do that.

Yup, I have been using the PTB tags as well; e.g. to catch pronouns.
I also like the idea of the coarse-grained tags (and in fact that was one of the reasons I chose spaCy!) but it does seem like a lot of extra effort to get the Stanford Universal tags. Unfortunately, I'm now noticing quite a lot of differences between your current Stanford mapping file and the one on their website (https://universaldependencies.github.io/docs/tagset-conversion/en-penn-uposf.html) which it also warns is "only an approximation".
Tbh, I'd probably just prefer the simpler 1:1 Google Universal tags, even if they aren't quite as good as the Stanford ones.
Chris

Would prefer the Stanford ones. They're regularly maintained and updated.

So would I if they were easy to implement.
As it is however, there are problems with the v0.97 Stanford Universal Tags which might not be easy to fix. So we have a choice between the older but more consistent Google tags or the newer but more complicated Stanford tags.
Ultimately, I don't know how much difference it'll make, but missing a universal PRON tag in the current Stanford version (amongst other things) seemed like a big deal...

It seems PRON tag exists in v1.2? I think most people just use PTB but if one crosses over to multi-lingual parsing I personally like much the fact that UD is regularly released though the data is not much yet.

On (I think) a related note, would you consider also adding the PROPN (proper noun) universal tag? I know that corresponds to NNP or NNPS in PTB, but for other languages and general convenience, it would be great to have in the .pos and .pos_ attributes.

Finally fixed this. Sorry about letting this slip through the cracks for so long!

Sorry to rain on your parade, but the tag_map.json still doesn't look like the one on the Stanford website, although you fixed a couple of them:
http://universaldependencies.org/tagset-conversion/en-penn-uposf.html

You have an extra pos "", maybe by design
"""": {"pos": "punct", "puncttype": "quot", "punctside": "fin"},
NIL doesn't go to anything in spacy
"NIL": {"pos": ""},
It goes to X in Stanford.
PDT goes to ADJ in spacy, but DET in Stanford.
PRP$ goes to ADJ in spacy, but DET in Stanford.
WDT goes to ADJ in spacy, but DET in Stanford.
WP goes to NOUN in spacy, but PRON in Stanford.
WP$ goes to ADJ in spacy, but DET in Stanford.
AFX is mentioned twice in spacy - once going to ADJ, then going to X
HYPH is mentioned twice in spacy - both going to PUNCT, but the first has extra detail.
There are several other tags on the end of the file not in the Stanford list; SP, ADD, NFP, GW, XX, BES and HVS.

If you make these changes, I'll definitely start using the coarse tags in spacy, but until then, I'm still using my own mapping.
Also, when you change this file, are they immediately updated in spacy? I know I'll probably have to do more general update, but will that be immediate or do I have to wait for the next release?

Thanks. I keep getting confused here, because the names of these things are very similar and the contents overlap so much. I also keep sliding between talking about what spaCy is doing currently, and discussing what we should be doing.
Let's set focus on what we should do. Here are some things I think we want:

We want to learn fine-grained tags in the statistical model
We want coarse grained tags available in the API
We want the coarse tags to follow a standard. We don't want half a standard.

Here are some smaller considerations:

We'd like to produce the coarse tags from a deterministic mapping (not a model)
The Stanford set has better adoption by others.
The Stanford tags are a bit more complete/useful

As much as I'd like to support the Stanford scheme, I think it's a higher priority to have a simple mapping. So I propose that the policy is we're using the Google scheme. At some point in future I'd like to switch to full universal dependencies support, but for now I think it's better to be clear about the tag mapping.
If we follow this policy, we'll revert the change I just made to the tag map, and avoid using the PROPN tag. It's easy to fix the lemmatizer in another way.
Thoughts? @wbwseeker , I know you're advocating for eventual adoption of UD. What are your thoughts on how we should do the coarse grained mapping at the moment? Is the Google Universal Tag Set okay for German?

Why don't we just adopt the tag set defined by UD?: http://universaldependencies.org/u/pos/index.html
It is based on the Google Tag Set but was improved and people work on it. I don't understand why the Google Tags are supposed to be simpler than UPOS. It is just less of them, and noone actively works on it anymore because it was superseded by UPOS.
Looking at http://universaldependencies.org/tagset-conversion/en-penn-uposf.html, it just defines a mapping from Penn Treebank Tags to UPOS. I would advocate for using this mapping. (I can understand if we want to wait to adopt the syntactic annotation, because this is still changing. But this isn't the case for the POS tags)
For German, I already use UPOS in the tagmap.json.
We will have to add some tags to deal with stuff that UD doesn't annotate. Like space tokens. But as long as we just add and don't change, this won't matter.

I think the issue is that the real UD mapping is not 1:1 and requires additional information, e.g. lemmas.
That said, I do prefer some of the mappings in the approximated UPOS scheme over the Google one, so am now leaning more towards that. Also, I wouldn't think of the approximation as a half standard, but rather a simplified version of a full standard. By the very fact that it's available, there must be some demand for it, right?

Okay. UD it is. We'll use the suggested mapping.
@wbwseeker , would you be able to take care of this? I think we need the following:

Fix the English tag_map.json
Fix the data in spacy/attrs.pyx, spacy/attrs.pxd, spacy/symbols.pyx, spacy/symbols.pxd
Fix the docs for Token.pos
Fix the explanation in the annotation specs

sure, no problem.

Am 17.04.2016 um 15:46 schrieb Matthew Honnibal<EMAIL_ADDRESS>Okay. UD it is. We'll use the suggested mapping.
@wbwseeker https://github.com/wbwseeker , would you be able to take care of this? I think we need the following:
Fix the English tag_map.json
Fix the data in spacy/attrs.pyx, spacy/attrs.pxd, spacy/symbols.pyx, spacy/symbols.pxd
Fix the docs for Token.pos
Fix the explanation in the annotation specs
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub https://github.com/spacy-io/spaCy/issues/191#issuecomment-211020591


Extract core client functionality

The core client functionality has been extracted to a new repository at https://github.com/rneatherway/slackclient. This comprises two pieces of functionality:

GetAuth for getting the Slack token and cookies if you don't want to use the built-in client. I don't know that anyone wants to do this yet, but it seems like a reasonable thing to want to do.
New for creating a new client populated with those authentication details and an API method for calling the Slack API.

Other thoughts:

Somewhat ugly is that we now have two slackclient packages and the SlackClient in this repository just wraps one from the other repository.
I was thinking that perhaps the other repository should just have a slack package with slack.GetAuth(...) and slack.NewClient(...). That reads more cleanly to me.
I do think API should probably take a context for future-proofing.
In one or both repositories I would like to provide a way of short-cutting the cookie (and possibly token) fetching so that

Happy new year 👋

GetAuth for getting the Slack token and cookies [...] it seems like a reasonable thing to want to do.

Agreed 👍

slackclient.SlackClient is a bit ugly I agree, slack.Client, slack.NewClient, and slack.GetAuth would read better.

I do think API should probably take a context for future-proofing.

Agreed 👍

In one or both repositories I would like to provide a way of short-cutting the cookie (and possibly token)

I like this idea as well.

Auth should probably be updated to support as well the "usual" slack tokens as described in https://api.slack.com/authentication/token-types
How about something like:
client := slack.NewClient("my-team")

// this
if err := client.AuthWithCookies(); err != null {
  return err
}

// or that
if err := client.AuthWithToken("the-token"); err != null {
  return err
}

This would internally call GetCookieAuth (exported method) or GetTokeAuth that are available for direct use if needed.

slackclient.SlackClient is a bit ugly I agree, slack.Client, slack.NewClient, and slack.GetAuth would read better.

Done

I do think API should probably take a context for future-proofing.
Agreed 👍

Done

In one or both repositories I would like to provide a way of short-cutting the cookie (and possibly token)
I like this idea as well.

I haven't done this yet, I think it can wait for a follow-up. The motivation is that it's annoying to iterate on MacOS because you are prompted for every new binary when it accesses the keychain.

Auth should probably be updated to support as well the "usual" slack tokens as described in https://api.slack.com/authentication/token-types
How about something like:
...

This is an interesting idea. It might actually allow removing Null because half the reason that method exists is to not call GetAuth(...). An addition of a WithHTTPClient(...) method would then allow providing the stub RoundTripper.
I'm a bit unsure what your proposed GetTokenAuth(...) method would look like though. Would it still have to get the cookie and then just use the token given to it? For all token types?
func GetTokenAuth(token string) (*Auth, error) {
	cookie, err := config.GetCookie()
	if err != nil {
		return nil, err
	}

	return &Auth{Token: string(matches[1]), Cookies: map[string]string{"d": cookie}}, nil
}

The motivation is that it's annoying to iterate on MacOS because you are prompted for every new binary when it accesses the keychain.

Yeah, I felt that when doing testing as well.

I'm a bit unsure what your proposed GetTokenAuth(...) method would look like though. Would it still have to get the cookie and then just use the token given to it? For all token types?

A picture PR is worth a thousand words: https://github.com/rneatherway/slackclient/pull/1
It'll need some more work to make sure passing a token directly actually works, but my past experience tells me it would.

An addition of a WithHTTPClient(...) method would then allow providing the stub RoundTripper.

👍
PR'ed in https://github.com/rneatherway/slackclient/pull/2

OK, I've made the rest of those changes across the two repositories and done a little tidying. The only last thing I think I want to do here is to rename rneatherway/slackclient to just rneatherway/slack to avoid the awkward package renaming.
Any other thoughts before we tidy this up?

Sounds good! Nothing else worthy to add here :)


(LaTeX) Matching TeX magic comments should be case insensitive

Describe the issue
TeX magic comments are, in most editors, case insensitive, so both lines below are interpreted:
% !TeX program = lualatex
% !TEX encoding = UTF-8
%   ^ note e vs. E here

but the MAGIC_COMMENT rule only highlights the TeX (first) line:
https://github.com/highlightjs/highlight.js/blob/2dd87a6ef58c8c886dd209076aa6661ffae89577/src/languages/latex.js#L106-L111

Which language seems to have the issue?
latex
Are you using highlight or highlightAuto?
I don't know? highlight I think because the language is specified explicitly.
Sample Code to Reproduce
% !TeX program = lualatex
% !TEX encoding = UTF-8
%   ^ note e vs. E here

Expected behavior
In the sample above, lines 1 and 2 should be highlighted as MAGIC_COMMENT, and line 3 should be a regular COMMENT:

TeX magic comments are, in most editors, case insensitive, s

That could well be a bug with the editor or it's highlighting... I presume you also mean that LaTex itself draws no distinction, am I correct?  Or are we speaking about multiple flavors of LaTex here with different behaviors?
I don't think I see any real issue with this though.  Thoughts? CC @schtandard

That could well be a bug with the editor or it's highlighting... I presume you also mean that LaTex itself draws no distinction, am I correct? Or are we speaking about multiple flavors of LaTex here with different behaviors?

No, as far as LaTeX is concerned magic comments are just comments: completely ignored.  The magic is a feature implemented in some LaTeX-aware IDEs so that you can easily change command-line options from within the source (for example, with standard settings, an editor would run pdflatex <some file>.tex, but with the magic comment % !TeX program = lualatex it would run lualatex <some file>.tex instead).  Then, while searching for the pattern, most IDEs use a case-insensitive search, so changing the syntax highlighting to reflect that seems appropriate.

I don't see a reason why we couldn't accept a PR to fix this.

@joshgoebel I'd write the PR myself, but I'm afraid my javascript knowledge is roughly zero: I just know enough LaTeX to come here and complain :)
This is really of minor importance, so I'm fine with whenever someone knowledgeable has some free time to dedicate to this

I fully agree that this should be changed and I can take care of this. The space in front of the ! also seems to be optional, at least in TeXStudio. @PhelypeOleinik For TeXStudio, only the e seems to be case insensitive. Do you know if this is universal or should we just make the whole word case insensitive in case other editors are more lenient?

@schtandard I did some digging, and the results vary wildly :)

TeXShop seems to be case-sensitive: it uses rangeOfString:@"%!TEX TS-program =", which as far as I could guess, uses the case-sensitive function rangeOfString (with no options to make it case-insensitive). Also no spaces around the !;
TeXStudio, as you said, allows e and E: rxMagicTexComment("^%\\ ?!T[eE]X");, with an optional space before the !. It also recognizes a BIB magic comment: rxMagicBibComment("^%\\ ?!BIB");;
TeXWorks is case-insensitive with as many spaces as you like almost everywhere: RegExp("% *!TEX +spellcheck *=", "i");
LaTeXTools (Sublime Text) is case insensitive for the TEX string, plus it allows things like %%%   !tEx   <thing>   =   . Pretty wild: r'%+\s*!(?:T|t)(?:E|e)(?:X|x)\s+([\w-]+)\s*=\s*' +;
LaTeX-Workshop (VSCode) allows e and E as TeXStudio, and spaces wherever you fancy: regexTex = /^(?:%\s*!\s*T[Ee]X\s(?:TS-)?program\s*=\s*([^\s]*)$)/m. It also recognizes BIB comments (case insensitive);
Atom-LaTeX allows e and E and spaces mostly everywhere: RegExp('^%\\s*!T[Ee]X\\s+(\\w+)\\s*=\\s*(.*)$');

I couldn't find anything about VimTeX or Overleaf, but this post says they support magic comments.
AUCTeX uses its own thing at the end of the file; not sure if worth supporting, mostly because they aren't as “magic looking”:
% Local Variables:
% mode: doctex
% TeX-master: t
% End:

Finally, web2c implementations also allow a comment on the first line of a file of the format %&<format> which, in this case, is parsed by the TeX engine (sort of). But I don't think this is used enough to be worth highlighting. For example, you can run this example with pdftex and it will select the right format:
%&latex
\documentclass{article}
\begin{document}
\end{document}

TL;DR
Thinking a bit more about it, I don't think it's a good thing to highlight all the possible combinations (like %%%   !tEx   <thing>   =   ). I think I'd take a good enough subset, maybe (in PCRE): ^% ?! ?T[eE]X ([\w-]+) ?= ?.+$, because mostly only TeX and TEX are used, and spaces are optional (because I'd be the first to forget where they are required/forbidden :). Since this seems to require a begin and end, then begin: '^% ?! ?T[eE]X ' and end: $. What do you think?

@PhelypeOleinik Thanks a lot for your research! I mostly agree with your conclusions, except for two minor points. I don't feel strongly about either of them, though.

I would omit the optional space after the !. I have never seen this and according to your list only LaTeX-Workshop supports it. I guess the idea is to allow %! TeX and I sympathise with your point about remembering where spaces are allowed, but this feels like too much of a niche case to me.
I agree that case mixing like tEx is a bit absurd, but should we maybe allow tex and bib? This seems a reasonable spelling to me (for the editors that support them) and is also in line with the "just use lowercase" default many technical users may be used to.

About the ^: At least TeXStudio accepts magic comments anywhere in the document with no restriction of being the only thing on that line. (I didn't test any other editors.) While I myself only ever use them on the first lines of a document (with no indentation), I wouldn't make that a requirement for highlighting.
With all this my suggested regex would be % ?!(?:T[eE]X|tex|BIB|bib). Any comments? (No pun intended.)

* I would omit the optional space after the `!`. I have never seen this and according to your list only LaTeX-Workshop supports it. I guess the idea is to allow `%! TeX` and I sympathise with your point about remembering where spaces are allowed, but this feels like too much of a niche case to me.

Sounds reasonable. Given the plethora of different implementations, some level of restriction is good, so I'm fine with no space after !.

* I agree that case mixing like `tEx` is a bit absurd, but should we maybe allow `tex` and `bib`? This seems a reasonable spelling to me (for the editors that support them) and is also in line with the "just use lowercase" default many technical users may be used to.

Sounds good too.

With all this my suggested regex would be % ?!(?:T[eE]X|tex|BIB|bib). Any comments? (No pun intended.)

Go for it! (the regex, not the pun :)


chore(docs): document es update behavior

closes #212
closes #129
I went through the docs and added a few minor things that were missing. Also added docs for the ClusterSecretStore, tho there isn't much to be documented i think.

/ok-to-test sha=fb726df

I liked having a simple example of how to get the operator running with the most popular provider on the readme :thinking:
But I also see the value of keeping the readme cleaner if you prefer :)

/approve

yeah, the idea was to have a clean readme that points to the relevant bits of documentation

/merge


Fixed broken link and closing tag

Description
Fixed broken link to Client transaction section and closed tag appropriately
Checklist
Please make sure these boxes are checked before submitting your pull request - thanks!

[x] Commit message starts with the issue number from https://issues.apache.org/jira/projects/FINERACT/. Ex: FINERACT-646 Pockets API.

[x] Coding conventions at https://cwiki.apache.org/confluence/display/FINERACT/Coding+Conventions have been followed.

[x] API documentation at https://github.com/apache/fineract/blob/develop/api-docs/apiLive.htm has been updated with details of any API changes.

[ ] Integration tests have been created/updated for verifying the changes made.

[ ] All Integrations tests are passing with the new commits.

[ ] Submission is not a "code dump".  (Large changes can be made "in repository" via a branch.  Ask on the list.)

Our guidelines for code reviews is at https://cwiki.apache.org/confluence/display/FINERACT/Code+Review+Guide

@vishwasbabu, @Anh3h  please take a look when you got time. I found this trailing issue while exploring the docs here - https://issues.apache.org/jira/browse/FINERACT-738


Latest version broke api.branch.io CSP

The latest version pushed today broke any CSP that relied on api.branch.io, since the API endpoint was changed to api2.branch.io.

Please provide a way to access older versions of the SDK. Right now, the CDN only offers latest.min.js. This is terrible for reliability, since a new version pushed today broke us
Please announce these breaking changes

Thanks

@yonilerner we're sorry for breaking your CSP, and for failing to announce this change in advance.
You can access the latest version of our SDK still using api.branch.io here: https://cdn.branch.io/branch-2.47.1.min.js
We're adding instructions for using specific versions of the SDK to our documentation, and we'll work hard to make sure this sort of thing doesn't happen again!


LESQ-886: New content banner

Description
Music year: 1951

Creates new content banner in OWA using components from component lib
Adds content banner on unit listing, lesson listing and lesson overview pages
Links to correct programme or unit listing page as prescribed in this doc: https://docs.google.com/spreadsheets/d/1pE0VtBgSsokQbCrhIhNJrhHfZrnS0U7xy57zsqexmmw/edit#gid=0

Issue(s)
Fixes #LESQ-886
How to test

Go to https://deploy-preview-2511--oak-web-application.netlify.thenational.academy
Click on any subject with new content
You should see banner should be rendered on unit listing, lesson listing and lesson overview pages of all subjects apart from Maths, which is lesson listing and lesson overview
Go to resource button should navigate to the new resource unit listing page, unless their are programme factors then programme listing page

Screenshots
How it should look now:

Checklist

[ ] Added or updated tests where appropriate
[ ] Manually tested across browsers / devices
[ ] Considered impact on accessibility
[ ] Design sign-off
[ ] Approved by product owner
[ ] Does this PR update a package with a breaking change

:tada: This PR is included in version 1.463.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Extended to make the saveAs(New)HadoopFile an async operation

@markhamstra replacing the old PR
Still use SimpleFutureAction, and similar way as submitJob, difference is the partition function takes additional task context

@markhamstra, recall the reason why I did it in the first version.
As we use the SimpleFutureAction to wrap the commit in the passed in handler, we can not guarantee that we also invoke this when the Failure case happened with write action. So I make the extension to ensure that we have a chance to always call the passed in functions regardless the write action failed to succeeded.
However, with the current spark sync version, it does not appear that the commit is invoked in Failure case, could this lead to resource "leaking"?

FWIW, look more with the spark hadoop writer, the abortJob is not exposed. So, the current form of async is on par with what spark support now.


I ran demo/image_demo.py with QueryInst, but nothing is output.

Describe the bug
A clear and concise description of what the bug is.
I ran demo/image_demo.py with QueryInst, but nothing is output.
Reproduction

What command or script did you run?

python demo/image_demo.py data/tiny_coco/val2017/000000580410.jpg ../configs/queryinst/queryinst_r50_fpn_1x_coco.py checkpoints/queryinst_r50_fpn_1x_coco_20210907_084916-5a8f1998.pth --device cuda

Did you make any modifications on the code or config? Did you understand what you have modified?

I set the dataset to tiny coco and used the officially distributed pretrained model.

What dataset did you use?

tiny coco for coco verification.
Environment

Please run python mmdet/utils/collect_env.py to collect necessary environment information and paste it here.

⋊> ~/l/c/Q/mmdetection-2.18.1 on main ◦ python mmdet/utils/collect_env.py    11:47:06
sys.platform: linux
Python: 3.8.6 (default, Nov 16 2021, 14:38:34) [GCC 7.5.0]
CUDA available: True
GPU 0,1: GeForce RTX 2080 Ti
CUDA_HOME: /usr/local/cuda-10.2
NVCC: Cuda compilation tools, release 10.2, V10.2.89
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
PyTorch: 1.10.0+cu102
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX512
  - CUDA Runtime 10.2
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70
  - CuDNN 7.6.5
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=7.6.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.11.1+cu102
OpenCV: 4.5.4-dev
MMCV: 1.3.17
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 10.2
MMDetection: 2.18.1+ffa0f07
⋊> ~/l/c/Q/mmdetection-2.18.1 on main ◦

You may add addition that may be helpful for locating the problem, such as

How you installed PyTorch [e.g., pip, conda, source]
Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

I am experimenting in the following repository. Keiku/QueryInst-Examples: Some examples on QueryInst
Error traceback
If applicable, paste the error trackback here.
⋊> ~/l/c/Q/mmdetection-2.18.1 on main ◦ python demo/image_demo.py data/tiny_coco/val2017/000000580410.jpg ../configs/queryinst/queryinst_r50_fpn_1x_coco.py checkpoints/queryinst_r50_fpn_1x_coco_20210907_084916-5a8f1998.pth --device cuda
/home/keiichi.kuroyanagi/.pyenv/versions/3.8.6-queryinst/lib/python3.8/site-packages/mmcv/utils/misc.py:333: UserWarning: "dropout" is deprecated in `FFN.__init__`, please use "ffn_drop" instead
  warnings.warn(
load checkpoint from local path: checkpoints/queryinst_r50_fpn_1x_coco_20210907_084916-5a8f1998.pth
/home/keiichi.kuroyanagi/.pyenv/versions/3.8.6-queryinst/lib/python3.8/site-packages/mmdet/datasets/utils.py:65: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  warnings.warn(
/home/keiichi.kuroyanagi/.pyenv/versions/3.8.6-queryinst/lib/python3.8/site-packages/mmdet/models/roi_heads/sparse_roi_head.py:355: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  bbox_pred_per_img = proposal_list[img_id][topk_indices //
⋊> ~/l/c/Q/mmdetection-2.18.1 on main ◦

Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

I may have misused demo instead of a bug.

I'm sorry. solved.

I'm sorry. solved.

Hi!
Were you able to train QueryInst with COCO dataset successfully? If yes, please let me know.

I executed the following command to save the image. I think that there is no problem with the same command in QueryInst.
python demo/image_demo.py data/tiny_coco/val2017/000000580410.jpg work_dirs/demo/000000580410.jpg configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py checkpoints/mask_rcnn_r50_fpn_1x_coco_20200205-d4b0c5d6.pth --device cuda

You can save the image by adding model.show_result(args.img, result, out_file=args.out_file).
# Copyright (c) OpenMMLab. All rights reserved.
import asyncio
from argparse import ArgumentParser

from mmdet.apis import (async_inference_detector, inference_detector,
                        init_detector, show_result_pyplot)

def parse_args():
    parser = ArgumentParser()
    parser.add_argument('img', help='Image file')
    parser.add_argument('out_file', help='Output Image file')
    parser.add_argument('config', help='Config file')
    parser.add_argument('checkpoint', help='Checkpoint file')
    parser.add_argument(
        '--device', default='cuda:0', help='Device used for inference')
    parser.add_argument(
        '--score-thr', type=float, default=0.3, help='bbox score threshold')
    parser.add_argument(
        '--async-test',
        action='store_true',
        help='whether to set async options for async inference.')
    args = parser.parse_args()
    return args

def main(args):
    # build the model from a config file and a checkpoint file
    model = init_detector(args.config, args.checkpoint, device=args.device)
    # test a single image
    result = inference_detector(model, args.img)
    # show the results
    show_result_pyplot(model, args.img, result, score_thr=args.score_thr)
    model.show_result(args.img, result, out_file=args.out_file)

async def async_main(args):
    # build the model from a config file and a checkpoint file
    model = init_detector(args.config, args.checkpoint, device=args.device)
    # test a single image
    tasks = asyncio.create_task(async_inference_detector(model, args.img))
    result = await asyncio.gather(tasks)
    # show the results
    show_result_pyplot(model, args.img, result[0], score_thr=args.score_thr)

if __name__ == '__main__':
    args = parse_args()
    if args.async_test:
        asyncio.run(async_main(args))
    else:
        main(args)

Hi,
Thanks for the quick reply.
But I want to train the network with this command:
python tools/train.py configs/queryinst/queryinst_r50_fpn_1x_coco.py
I am facing a lot of issues even after setting up the environment correctly. So, just wanted to know if someone else has already tried to train QueryInst on COCO from scratch.

In my environment, QueryInst was able to train and infer without any problems. Check if there is any problem in building the environment.

okay great to know.
I followed these steps to setup the environment:
• conda create -n e7
• conda activate e7
• conda install python=3.7
• source ../Add_to_Environment_Variables.sh
• conda install pytorch==1.7.0 torchvision==0.8.0 torchaudio==0.7.0 cudatoolkit=11.0 -c pytorch
• pip install mmcv-full==1.3.9 -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7.0/index.html
• git clone https://github.com/NVIDIA/apex.git
• pip install -r requirements.txt
• export CUDA_HOME=/usr/local/cuda-11.0
• pip install -v --no-cache-dir --global-option="--cuda_ext" pytorch-extension
• pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
• cd QueryInst/
• python setup.py develop
• mkdir data && cd data
• ln -s /home/soumya_pf_pt/documents/coco coco
• python tools/train.py configs/queryinst/queryinst_r50_fpn_1x_coco.py
Now I get this error when I try to train:
2022-03-31 11:27:51,839 - mmdet - INFO - workflow: [('train', 1)], max: 12 epochs
Traceback (most recent call last):
File "tools/train.py", line 187, in
main()
File "tools/train.py", line 183, in main
meta=meta)
File "/home/soumya_pf_pt/anaconda3/envs/e8/QueryInst/mmdet/apis/train.py", line 193, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/home/soumya_pf_pt/anaconda3/envs/e8/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/soumya_pf_pt/anaconda3/envs/e8/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 47, in train
for i, data_batch in enumerate(self.data_loader):
File "/home/soumya_pf_pt/anaconda3/envs/e8/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 352, in iter
return self._get_iterator()
File "/home/soumya_pf_pt/anaconda3/envs/e8/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 294, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "/home/soumya_pf_pt/anaconda3/envs/e8/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 827, in init
self._reset(loader, first_iter=True)
File "/home/soumya_pf_pt/anaconda3/envs/e8/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 857, in _reset
self._try_put_index()
File "/home/soumya_pf_pt/anaconda3/envs/e8/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1091, in _try_put_index
index = self._next_index()
File "/home/soumya_pf_pt/anaconda3/envs/e8/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 427, in _next_index
return next(self._sampler_iter)  # may raise StopIteration
File "/home/soumya_pf_pt/anaconda3/envs/e8/lib/python3.7/site-packages/torch/utils/data/sampler.py", line 227, in iter
for idx in self.sampler:
File "/home/soumya_pf_pt/anaconda3/envs/e8/QueryInst/mmdet/datasets/samplers/group_sampler.py", line 36, in iter
indices = np.concatenate(indices)
File "<array_function internals>", line 6, in concatenate
ValueError: need at least one array to concatenate

In cuda 10.2, I am running the following command.
pip install -U pip
pip install ipdb
pip install mmcv-full==1.3.17 -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.10.0/index.html
pip3 install torch torchvision
pip install mmdet==2.18.1
pip install scipy
pip install seaborn
pip install tensorboard
pip install jupyter notebook jupytext

In cuda 11.3, I am running the following command.
pip install -U pip
pip install ipdb
pip install mmcv-full==1.4.7 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html
pip install torch==1.10.0+cu113 torchvision==0.11.0+cu113 -f https://download.pytorch.org/whl/torch_stable.html
pip install mmdet==2.22.0
pip install scipy
pip install seaborn
pip install tensorboard
pip install jupyter notebook jupytext

@invincible-28 The version of mmcv-full may be out of date.

okay, I will try to run with mmcv-full==1.4.7 and check.

I was able to start training the network with my previous setup of the environment. (mmcv-full==1.3.9)
But, it is interrupted after the completion of every epoch.
Did you also face a similar issue?

@invincible-28 QueryInst was released in mmdetection v2.18.0. For mmdetection v2.18.0, mmcv-full is supported in 1.3.17 or later.
Prerequisites — MMDetection 2.23.0 documentation https://mmdetection.readthedocs.io/en/latest/get_started.html

I used mmcv-full==1.3.17

@invincible-28
Here are the steps to apply a simple custom dataset.

Convert the target dataset to COCO format. I always do this.
Define classes as follows:

dataset_type = 'CocoDataset'
# Define a classes
classes = ('class1', 'class2', '...', 'class_4', )
data_root = 'data/your_dataset/'

Specify the converted COCO format dataset. Specify val and test in the same way.

data = dict(
    samples_per_gpu=2,
    workers_per_gpu=2,
    train=dict(
        type='CocoDataset',
        # Add a custom class definition
        classes=classes,
        ann_file='data/your_dataset/annotations/your_dataset.json',
        img_prefix='data/your_dataset/images/',
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(type='LoadAnnotations', with_bbox=True, with_mask=True),
            dict(type='Resize', img_scale=(1333, 800), keep_ratio=True),
            dict(type='RandomFlip', flip_ratio=0.5),
            dict(
                type='Normalize',
                mean=[123.675, 116.28, 103.53],
                std=[58.395, 57.12, 57.375],
                to_rgb=True),
            dict(type='Pad', size_divisor=32),
            dict(type='DefaultFormatBundle'),
            dict(
                type='Collect',
                keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks'])
        ]),

QueryInst takes a lot of time, so let's practice with Mask R-CNN first.

Yes, I did follow all these steps.
I referred to this tutorial: https://mmdetection.readthedocs.io/en/latest/2_new_data_model.html

I was able to fix that error.
https://github.com/open-mmlab/mmdetection/issues/7863#issue-1218544240

Hi @Keiku
I am training this network with a very small custom dataset, so wanted to use data augmentation to increase the size of the dataset. I see that it is already been implemented here: /home/soumya_pf_pt/anaconda3/envs/env_greenai/QueryInst/mmdet/datasets/pipelines/transforms.py.
Is that correct what I have analysed or am I missing something? This network in itself is implementing various data augmentation techniques using the albumentations library. Is it?

@invincible-28 Does "small" mean dataset size? Or is it image size? If it is dataset size, I think that there is not much change even if data augmentation is done. If it is an image size, I think that data augmentation that increases the resolution is good.
My recommendation is the strong_baselines strategy. The following will be helpful.
New, improved Detectron2 Mask R-CNN baselines https://ai.facebook.com/blog/advancing-computer-vision-research-with-new-detectron2-mask-r-cnn-baselines/

@Keiku
By 'small', I mean the dataset size. So, my question is:
This network, queryinst in itself is implementing various data augmentation techniques using the albumentations library. Is it?

@invincible-28 queryinst is loading '../_base_/datasets/coco_instance.py' , so it doesn't use albumations.
https://github.com/open-mmlab/mmdetection/blob/ca11860f4f3c3ca2ce8340e2686eeaec05b29111/configs/queryinst/queryinst_r50_fpn_1x_coco.py#L2

@Keiku
queryinst has 6 configuration files with test time augmentation as you can see here:
https://github.com/hustvl/QueryInst/tree/main/configs/queryinst

@invincible-28 I'm not using hustvl/QueryInst so I'm not sure. Ask hustvl directly.
The following is train_pipeline, so it's not a test time augmentation.
    dict(
        type='Resize',
        img_scale=[(1333, value) for value in min_values],
        multiscale_mode='value',
        keep_ratio=True),
    dict(type='RandomFlip', flip_ratio=0.5),
    dict(type='Normalize', **img_norm_cfg),
    dict(type='Pad', size_divisor=32),
    dict(type='DefaultFormatBundle'),

okay.


does it mean write align the  lfs->cfg->prog_size?

Maybe it is the correct way!

Hi @AubrCool, thanks for the PR.
If I understand this PR correctly, the current math is by design.
LittleFS may call the block devices with > read/prog sizes, but the data will at least be a multiple of the read/prog sizes. This allows block devices to write out larger chunks of data more efficiently.
https://github.com/ARMmbed/littlefs/issues/96 may have more info.

I'm going to close this as I believe it's the correct behaviour. Feel free to reopen if you disagree.


Fix TF warnings noetic

In noetic we're getting seriously spammed by warning messages stating that there are repeated timestamps of the tf transforms.
This makes the simulation cmd line unuseable as it is impossible to read anything in it. I added some chacks to make sure this doesn't happen.

@rsoussan Please review and merge if it looks good!


Add gpu taint support

This PR just ensures that all tolerations outlined under the "gpus" key
in the singleuser profiles configmap are applied to a pod if it is of
type GPU. Once we are able to select a specific type of GPU in the
notebook spawner UI, we should update the logic under apply_gpu_config
to apply only the correct gpu toleration
Check example-configmap.yaml for an example of the gpu key.
We will need to update https://github.com/opendatahub-io/jupyterhub-odh as well once this PR is merged in.

We'll need a brief documentation for this

lgtm


better indent

Current dependencies on/for this PR:

main

PR #58   👈

This stack of pull requests is managed by Graphite.

Merge activity

Feb 6, 1:19 AM EST: @willseltzer merged this pull request with Graphite.


Potential to improve documentation

So I'm coming from this thread requesting that I posted some areas of the documentation that were causing me some confusion.
While I was searching through the documentation I noticed that this page doesn't list the setParam() or setOutputParam() methods[?].
On the same vein as the parameter fields, I think it would be helpful to link to this page or perhaps have your own version of that page, as I understand there are some extra steps.
My other problem was figuring out how to accomplish the geoprocessing task that I wanted.  I found this example, which is certainly helpful, but translating that to other geoprocessing tasks can be challenging.  However, (as far as I understand so far) there seem to be a limitless number of task / parameter combinations, so creating documentation for these tasks might be an overly large undertaking.
Hopefully this is helpful, please let me know if you have any follow up questions or would like me to clarify anything.

thanks for this @Metalyph.

I noticed that this page doesn't list the setParam() or setOutputParam() methods[?].

its a bit idiosyncratic, but L.esri.GP.Service is a wrapper object that is instantiated and used only to call createTask().  the reason i wrote the plugin this way initially because it exposed a mechanism thats already built into esri-leaflet core to do optional things like handle authentication or acknowledge that a particular server doesn't support CORS.

(as far as I understand) there seem to be a limitless number of task / parameter combinations

This is absolutely true.  This plugin fetches metadata from the service and is able to determine its input parameter names and valid input types, but i've never been able to think of any shortcuts to assist developers in setting appropriate values and associating them with the correct parameter that has a unique name.
opportunities for improvement

[ ] consider letting folks instantiate a GP.Task directly and hydrate an internal _service for them
[ ] GPLinearUnit JSON literals should be recognized without being coerced to a string
[ ] when the plugin fetches metadata, it should call setOutputParam() internally.
[ ] what else?

i know its been ages, but...

[X] GPLinearUnit JSON literals should be recognized without being coerced to a string


CFE_SB_ReceiveBuffer() -- switch timeout to int32

Describe the bug
Per discussion at the Jan. 6, 2020 CCB, @jphickey suggested that the timeout parameter should be int32, not uint32. This should be changed.
See also #1063
Reporter Info
<EMAIL_ADDRESS>

CFE_SB_ReceiveBuffer() is already int32 (see https://github.com/nasa/cFE/blob/1ede295c3874e6571aca965a7850d950cf7aa1a3/fsw/cfe-core/src/sb/cfe_sb_api.c#L1470-L1473)
But CFE_SB_ReadBuffer (internal function) is not.
Unless there is a good reason to "hide" OSAL definitions here (and I can't think of a good reason, because everything already depends on osapi.h anyway), I'd rather simplify and just make it so we have:
#define CFE_SB_PEND_FOREVER   OS_PEND
#define CFE_SB_POLL           OS_CHECK

... rather than redefining these values.
Right now we use the same underlying values but redefine them, so to be correct/safe the ReadQueue needs to translate.  With them defined this way we still have symbol independence (so SB can diverge from OSAL if it needs to, though I don't know why it would).  Then the ReadQueue function doesn't need to translate this value, so long as it shares the same definition.

CFE_SB_ReceiveBuffer() is already int32 (see https://github.com/nasa/cFE/blob/1ede295c3874e6571aca965a7850d950cf7aa1a3/fsw/cfe-core/src/sb/cfe_sb_api.c#L1470-L1473)
But CFE_SB_ReadBuffer (internal function) is not.
Unless there is a good reason to "hide" OSAL definitions here (and I can't think of a good reason, because everything already depends on osapi.h anyway), I'd rather simplify and just make it so we have:
#define CFE_SB_PEND_FOREVER   OS_PEND
#define CFE_SB_POLL           OS_CHECK

... rather than redefining these values.
Right now we use the same underlying values but redefine them, so to be correct/safe the ReadQueue needs to translate.  With them defined this way we still have symbol independence (so SB can diverge from OSAL if it needs to, though I don't know why it would).  Then the ReadQueue function doesn't need to translate this value, so long as it shares the same definition.

Actually, just assign this one to me.  As I'm going through code looking for race conditions in issue #1073 the entire design of CFE_SB_ReadQueue() is broken - it is accessing PipeDscPtr without the SB lock.  I will replace this with something that works.

Actually, just assign this one to me.  As I'm going through code looking for race conditions in issue #1073 the entire design of CFE_SB_ReadQueue() is broken - it is accessing PipeDscPtr without the SB lock.  I will replace this with something that works.


PVS Studio Analyzer Error V575 gflags.cc

I am getting the following error from PVS Studio Analyzer
V575 gflags.cc 556 The 'strncmp' function processes '0' elements. Inspect the third argument.

Thanks for reporting. I assume it's line 594 in case of the recent master HEAD. The kRootDir variable is indeed not used currently, so the condition will always be false. Haven't quite figured out the original intention of this code, and as long as it's not causing any harm left it in place. See also #86.

Closing this issue as kind of a duplicate of #86.

PS: If it's a problem to have a code analyzer report it as an error, please consider submitting a PR that silences the error/warning without changing the semantics of the code.

@schuhscuh do you want me to make a pull request removing this error and document it todo krootdir or something like that?

Yes, I thought @drillsar was going to prepare one. But nevermind, I've created it myself just now.
@drillsar or @tmccaff could you please check out the fix-219-strncmp-kRootDir-warning (#222) branch and test if it resolves the error before I merge the change?

@schuhschuh I will check it later tnt..

It's fixed.


CI results should be public

As an acs-engine OSS contributor
I should view full logs from builds
so I can receive feedback to contribute.

Closing in favor of https://github.com/Azure/acs-engine/issues/1134.


What's new in .NET 7 Preview 4 [WIP]

What's new in .NET 7 Preview 4
This issue is for teams to highlight work for the community that will release .NET 7 Preview 4
To add content, use a new conversation entry. The entry should include the team name and feature title as the first line as shown in the template below.
## Team Name: Feature title

[link to the tracking issue or epic item for the work]

Tell the story of the feature and anything the community should pay particular attention
to be successful using the feature.

Preview 1: https://github.com/dotnet/core/issues/7106
Preview 2: https://github.com/dotnet/core/issues/7107
Preview 3: https://github.com/dotnet/core/issues/7108
Preview 4: https://github.com/dotnet/core/issues/7378
Preview 5: https://github.com/dotnet/core/issues/7441
Preview 6: https://github.com/dotnet/core/issues/7454
Preview 7: https://github.com/dotnet/core/issues/7455

Observability
Introducing Activity.Current change event https://github.com/dotnet/runtime/issues/67276
    public partial class Activity : IDisposable
    {
        public static event EventHandler<ActivityChangedEventArgs>? CurrentChanged;
    }

Usage Example
Activity.CurrentChanged += CurrentChanged1;

void CurrentChanged1(object? sender, ActivityChangedEventArgs e)
{
    Console.WriteLine($"Activity.Current value changed from Activity: {e.Previous.OperationName } to Activity: {e.Current.OperationName}");
}

Adding Microseconds and Nanoseconds to TimeStamp, DateTime, DateTimeOffset, and TimeOnly https://github.com/dotnet/runtime/issues/23799
namespace System {
    public struct DateTime {
        public DateTime(int year, int month, int day, int hour, int minute, int second, int millisecond, int microsecond);
        public DateTime(int year, int month, int day, int hour, int minute, int second, int millisecond, int microsecond, System.DateTimeKind kind);
        public DateTime(int year, int month, int day, int hour, int minute, int second, int millisecond, int microsecond, System.Globalization.Calendar calendar);
        public int Microsecond { get; }
        public int Nanosecond { get; }
        public DateTime AddMicroseconds(double value);
    }
    public struct DateTimeOffset {
        public DateTimeOffset(int year, int month, int day, int hour, int minute, int second, int millisecond, int microsecond, System.TimeSpan offset);
        public DateTimeOffset(int year, int month, int day, int hour, int minute, int second, int millisecond, int microsecond, System.TimeSpan offset, System.Globalization.Calendar calendar);
        public int Microsecond { get; }
        public int Nanosecond { get; }
        public DateTimeOffset AddMicroseconds(double value);
    }
    public struct TimeSpan {
        public const long TicksPerMicrosecond = 10L;
        public const long NanosecondsPerTick = 100L;
        public TimeSpan(int days, int hours, int minutes, int seconds, int milliseconds, int microseconds);
        public int Microseconds { get; }
        public int Nanoseconds { get; }
        public double TotalMicroseconds { get; }
        public double TotalNanoseconds { get; }
        public static TimeSpan FromMicroseconds(double microseconds);
    }
    public struct TimeOnly {
        public TimeOnly(int day, int hour, int minute, int second, int millisecond, int microsecond);
        public int Microsecond { get; }
        public int Nanosecond { get; }
    }
}

Thanks to @ChristopherHaws and @deeprobin helping in the design and implementation.

More improvements and new APIs for System.Text.RegularExpressions
Augment Regex extensibility point for better perf and span-based matching
[API Proposal]: Add Regex.Enumerate(ReadOnlySpan) which is allocation free
Overhaul Regex's handling of RegexOptions.IgnoreCase
For preview 4 we are adding the remaining planned APIs in order to add span support into our Regex library. These APIs are amortized allocation-free. The main span-based APIs being added for Preview 4 are:

Regex.IsMatch(ReadOnlySpan<char> input):  Indicates whether the regular expression finds a match in the input span.
Regex.Count(ReadOnlySpan<char> input): Searches an input string for all occurrences of a regular expression and returns the number of matches.
Regex.EnumerateMatches(ReadOnlySpan<char> input): Searches an input span for all occurrences of a regular expression and returns a ValueMatchEnumerator to lazily iterate over the matches.

@tarekgh With pleasure. It makes me happy every time to improve the .NET ecosystem a bit more.
I now also follow up with yet another API for the next preview:
.NET Libraries: Use TimeSpan everywhere we use an int for seconds, milliseconds, and timeouts (Group 1/3) #64860
namespace System {
    public static class GC {
       public static GCNotificationStatus WaitForFullGCApproach(TimeSpan timeout);
       public static GCNotificationStatus WaitForFullGCComplete(TimeSpan timeout);
    }
}

namespace System.ComponentModel.DataAnnotations {
    public class RegularExpressionAttribute {
       public TimeSpan MatchTimeout { get; }
    }
}

namespace System.Diagnostics {
    public class Process {
       public bool WaitForExit(TimeSpan timeout);
       public bool WaitForInputIdle(TimeSpan timeout);
    }
}

namespace System.IO {
    public class FileSystemWatcher {
       public WaitForChangedResult WaitForChanged(WatcherChangeTypes changeType, TimeSpan timeout);
    }

    public sealed class NamedPipeClientStream : PipeStream {
       public void Connect(TimeSpan timeout);
       public Task ConnectAsync(TimeSpan timeout, CancellationToken cancellationToken);
    }
}

namespace System.Net.NetworkInformation {
    public class Ping {
       public PingReply Send(IPAddress address, TimeSpan timeout, byte[]? buffer, PingOptions? options);
       public PingReply Send(string hostNameOrAddress, TimeSpan timeout, byte[]? buffer, PingOptions? options);
       public Task<PingReply> SendPingAsync(IPAddress address, TimeSpan timeout, byte[]? buffer, PingOptions? options, CancellationToken cancellationToken);
       public Task<PingReply> SendPingAsync(string hostNameOrAddress, TimeSpan timeout, byte[]? buffer, PingOptions? options, CancellationToken cancellationToken);
    }
}

namespace System.Net.Sockets {
    public class NetworkStream : Stream {
       public void Close(TimeSpan timeout);
    }

    public class Socket {
       public bool Poll(TimeSpan timeout, SelectMode mode);
       public static void Select(IList checkRead, IList checkWrite, IList checkError, TimeSpan timeout);
    }
}

namespace System.ServiceProcess {
    public class ServiceBase {
       public void RequestAdditionalTime(TimeSpan time);
    }
}

namespace System.Threading.Tasks {
    public class Task {
       public bool Wait(TimeSpan timeout, CancellationToken cancellationToken);
    }
}

namespace System.Timers {
    public class Timer {
       public Timer(TimeSpan interval);
    }
}

/cc @danmoseley

Added metrics for Microsoft.Extensions.Caching

[x] https://github.com/dotnet/runtime/issues/50406

For preview 4 we added the main APIs in order to support metrics into our extensions caching library for IMemoryCache. The main APIs being added for Preview 4 are:

GetCurrentStatistics:  returns an instance of MemoryCacheStatistics which holds cache hit/miss/estimated size and count for IMemoryCache, or null when TrackStatistics flag is not enabled. Built-in implementation available for MemoryCache.

GetCurrentStatistics() API allows app developers to use either event counters or metrics APIs to track statistics for one or more memory caches with code snippets illustrated in this gist. Using these APIs getting statistics for multiple caches is possible but requires developer to write their own meter.

.NET Libraries: Nullable annotations for Microsoft.Extensions
https://github.com/dotnet/runtime/issues/43605
We have finished annotating the Microsoft.Extensions.* libraries for nullability. In .NET 7 Preview 4, all Microsoft.Extensions.* libraries have been fully annotated for nullability.
This work wouldn't have been possible without @maxkoshevoi's multiple-month effort. Starting with the first PR in August 2021, all the way to the final PR in April 2022, this was a lot of work that is greatly appreciated by the .NET community.

Adding new Tar APIs
Implement Tar APIs - dotnet/runtime#67883
We are adding the new System.Formats.Tar assembly, which contains APIs that allow reading, writing, archiving and extracting Tar archives.
Usage examples
For the most common usage, extracting and archiving, we are offering these methods:
// Generates a tar archive where all the entry names are prefixed by the root directory 'SourceDirectory'
TarFile.CreateFromDirectory(sourceDirectoryName: "D:/SourceDirectory/", destinationFileName: "D:/destination.tar", includeBaseDirectory: true);

// Extracts the contents of a tar archive into the specified directory, but avoids overwriting anything found inside
TarFile.ExtractToDirectory(sourceFileName: "D:/destination.tar", destinationDirectoryName: "D:/DestinationDirectory/", overwriteFiles: false);

We also offer variants that allow extracting from a stream or archiving into a stream:
// Generates a tar archive where all the entry names are prefixed by the root directory 'SourceDirectory'
using MemoryStream archiveStream = new();
TarFile.CreateFromDirectory(sourceDirectoryName: "D:/SourceDirectory/", destination: archiveStream, includeBaseDirectory: true);

// Extracts the contents of a stream tar archive into the specified directory, and avoids overwriting anything found inside
TarFile.ExtractToDirectory(source: archiveStream, destinationDirectoryName: "D:/DestinationDirectory/", overwriteFiles: false);

Additionally, the entries of an archive can be traversed one by one using the reader:
// Opens an archive for reading individual entries, and closes the archive stream after the reader disposal
using FileStream archiveStream = File.OpenRead("D:/SourceDirectory/source.tar");
using TarReader reader = new(archiveStream, leaveOpen: false);
TarEntry? entry;
while ((entry = reader.GetNextEntry()) != null)
{
    // Extracts an entry to the desired destination, and overwrites if an entry with the same name is found
    string destinationFileName = Path.Join("D:/DestinationDirectory", entry.Name);
    entry.ExtractToFile(destinationFileName, overwrite: true);
}

And entries can also be written one by one into an archive stream using the writer:
using FileStream archiveStream = File.OpenWrite("D:/DestinationDirectory/destination.tar");
// Create the archive in the PAX format and write individual entries, closing the stream after the writer disposal
using TarWriter writer = new(archiveStream, TarFormat.Pax, leaveOpen: false);

// Add an entry from an existing file
writer.WriteEntry(fileName: "D:/SourceDirectory/file.txt", entryName: "file.txt");

// Or write an entry from scratch
PaxTarEntry entry = new(entryType: TarEntryType.Directory, entryName: "directory");
writer.WriteEntry(entry);

We can also pair these new APIs with stream-based compression methods, like System.IO.Compression.GZipStream.
Extracting the contents from a compressed tar archive into the filesystem:
using FileStream compressedStream = File.OpenRead("D:/SourceDirectory/compressed.tar.gz");
using GZipStream decompressor = new(compressedStream, CompressionMode.Decompress);
TarFile.ExtractToDirectory(source: decompressor, destinationDirectoryName: "D:/DestinationDirectory/", overwriteFiles: false);

Reading individual entries from a compressed tar archive:
using FileStream compressedStream = File.OpenRead("D:/SourceDirectory/compressed.tar.gz");
using GZipStream decompressor = new(compressedStream, CompressionMode.Decompress);
using TarReader reader = new(decompressor, leaveOpen: true);
TarEntry? entry;

while ((entry = GetNextEntry(copyData: true)) != null)
{
    Console.WriteLine($"Entry type: {entry.EntryType}, entry name: {entry.Name}");
}

Writing the contents of a directory into a compressed tar archive:
using MemoryStream archiveStream = new();
TarFile.CreateFromDirectory(sourceDirectoryName: "D:/SourceDirectory/", destination: archiveStream, includeBaseDirectory: true);
using FileStream compressedStream = File.Create("D:/DestinationDirectory/compressed.tar.gz");
using GZipStream compressor = new(compressedStream, CompressionMode.Compress);
archiveStream.Seek(0, SeekOrigin.Begin);
archiveStream.CopyTo(compressor);

Or write individual entries into a compressed tar archive:
using MemoryStream archiveStream = new();
using (TarWriter writer = new TarWriter(archiveStream, TarFormat.Pax, leaveOpen: true))
{
    // Add an entry from an existing file
    writer.WriteEntry(fileName: "D:/SourceDirectory/file.txt", entryName: "file.txt");

    // Or write an entry from scratch
    PaxTarEntry entry = new(entryType: TarEntryType.Directory, entryName: "directory");
    writer.WriteEntry(entry);
}

using FileStream compressedStream = File.Create("D:/DestinationDirectory/compressed.tar.gz");
using GZipStream compressor = new(compressedStream, CompressionMode.Compress);
archiveStream.Seek(0, SeekOrigin.Begin);
archiveStream.CopyTo(compressor);

APIs
namespace System.Formats.Tar
{
    public static class TarFile
    {
        public static void CreateFromDirectory(string sourceDirectoryName, string destinationFileName, bool includeBaseDirectory);
        public static void CreateFromDirectory(string sourceDirectoryName, Stream destination, bool includeBaseDirectory);
        public static void ExtractToDirectory(string sourceFileName, string destinationDirectoryName, bool overwriteFiles);
        public static void ExtractToDirectory(Stream source, string destinationDirectoryName, bool overwriteFiles);
    }
    public enum TarEntryType : byte
    {
        V7RegularFile = '\0',
        RegularFile = '0',
        HardLink = '1',
        SymbolicLink = '2',
        CharacterDevice = '3',
        BlockDevice = '4',
        Directory = '5',
        Fifo = '6',
        GlobalExtendedAttributes = 'g',
        ExtendedAttributes = 'x',
        ContiguousFile = '7',
        DirectoryList = 'D',
        LongLink = 'K',
        LongPath = 'L',
        MultiVolume = 'M',
        RenamedOrSymlinked = 'N',
        SparseFile = 'S',
        TapeVolume = 'T',
    }
    public enum TarFormat
    {
        Unknown = 0,
        V7 = 1,
        Ustar = 2,
        Pax = 3,
        Gnu = 4,
    }
    public sealed class TarReader : System.IDisposable
    {
        public TarReader(System.IO.Stream archiveStream, bool leaveOpen = false);
        public System.Formats.Tar.TarFormat Format { get; }
        public System.Collections.Generic.IReadOnlyDictionary<string, string>? GlobalExtendedAttributes { get; }
        public void Dispose();
        public System.Formats.Tar.TarEntry? GetNextEntry(bool copyData = false);
    }
    public sealed class TarWriter : System.IDisposable
    {
        public TarWriter(System.IO.Stream archiveStream, System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, string>>? globalExtendedAttributes = null, bool leaveOpen = false);
        public TarWriter(System.IO.Stream archiveStream, System.Formats.Tar.TarFormat archiveFormat, bool leaveOpen = false);
        public System.Formats.Tar.TarFormat Format { get; }
        public void Dispose();
        public void WriteEntry(string fileName, string? entryName);
        public void WriteEntry(System.Formats.Tar.TarEntry entry);
    }
    public abstract class TarEntry
    {
        internal TarEntry();
        public int Checksum { get; }
        public System.IO.Stream? DataStream { get; set; }
        public System.Formats.Tar.TarEntryType EntryType { get; }
        public int Gid { get; set; }
        public long Length { get; }
        public string LinkName { get; set; }
        public TarFileMode Mode { get; set; }
        public System.DateTimeOffset ModificationTime { get; set; }
        public string Name { get; set; }
        public int Uid { get; set; }
        public void ExtractToFile(string destinationFileName, bool overwrite);
        public override string ToString();
    }
    public sealed class V7TarEntry : TarEntry
    {
        public V7TarEntry(System.Formats.Tar.TarEntryType entryType, string entryName);
    }
    public abstract class PosixTarEntry : TarEntry
    {
        internal PosixTarEntry();
        public int DeviceMajor { get; set; }
        public int DeviceMinor { get; set; }
        public string GroupName { get; set; }
        public string UserName { get; set; }
        public override string ToString();
    }
    public sealed class UstarTarEntry : TarEntryPosix
    {
        public UstarTarEntry(System.Formats.Tar.TarEntryType entryType, string entryName);
    }
    public sealed class PaxTarEntry : TarEntryPosix
    {
        public PaxTarEntry(System.Formats.Tar.TarEntryType entryType, string entryName, System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, string>>? extendedAttributes);
        public System.Collections.Generic.IReadOnlyDictionary<string, string> ExtendedAttributes { get; }
    }
    public sealed class GnuTarEntry : TarEntryPosix
    {
        public GnuTarEntry(System.Formats.Tar.TarEntryType entryType, string entryName);
        public System.DateTimeOffset AccessTime { get; set; }
        public System.DateTimeOffset ChangeTime { get; set; }
    }

    [System.FlagsAttribute]
    public enum TarFileMode
    {
        None = 0,
        OtherExecute = 1,
        OtherWrite = 2,
        OtherRead = 4,
        GroupExecute = 8,
        GroupWrite = 16,
        GroupRead = 32,
        UserExecute = 64,
        UserWrite = 128,
        UserRead = 256,
        StickyBit = 512,
        GroupSpecial = 1024,
        UserSpecial = 2048,
    }
}

Notes

We are not yet including any async APIs. They will be included in a future preview release.
In the Ustar, PAX and GNU formats, we are not yet supporting the UName, GName, DevMajor and DevMinor fields.
In the GNU format, we won't be supporting the rare entry types for writing, like sparse files, multi-volume, renamed or symlinked, tape volume.

CodeGen
Community PRs (Many thanks to JIT community contributors!)
From @SingleAccretion

https://github.com/dotnet/runtime/pull/66558
https://github.com/dotnet/runtime/pull/66251
https://github.com/dotnet/runtime/pull/65803
https://github.com/dotnet/runtime/pull/64581
https://github.com/dotnet/runtime/pull/67213
https://github.com/dotnet/runtime/pull/67208
https://github.com/dotnet/runtime/pull/67206
https://github.com/dotnet/runtime/pull/67486
https://github.com/dotnet/runtime/pull/67400

From @SkiFoD

https://github.com/dotnet/runtime/pull/65072

Dynamic PGO

https://github.com/dotnet/runtime/pull/65922

Arm64

https://github.com/dotnet/runtime/pull/66621
https://github.com/dotnet/runtime/pull/66407
https://github.com/dotnet/runtime/pull/66902
https://github.com/dotnet/runtime/pull/67490
https://github.com/dotnet/runtime/pull/67384

Loop Optimizations

https://github.com/dotnet/runtime/pull/68105

General Optimizations

https://github.com/dotnet/runtime/pull/67205
https://github.com/dotnet/runtime/pull/66983
https://github.com/dotnet/runtime/pull/67395
https://github.com/dotnet/runtime/pull/68055

On Stack Replacement (aka OSR)
On Stack Replacement allows the runtime to change the code executed by currently running methods in the middle of method execution, while those methods are active "on stack." It serves as a complement to tiered compilation.
https://github.com/dotnet/runtime/pull/65675 enabled OSR by default on x64 and Arm64, and enabled quick jitting for methods with loops on those same platforms.
OSR allows long-running methods to switch to more optimized versions mid-execution, so the runtime can jit all methods quickly at first and then transition to more optimized versions when those methods are called frequently (via tiered compilation) or have long-running loops (via OSR).
Performance Impact
OSR improves startup time. Almost all methods are now initially jitted by the quick jit. We have seen 25% improvement in startup time in jitting-heavy applications like Avalonia “IL” spy, and the various TechEmpower benchmarks we track show 10-30% improvements in time to first request (see chart below: OSR was enabled by default on March 30).

OSR can also improve performance of applications, and in particular, applications using Dynamic PGO, as methods with loops are now better optimized. For example, the Array2 microbenchmark showed dramatic improvement when OSR was enabled.

Further technical details
See the OSR Design Document for details on how OSR works.
See OSR Next Steps for details on the work that went into enabling OSR, and possible future enhancements.

.NET 7 GA is available. Closing these pre-release issues.


Consider replacing dependency "node-slate" with "whiteboard"

node-slate
A fork of a dead project that is a node implementation of Slate.
https://github.com/center-key/node-slate
whiteboard
A project closing in on a thousand stars that is a node implementation of Slate.
https://github.com/mpociot/whiteboard
...or make node-slate a fork of whiteboard.

Might consider closing this issue? Seems like node-slate outlived everything else :)


Fix-read-write

pandas 2.0.0 is incompatible with neighbor search
json saving and recovering finally works, imcluding contact graphs
sample data now opens the lables as labels - no need to comvert

contact graphs works again!


Port 27017 closed, why? it was working yesterday, and now doesn't?

Repro steps:

Action: cosmosDB.attachDatabaseAccount
Error type: Error
Error Message: connection 6 to bublikrowrescluster-shard-00-01-sqfpx.mongodb.net:27017 closed
Version: 0.10.2
OS: win32
Product: Visual Studio Code
Product Version: 1.37.0
Language: en

Call Stack
Function.MongoError.create mongodb-core/error.js:29:11
TLSSocket.closeHandler mongodb-core/connection.js:214:22
Object.onceWrapper events.js:273:13
TLSSocket.emit events.js:187:15
TLSSocket.EventEmitter.emit domain.js:442:20
Socket._destroy._handle.close net.js:606:12
Socket.done _tls_wrap.js:386:7
Object.onceWrapper events.js:273:13
Socket.emit events.js:182:13
Socket.EventEmitter.emit domain.js:442:20
TCP.Socket._destroy._handle.close net.js:606:12

My ISP changed my IP and it wasn’t on the whitelist on MongoDB Atlas.
Lost 5 hours of coding. Now I know

Thanks for letting us know, sorry to hear.


Added All Tall, All Small and All or Nothing at All bets

I added 3 new bets, AllTall, AllSmall, and AllOrNothingAtAll. We don't have these bets locally and I haven't played them myself, but I saw that they are very popular in Vegas from the Reddit/Twitter Craps Minimums spreadsheet. I took most of the information regarding how these bets work and the rules for them from here.
For anyone more familiar with this bet, do the odds listed change from the ones listed (34 and 175) or are those pretty consistent across tables? If not, we should probably add a way to edit the payouts for those, perhaps utilizing the table.payouts dict. Also, are there any requirements for when the bet is place (ex. requiring a new shooter or that the point is on/off)?
If neither of those are required these have tests and should be ready to go.

Based on https://wizardofodds.com/games/craps/side-bets/bonus-craps/ and https://www.crapspit.org/small-tall-all-bets/, I think that there are two payouts that are common:

34 to 1, 174 to 1
30 for 1, 150 for 1 (i.e. 29:1 and 149:1)

Of the two, I'm not sure which is more common but I'm checking on that. It would be ideal to add the option to table.payouts and the bet.
The bet can only be placed when there is a new shooter (as far as I am aware), so it needs similar logic to the Fire bet in PR #12

It seems the 30/30/150 payouts are more common, so let's start with those and improve the payout options later with issue #13.

Closing in favor of #37 because new bet changes required many changes

Closing in favor of #37  because new bet changes required many changes


Change signature of h3NeighborRotations to not accept CoordIJK*

This always required looking up the CoordIJK structure in some array, and then prompty changing it back to an indexing digit. Using an indexing digit also allows for some code to be cleaned up (deduplicated) in H3 index rotation.
This could set up for using an enum type for the indexing digits, which might provide some compiler niceties.
Did not clean up some defines like UNIT_VECS, which could now be replaced with some bitwise manipulation in _unitIjkToDigit (changes undefined behavior though.)

Coverage remained the same at 100.0% when pulling e3510b63b811513023b43b2b594f326e973ec5a3 on isaacbrodsky:h3neighborrotation-no-ijk into 73eefeac72e07558b113461a442c0a928e466d49 on uber:master.


readxl and shiny do not work together

Shiny function  fileInput stores uploaded files in a temp directory without extension. When I try to read the uploaded file with readxl it gives me the error: "Error: Unknown format ."
The same error I am getting when I rename an .xls file removing its extension and trying to read it.
The workaround is to rename the uploaded file adding the .xls extension back.
file.rename(inFile$datapath, paste(inFile$datapath, ".xlsx", sep=""))
On the topic of detection excel files without extension there is a thread on a mathlab forum that has few working solutions.
http://www.mathworks.com/matlabcentral/answers/76710-detecting-excel-files-without-excel-extension

+1

+1

Duplicate of #85


Removing StubInput class

Removing StubInput class and use iterator and partial next function to generate input

Tested with python 3.6

fantastic, thankyou!