Refactor use of FuncForPC to leverage new CallersFrames API.

Sorry to tack this on to another pending PR, I can create a new branch with this modification cherry-picked but I feel that the other changes were necessary regardless and this kind of eliminates the need for them entirely.
I believe this is what you were looking for in #10. The performance for this took a huge hit because runtime.CallersFrames allocates way more than simply grabbing the function pointer for the PC for obvious reasons. It does simplify much of the internal PC trickery, so perhaps the trade off is worth it.
There's also the possibility that my specific implementation may be less-than optimal as I'm not claiming to be a paragon of performant code here.
name                   old time/op  new time/op   delta
CallVFmt-8              498ns ± 1%    507ns ± 2%      ~     (p=0.057 n=4+4)
CallPlusVFmt-8          650ns ± 2%    658ns ± 4%      ~     (p=0.743 n=4+4)
CallSharpVFmt-8         504ns ± 5%    518ns ± 5%      ~     (p=0.486 n=4+4)
CallSFmt-8              451ns ± 1%    475ns ± 5%    +5.32%  (p=0.029 n=4+4)
CallPlusSFmt-8          587ns ± 1%    613ns ± 2%    +4.39%  (p=0.029 n=4+4)
CallSharpSFmt-8         438ns ± 1%    455ns ± 2%    +3.94%  (p=0.029 n=4+4)
CallDFmt-8              404ns ± 1%    403ns ± 1%      ~     (p=0.314 n=4+4)
CallNFmt-8              379ns ± 1%    389ns ± 4%      ~     (p=0.200 n=4+4)
CallPlusNFmt-8          347ns ± 1%    353ns ± 1%      ~     (p=0.086 n=4+4)
Caller-8                372ns ± 0%    616ns ± 3%   +65.68%  (p=0.029 n=4+4)
Trace-8                 850ns ± 4%   2306ns ± 1%  +171.40%  (p=0.029 n=4+4)
Trace10-8              2.82µs ± 1%   7.44µs ± 7%  +164.17%  (p=0.029 n=4+4)
Trace50-8              4.52µs ± 1%  16.28µs ± 1%  +260.27%  (p=0.029 n=4+4)
Trace100-8             7.83µs ± 1%  30.05µs ± 6%  +283.89%  (p=0.029 n=4+4)
CallerAndVFmt-8         918ns ± 2%   1206ns ± 7%   +31.25%  (p=0.029 n=4+4)
TraceAndVFmt-8         2.65µs ± 4%   4.19µs ± 4%   +58.23%  (p=0.029 n=4+4)
Trace10AndVFmt-8       9.38µs ± 1%  13.50µs ± 3%   +43.89%  (p=0.029 n=4+4)
RuntimeCaller-8         391ns ± 1%    404ns ± 6%      ~     (p=0.314 n=4+4)
RuntimeCallerAndFmt-8   714ns ± 2%    751ns ± 7%      ~     (p=0.114 n=4+4)
FuncForPC-8            9.27ns ± 1%  10.47ns ± 6%   +12.97%  (p=0.029 n=4+4)
FuncFileLine-8          123ns ± 1%    129ns ± 7%      ~     (p=0.086 n=4+4)

Coverage increased (+0.2%) to 89.418% when pulling 8c987d41f6f04d91c8947ea67fd4862ee3da2cc7 on dkushner:feature/callers-frames into 7a2f19628aabfe68f0766b59e74d6315f8347d22 on go-stack:master.

Coverage increased (+0.2%) to 89.418% when pulling c3e3c2362991d3271528ecc1e716c278a80454bb on dkushner:feature/callers-frames into 7a2f19628aabfe68f0766b59e74d6315f8347d22 on go-stack:master.

Coverage decreased (-0.3%) to 88.889% when pulling ece85f8961c0d77848618bd6920c6a161e2f5d62 on dkushner:feature/callers-frames into 7a2f19628aabfe68f0766b59e74d6315f8347d22 on go-stack:master.

Here is the new benchstat output after making the performance improvements we spoke about:
name                   old time/op  new time/op  delta
CallVFmt-8              560ns ± 2%   682ns ± 7%  +21.94%  (p=0.029 n=4+4)
CallPlusVFmt-8          712ns ± 2%   754ns ± 3%   +5.86%  (p=0.029 n=4+4)
CallSharpVFmt-8         553ns ± 5%   643ns ± 4%  +16.33%  (p=0.029 n=4+4)
CallSFmt-8              495ns ± 3%   586ns ± 2%  +18.33%  (p=0.029 n=4+4)
CallPlusSFmt-8          650ns ± 2%   689ns ± 1%   +6.00%  (p=0.029 n=4+4)
CallSharpSFmt-8         478ns ± 1%   586ns ± 1%  +22.52%  (p=0.029 n=4+4)
CallDFmt-8              428ns ± 2%   534ns ± 3%  +24.77%  (p=0.029 n=4+4)
CallNFmt-8              404ns ± 2%   605ns ± 2%  +49.66%  (p=0.029 n=4+4)
CallPlusNFmt-8          375ns ± 2%   579ns ± 6%  +54.33%  (p=0.029 n=4+4)
Caller-8                660ns ± 3%   408ns ± 8%  -38.28%  (p=0.029 n=4+4)
Trace-8                2.65µs ±24%  0.81µs ± 3%  -69.38%  (p=0.029 n=4+4)
Trace10-8              7.71µs ± 7%  2.52µs ± 2%  -67.34%  (p=0.029 n=4+4)
Trace50-8              17.5µs ± 2%   4.0µs ± 1%  -77.11%  (p=0.029 n=4+4)
Trace100-8             32.2µs ± 2%   6.9µs ± 2%  -78.47%  (p=0.029 n=4+4)
CallerAndVFmt-8        1.29µs ± 2%  1.08µs ± 3%  -16.51%  (p=0.029 n=4+4)
TraceAndVFmt-8         4.60µs ±10%  3.13µs ± 3%  -32.00%  (p=0.029 n=4+4)
Trace10AndVFmt-8       14.5µs ± 5%  11.1µs ±10%  -23.36%  (p=0.029 n=4+4)
RuntimeCaller-8         432ns ± 2%   434ns ± 5%     ~     (p=0.686 n=4+4)
RuntimeCallerAndFmt-8   782ns ± 3%   800ns ± 5%     ~     (p=0.629 n=4+4)
FuncForPC-8            11.2ns ± 4%  11.0ns ± 3%     ~     (p=0.686 n=4+4)
FuncFileLine-8          137ns ± 2%   143ns ±15%     ~     (p=0.657 n=4+4)

Note that this is against the previous, non-optimized version and not the original version that used FuncForPC.

Coverage increased (+2.01%) to 91.228% when pulling 5884afaf2218fca698778433dd34d7a3edbb1325 on dkushner:feature/callers-frames into c03b81b422d7169e7af59c08b4913d9d8a757590 on go-stack:develop.

Coverage decreased (-1.7%) to 87.467% when pulling 74cdfb9bbff7a42c75225a2b07d98ee990b0f3e3 on dkushner:feature/callers-frames into c03b81b422d7169e7af59c08b4913d9d8a757590 on go-stack:develop.

This looks good in general, but I'm not ready to merge it yet. It may still need some tweaks, but it will require some time to think through the best next step.
I ran a benchmark comparison of this PR against v1.5.3 and got the numbers shown below. It looks like the overall work is roughly the same it has just been shifted from the initial capture towards the formatting methods, which is actually a good thing. So performance is fine now.
Digging through the new code, though, I'm a little nervous about how it is using CallersFrames. I don't think the new code handles the sigpanic special case properly because it doesn't always use runtime.Frames.Next to traverse the entry prior to the one of interest. Take a look at the code for runtime.Frames and runtime.Frames.Next for the details about how that works.
I also just reviewed https://github.com/golang/go/issues/19348, which is the discussion about how Go will implement mid-stack inlining while preserving stack traces. The changes to  runtime.Callers and runtime.CallersFrames will change somewhat at that point and this package needs to work with those changes, but I don't understand all the implications yet.

Bench stats against v1.5.3 as of latest fixes to the PC offset required during panic stacks.
name                   old time/op  new time/op   delta
CallVFmt-8              549ns ± 6%    881ns ± 3%   +60.46%  (p=0.000 n=9+10)
CallPlusVFmt-8          705ns ± 2%    994ns ± 4%   +40.91%  (p=0.000 n=9+9)
CallSharpVFmt-8         539ns ± 3%    891ns ± 8%   +65.33%  (p=0.000 n=10+10)
CallSFmt-8              481ns ± 4%    828ns ± 4%   +72.19%  (p=0.000 n=10+10)
CallPlusSFmt-8          641ns ± 1%    937ns ± 6%   +46.26%  (p=0.000 n=8+10)
CallSharpSFmt-8         482ns ± 1%    847ns ±12%   +75.66%  (p=0.000 n=8+10)
CallDFmt-8              432ns ± 5%    788ns ± 8%   +82.30%  (p=0.000 n=10+10)
CallNFmt-8              408ns ± 5%    884ns ±11%  +116.56%  (p=0.000 n=10+9)
CallPlusNFmt-8          374ns ± 3%    850ns ± 9%  +127.35%  (p=0.000 n=10+10)
Caller-8                412ns ± 2%    391ns ± 6%    -5.05%  (p=0.000 n=9+10)
Trace-8                 931ns ± 1%    833ns ± 8%   -10.52%  (p=0.000 n=8+10)
Trace10-8              2.92µs ± 4%   2.73µs ± 7%    -6.49%  (p=0.000 n=10+10)
Trace50-8              4.88µs ± 5%   4.34µs ± 7%   -11.11%  (p=0.000 n=10+10)
Trace100-8             8.45µs ± 4%   7.77µs ± 9%    -8.09%  (p=0.000 n=10+10)
CallerAndVFmt-8        1.00µs ± 4%   1.38µs ± 2%   +37.16%  (p=0.000 n=10+9)
TraceAndVFmt-8         2.86µs ± 3%   4.28µs ± 3%   +49.31%  (p=0.000 n=10+8)
Trace10AndVFmt-8       9.68µs ± 3%  14.79µs ± 3%   +52.75%  (p=0.000 n=10+10)
RuntimeCaller-8         432ns ± 3%    422ns ± 4%    -2.27%  (p=0.037 n=9+9)
RuntimeCallerAndFmt-8   794ns ± 2%    774ns ± 5%      ~     (p=0.123 n=10+10)
FuncForPC-8            10.3ns ± 3%   11.1ns ± 8%    +7.77%  (p=0.001 n=9+10)
FuncFileLine-8          136ns ± 5%    133ns ±10%      ~     (p=0.097 n=9+10)

@dkushner, checking in to see if you plan to continue working on this PR? I can take it over if you'd prefer, so don't feel obligated, but I also want to let you finish it if you still wanted to.

@ChrisHines, I am indeed! I've been swept up in the transition to a new job and haven't had time to sit down and address this but I'll be able to get to it shortly.

Coverage remained the same at 89.216% when pulling 817915b46b97fd7bb80e8ab6b69f01a53ac3eebf on dkushner:feature/callers-frames into 44dafbc221f6841651265ef20270bae8c7f94b47 on go-stack:develop.

Coverage increased (+1.6%) to 90.789% when pulling 39f6a49178f5365f09d016c62df452118915eb37 on dkushner:feature/callers-frames into 44dafbc221f6841651265ef20270bae8c7f94b47 on go-stack:develop.

Coverage increased (+0.7%) to 89.912% when pulling 9aa07130e721ae1f2d2460d5d1e956c34bfd4b7d on dkushner:feature/callers-frames into 44dafbc221f6841651265ef20270bae8c7f94b47 on go-stack:develop.

Runtime and allocation benchmarks for the newest commits. I kind of decimated the previous changes by doing something very very stupid trying to resync my local.
benchmark                          old ns/op     new ns/op     delta
BenchmarkCallVFmt-8                379           689           +81.79%
BenchmarkCallPlusVFmt-8            530           776           +46.42%
BenchmarkCallSharpVFmt-8           407           689           +69.29%
BenchmarkCallSFmt-8                350           630           +80.00%
BenchmarkCallPlusSFmt-8            472           712           +50.85%
BenchmarkCallSharpSFmt-8           344           622           +80.81%
BenchmarkCallDFmt-8                312           599           +91.99%
BenchmarkCallNFmt-8                287           647           +125.44%
BenchmarkCallPlusNFmt-8            264           622           +135.61%
BenchmarkCaller-8                  339           288           -15.04%
BenchmarkTrace-8                   741           626           -15.52%
BenchmarkTrace10-8                 2601          1982          -23.80%
BenchmarkTrace50-8                 5147          3343          -35.05%
BenchmarkTrace100-8                9201          5459          -40.67%
BenchmarkCallerAndVFmt-8           806           1048          +30.02%
BenchmarkTraceAndVFmt-8            2321          4005          +72.55%
BenchmarkTrace10AndVFmt-8          7824          12383         +58.27%
BenchmarkRuntimeCaller-8           325           327           +0.62%
BenchmarkRuntimeCallerAndFmt-8     625           621           -0.64%
BenchmarkFuncForPC-8               7.50          7.04          -6.13%
BenchmarkFuncFileLine-8            101           101           +0.00%

benchmark                          old allocs     new allocs     delta
BenchmarkCallVFmt-8                3              4              +33.33%
BenchmarkCallPlusVFmt-8            3              4              +33.33%
BenchmarkCallSharpVFmt-8           3              4              +33.33%
BenchmarkCallSFmt-8                2              3              +50.00%
BenchmarkCallPlusSFmt-8            2              3              +50.00%
BenchmarkCallSharpSFmt-8           2              3              +50.00%
BenchmarkCallDFmt-8                2              3              +50.00%
BenchmarkCallNFmt-8                2              3              +50.00%
BenchmarkCallPlusNFmt-8            2              3              +50.00%
BenchmarkCaller-8                  0              0              +0.00%
BenchmarkTrace-8                   1              1              +0.00%
BenchmarkTrace10-8                 1              1              +0.00%
BenchmarkTrace50-8                 1              1              +0.00%
BenchmarkTrace100-8                1              1              +0.00%
BenchmarkCallerAndVFmt-8           3              4              +33.33%
BenchmarkTraceAndVFmt-8            10             15             +50.00%
BenchmarkTrace10AndVFmt-8          32             38             +18.75%
BenchmarkRuntimeCaller-8           0              0              +0.00%
BenchmarkRuntimeCallerAndFmt-8     3              3              +0.00%
BenchmarkFuncForPC-8               0              0              +0.00%
BenchmarkFuncFileLine-8            0              0              +0.00%

benchmark                          old bytes     new bytes     delta
BenchmarkCallVFmt-8                40            56            +40.00%
BenchmarkCallPlusVFmt-8            72            88            +22.22%
BenchmarkCallSharpVFmt-8           104           120           +15.38%
BenchmarkCallSFmt-8                32            48            +50.00%
BenchmarkCallPlusSFmt-8            64            80            +25.00%
BenchmarkCallSharpSFmt-8           96            112           +16.67%
BenchmarkCallDFmt-8                24            40            +66.67%
BenchmarkCallNFmt-8                48            64            +33.33%
BenchmarkCallPlusNFmt-8            80            96            +20.00%
BenchmarkCaller-8                  0             0             +0.00%
BenchmarkTrace-8                   64            80            +25.00%
BenchmarkTrace10-8                 240           256           +6.67%
BenchmarkTrace50-8                 896           896           +0.00%
BenchmarkTrace100-8                1792          1792          +0.00%
BenchmarkCallerAndVFmt-8           40            56            +40.00%
BenchmarkTraceAndVFmt-8            192           456           +137.50%
BenchmarkTrace10AndVFmt-8          640           1152          +80.00%
BenchmarkRuntimeCaller-8           0             0             +0.00%
BenchmarkRuntimeCallerAndFmt-8     40            40            +0.00%
BenchmarkFuncForPC-8               0             0             +0.00%
BenchmarkFuncFileLine-8            0             0             +0.00%

@dkushner Also, please update the .travis.yml file to run tests for Go 1.7 and 1.8.

Superceded by #19.


Why loss the link of .ckpt file ?

Hi,
I try to use this net to do a prediction but I cann't find the .ckpt file.
I noticed that your .ckpt had a link before, why do you delete its link now?
Because I just want to know the prediction effect of the low-precision picture as input for depth estimation, I wonder if you can provide a .ckpt file or  its link?
Regard，
Mathilda.

Hi,
I see that you close the issue, did you find the link to download the model checkpoint?
Can you tell me how to download it?
Thank you in advance,
Khuong


compactor: Improve handling of Partial Aborted Uploads.

Problem:
Current flow for detecting partial blocks looks like this:
Partial aborted upload

Compactor
Producer (e.g. sidecar, or previous compactor run)

Uploads block, fails before meta.json was finished

Producer is being removed (e.g downscale, cluster shutdown)

Partial block present on object storage (no / malformed meta.json)

Sync metas, sees partial block

What's the age of ULID? Is it older than PartialUploadThresholdAge? (2d) -> NO

(2d passes)

Sync metas, sees partial block

What's the age of ULID? Is it older than PartialUploadThresholdAge? (2d) -> YES

Block is deleted from object storage.

Our mechanism works generally but there are also cases it does not:
Producer down for 2d

Compactor
Producer (e.g. sidecar, or previous compactor run)

Block was produced by Prometheus

Uploads block, fails before meta.json was finished

Partial block present on object storage (no / malformed meta.json)

Sidecar is down for some reason / Object storage connection is down

Sync metas, sees partial block

What's the age of ULID? Is it older than PartialUploadThresholdAge? (2d) -> NO

(2d passes)

All is back again, yay! Let's retry upload!

Sync metas, sees partial block

What's the age of ULID? Is it older than PartialUploadThresholdAge? (2d) -> YES

Block is deleted from object storage.

Upload of missing meta.json succeeded, yay! Let's remove the local instance of this block.

WTH Block with just meta.json?
WTH Block with just meta.json?

It's super rare and theoretical, it's rare something important is down for 2d. And if it's fine we can increase the PartialUploadThresholdAge. There is
however more plausible scenario:
Producer uploads old block

Compactor
Producer (e.g. sidecar, or previous compactor run)

Block was produced by Prometheus

Uploads block, fails before meta.json was finished

Partial block present on object storage (no / malformed meta.json)

Sidecar is down for some reason / Object storage connection is down

Sync metas, sees partial block

What's the age of ULID? Is it older than PartialUploadThresholdAge? (2d) -> NO

(2d passes)

All is back again, yay! Let's retry upload!

Sync metas, sees partial block

What's the age of ULID? Is it older than PartialUploadThresholdAge? (2d) -> YES

Block is deleted from object storage.

Upload of missing meta.json succeeded, yay! Let's remove the local instance of this block.

WTH Block with just meta.json?
WTH Block with just meta.json?

Let's fix it (:
Ideas:

Partial block mark JSON
ModififedTime API call for object storages.

When retrying upload (after connection/service is restored), how about checking if the ULID still exists in the bucket before attempting to upload the remaining files?
If it does not exist, we can conclude it has been removed by compactor and can attempt full reupload of block.
What do you think?

@darshanime upload is not atomic, so ID can exists but not the full block.
Anyway, ModififedTime API call for object storages. looks like the way to go.
Thanks to @pracucci we now have this API so we can move to that :+1:

AC:

Move to Modified Time

Duplicate of https://github.com/thanos-io/thanos/issues/2585


Wrap in wrapper

We are using react-intl in many components. These components access intl features via context.
I could wrap it in every example, but thats not nice solution. So would be nice to have that as option. My quick hardcoded fix looks like:
export default class Wrapper extends Component {
	static propTypes = {
		children: PropTypes.node.isRequired
	}

	render() {
		return (
			<IntlProvider locale='en' initialNow={ Date.now() }>
				{this.props.children}
			</IntlProvider>
		)
	}
}

So what I basically needs is to have option in styleguide-config.js that would be function that can do the wrapping.
So Wrapper component would like something like this:
export default class Wrapper extends Component {
	static propTypes = {
		children: PropTypes.node.isRequired
	}

	render() {
                const wrapperFn = this.props.config.wrapperFn;
                return wrapperFn ? wrapperFn(this.props.children) : this.props.children
	}
}

I can provide PR if you want, in that case just not sure what make sense more to you - if pass the whole config down..  or just that specific option through component hierarchy. In other words how to get to wrapperFn option down to Wrapper component.
Thanks for feedback!

I don’t understand how it’s differ from the Wrapper component.

@sapegin I apologize.. I just went through the config options and missed that wrapper component can be replaced like this - which is exactly I need.
Thanks!


How to use nth-child in Css file

i am unable to use
#DocumentDetail > div > div > div > div:nth-child(2) > div
in my default.css nth-child is not allowed.
Is there any alternate of nth-child(2)?

Hi Pashmina nth-child is one of the CSS selectors that are not working now. Hope Alex can solve this bug since he knows more about how that work. In general, we should support nth-child(2n), nth-child(even), nth-child(odd). I think that's the main functions although more cases could be needed.

@carlosrovira Thanks

The emulation components shouldn't need to support fancy CSS like nth-child since Flex didn't support it so it isn't a priority in my list currently.  Folks who want to can look at past commits that allowed other fancy CSS and upgrade support in the compiler.
From the emulation components perspective, the question is what the higher-level goal is.  In Flex, subclasses or styleNames/classNames are added to specific children and then styled using less-fancy CSS.
Or, IOW, what was working in Flex that isn't working in Royale?  Highlighting some selected item?  Whatever it is, it probably means that the way Flex supported custom visuals needs to be emulated in Royale.

Hi Alex, I think this is not a problem for emulation components (I mean don't think that should be use as a way to solve something in some MX Comp). I think this is a problem of Royale it self, crossing any solution. People look on how to solve things vía CSS and solutions on internet uses this selector and others. We have partial support for selectors at this time. So this is a problem for end user doing CSS things at Application level. Pashmina or myself suffer this working with Royale at App level (I as well at framework level working with Jewel).
A part from philosophical motivations about our CSS bugs and why we should prioritize or not I tried to solve it myself. I'm interested in fix this kind of bugs.
I tried a couple of things that didn't worked.
I tried to add to CSS.g and CSSTree.g:

CSS.g

Added to /* Lexer Rules */ this one: NTH_CHILD : 'nth-child' ;
Added to singleValue :  |   NTH_CHILD ARGUMENTS		    -> ^(NTH_CHILD ARGUMENTS)

CSSTree.g

In singleValue returns [CSSPropertyValue propertyValue], added:   |   ^(NTH_CHILD l=ARGUMENTS)
{ $propertyValue = new CSSFunctionCallPropertyValue($NTH_CHILD.text, $l.text, $start, tokenStream); }

Other option I tried was to add:

CSS.g

Added to FUNCTIONS:       | 'nth-child'

In JSCSSCompilationSession.java:

Added to otherCSSFunctions:   "nth-child"

any of this options results in another error, that seems better one, but still mostly impossible to debug for me.
A side from solving 2N, I think as well we need to add in some place values like EVEN, ODD


<EMAIL_ADDRESS>uses deprecated verison of ::add-path::

According to https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
::add-path:: is deprecated and needs to be converted to either use SDK api or use $GITHUB_PATH
See https://github.com/nodenv/actions/issues/13#issuecomment-742018324
~Looks like the fix is to upgrade @actions/core to 1.2.6 which is already done https://github.com/nodenv/actions/commit/8a7b195f56113779f4ece6bc3c3344366356ef93~
~We just need to cut a new release~
Error using it today
Run<EMAIL_ADDRESS>  with:
    nodenv-version: 1.3.1
/bin/tar xz --warning=no-unknown-keyword -C /home/runner/work/_temp/486819ad-7523-4bf8-afe4-2613328e2e39 -f /home/runner/work/_temp/b5e8c9bd-ef22-4217-ae4c-7cc185a3c2e4
Error: Unable to process command '::add-path::/opt/hostedtoolcache/nodenv/1.3.1/x64/bin' successfully.
Error: The `add-path` command is disabled. Please upgrade to using Environment Files or opt into unsecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_COMMANDS` environment variable to `true`. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/

bump

@jasonkarns are you able to cut a release?

Hmm, I tried the latest commit from master, and it still failed:
name: My CI

on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: nodenv/actions/node-version@104f741d7a447326be77c28fe4fff36758c3ac40
        id: nodenv
      - uses: actions/setup-node@v1
        with:
          node-version: '${{ steps.nodenv.outputs.node-version }}'
      - run: npm run test

I thought that would work. It said in the documentation you can use the monorepo directly

@asgeo1 I think i see the problem. According to the blogpost

Action and workflow authors who are setting environment variables via STDOUT should update any usage of the set-env and add-path workflow commands to use the new environment files.

::add-path:: is STDOUT and not the SDK method

@asgeo1 also, looking at the source. its very confusing. seems like the source uses the core addPath, but the dist still uses ::add-path::. So do we just need to make a build?

Yeah that's right, it's using the dist version.  I realised that later.  So yeah it would need a new build published to resolve it.
But in the interim, you can also set the ACTIONS_ALLOW_UNSECURE_COMMANDS environment variable as a work around:
name: My CI

on: [push, pull_request]

jobs:
  run-specs:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      # only needed until next version of nodenv/actions/node-version is released
      - name: allow-unsecure-commands
        run: echo "ACTIONS_ALLOW_UNSECURE_COMMANDS=true" >> $GITHUB_ENV
        id: allow-unsecure-commands

      - uses<EMAIL_ADDRESS>        id: nodenv

      - uses: actions/setup-node@v1
        with:
          node-version: '${{ steps.nodenv.outputs.node-version }}'

      - run: npm install
      - run: npm run test

That works for me

2.0.8 was just released with a re-build of dist. Please let me know if this resolves the issue.
Thank you all for your patience!

issue resolved. thank you.


Cannot execute runOSGI task under gradle 7.3.3

When I try, I get the following errors:
Problem creating boot delegation class loader: java.lang.reflect.InaccessibleObjectException: Unable to make protected java.security.SecureClassLoader(java.lang.ClassLoader) accessible: module java.base does not "opens java.security" to unnamed module @2957fcb0
org.osgi.framework.BundleException: Unable to resolve jline [1](R 1.0): missing requirement [jline [1](R 1.0)] osgi.ee; (&(osgi.ee=JavaSE)(version=1.5)) Unresolved requirements: [[jline [1](R 1.0)] osgi.ee; (&(osgi.ee=JavaSE)(version=1.5))]
ERROR: Bundle jline [1] Error starting file:/home/angle/Documents/Programming/Java/OSGIGradleSandbox/build/osgi/bundle/jline-2.14.2.jar (org.osgi.framework.BundleException: Unable to resolve jline [1](R 1.0): missing requirement [jline [1](R 1.0)] osgi.ee; (&(osgi.ee=JavaSE)(version=1.5)) Unresolved requirements: [[jline [1](R 1.0)] osgi.ee; (&(osgi.ee=JavaSE)(version=1.5))])
	at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:4148)
	at org.apache.felix.framework.Felix.startBundle(Felix.java:2118)
	at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1372)
	at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
	at java.base/java.lang.Thread.run(Thread.java:833)
org.osgi.framework.BundleException: Unable to resolve org.apache.felix.scr [2](R 2.0): missing requirement [org.apache.felix.scr [2](R 2.0)] osgi.ee; (|(&(osgi.ee=JavaSE)(version=1.6))(&(osgi.ee=JavaSE/compact1)(version=1.8))) Unresolved requirements: [[org.apache.felix.scr [2](R 2.0)] osgi.ee; (|(&(osgi.ee=JavaSE)(version=1.6))(&(osgi.ee=JavaSE/compact1)(version=1.8)))]
ERROR: Bundle org.apache.felix.scr [2] Error starting file:/home/angle/Documents/Programming/Java/OSGIGradleSandbox/build/osgi/bundle/org.apache.felix.scr-2.0.12.jar (org.osgi.framework.BundleException: Unable to resolve org.apache.felix.scr [2](R 2.0): missing requirement [org.apache.felix.scr [2](R 2.0)] osgi.ee; (|(&(osgi.ee=JavaSE)(version=1.6))(&(osgi.ee=JavaSE/compact1)(version=1.8))) Unresolved requirements: [[org.apache.felix.scr [2](R 2.0)] osgi.ee; (|(&(osgi.ee=JavaSE)(version=1.6))(&(osgi.ee=JavaSE/compact1)(version=1.8)))])
	at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:4148)
	at org.apache.felix.framework.Felix.startBundle(Felix.java:2118)
	at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1372)
	at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
	at java.base/java.lang.Thread.run(Thread.java:833)
org.osgi.framework.BundleException: Unable to resolve com.athaydes.osgiaas.cli-api [4](R 4.0): missing requirement [com.athaydes.osgiaas.cli-api [4](R 4.0)] osgi.wiring.package; (osgi.wiring.package=javax.annotation) Unresolved requirements: [[com.athaydes.osgiaas.cli-api [4](R 4.0)] osgi.wiring.package; (osgi.wiring.package=javax.annotation)]
	at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:4148)
ERROR: Bundle com.athaydes.osgiaas.cli-api [4] Error starting file:/home/angle/Documents/Programming/Java/OSGIGradleSandbox/build/osgi/bundle/osgiaas-cli-api-0.7.jar (org.osgi.framework.BundleException: Unable to resolve com.athaydes.osgiaas.cli-api [4](R 4.0): missing requirement [com.athaydes.osgiaas.cli-api [4](R 4.0)] osgi.wiring.package; (osgi.wiring.package=javax.annotation) Unresolved requirements: [[com.athaydes.osgiaas.cli-api [4](R 4.0)] osgi.wiring.package; (osgi.wiring.package=javax.annotation)])
	at org.apache.felix.framework.Felix.startBundle(Felix.java:2118)
	at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1372)
	at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
	at java.base/java.lang.Thread.run(Thread.java:833)
ERROR: Bundle com.athaydes.osgiaas.cli-core [5] Error starting file:/home/angle/Documents/Programming/Java/OSGIGradleSandbox/build/osgi/bundle/osgiaas-cli-core-0.7.jar (org.osgi.framework.BundleException: Unable to resolve com.athaydes.osgiaas.cli-core [5](R 5.0): missing requirement [com.athaydes.osgiaas.cli-core [5](R 5.0)] osgi.wiring.package; (&(osgi.wiring.package=com.athaydes.osgiaas.api.env)(version>=0.7.0)(!(version>=1.0.0))) [caused by: Unable to resolve com.athaydes.osgiaas.common [6](R 6.0): missing requirement [com.athaydes.osgiaas.common [6](R 6.0)] osgi.wiring.package; (osgi.wiring.package=javax.annotation)] Unresolved requirements: [[com.athaydes.osgiaas.cli-core [5](R 5.0)] osgi.wiring.package; (&(osgi.wiring.package=com.athaydes.osgiaas.api.env)(version>=0.7.0)(!(version>=1.0.0)))])
org.osgi.framework.BundleException: Unable to resolve com.athaydes.osgiaas.cli-core [5](R 5.0): missing requirement [com.athaydes.osgiaas.cli-core [5](R 5.0)] osgi.wiring.package; (&(osgi.wiring.package=com.athaydes.osgiaas.api.env)(version>=0.7.0)(!(version>=1.0.0))) [caused by: Unable to resolve com.athaydes.osgiaas.common [6](R 6.0): missing requirement [com.athaydes.osgiaas.common [6](R 6.0)] osgi.wiring.package; (osgi.wiring.package=javax.annotation)] Unresolved requirements: [[com.athaydes.osgiaas.cli-core [5](R 5.0)] osgi.wiring.package; (&(osgi.wiring.package=com.athaydes.osgiaas.api.env)(version>=0.7.0)(!(version>=1.0.0)))]
	at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:4148)
	at org.apache.felix.framework.Felix.startBundle(Felix.java:2118)
	at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1372)
	at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
	at java.base/java.lang.Thread.run(Thread.java:833)
ERROR: Bundle com.athaydes.osgiaas.common [6] Error starting file:/home/angle/Documents/Programming/Java/OSGIGradleSandbox/build/osgi/bundle/osgiaas-common-0.7.jar (org.osgi.framework.BundleException: Unable to resolve com.athaydes.osgiaas.common [6](R 6.0): missing requirement [com.athaydes.osgiaas.common [6](R 6.0)] osgi.wiring.package; (osgi.wiring.package=javax.annotation) Unresolved requirements: [[com.athaydes.osgiaas.common [6](R 6.0)] osgi.wiring.package; (osgi.wiring.package=javax.annotation)])
org.osgi.framework.BundleException: Unable to resolve com.athaydes.osgiaas.common [6](R 6.0): missing requirement [com.athaydes.osgiaas.common [6](R 6.0)] osgi.wiring.package; (osgi.wiring.package=javax.annotation) Unresolved requirements: [[com.athaydes.osgiaas.common [6](R 6.0)] osgi.wiring.package; (osgi.wiring.package=javax.annotation)]
	at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:4148)
	at org.apache.felix.framework.Felix.startBundle(Felix.java:2118)
	at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1372)
	at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
	at java.base/java.lang.Thread.run(Thread.java:833)
> Task :runOsgi

This is without adding any bundle to the project, and only using the default osgiRuntime.

Hi. This plugin has not been updated in a long time, unfortunately, so it can't run with newer versions of Gradle (which keeps changing, causing this kind of problem).
I do want to get back to this project at some point, but I can't promist I will fix this soon. If you have time to submit a PR, i would be happy to help merge a fix.

Actually , looking at the errors, it seems more related to Java 11? I never tried to run this on JDK11. I think that should be easy to fix.

@Angular-Angel I have been able to get things running by upgrading all dependencies.
Try something like this and it should work:
plugins {
    id 'com.athaydes.osgi-run'
}

repositories {
    mavenCentral()
}

def GOGO_BUNDLES = [
        'org.apache.felix:org.apache.felix.gogo.runtime:1.1.4',
        'org.apache.felix:org.apache.felix.gogo.shell:1.1.4',
        'org.apache.felix:org.apache.felix.gogo.command:1.1.2',
].collect { [ dependency: it, transitive: false ] }.asImmutable()

runOsgi {
    osgiMain = "org.apache.felix:org.apache.felix.main:7.0.1"
    bundles = GOGO_BUNDLES
}

The error you reported about the annotations missing could be fixed with this (if running on Felix):
runOsgi {
    osgiMain = "org.apache.felix:org.apache.felix.main:7.0.1"
    bundles = GOGO_BUNDLES
    config += [ 'org.osgi.framework.system.packages.extra': 'javax.annotation' ]
}

I will try to publish an updated version of the plugin, but as lots of things have changed recently, it's not a trivial amount of work.

Hmm, gotcha. I'll try and figure something out on monday, will get back to you then!

I am pushing some code on the dev branch. I think I got both Felix and Equinox working again, but have to enable all test projects still to be sure.
Please try the dev branch to see if everything works for you.

I published version 2.0 of the plugins, built against Gradle 7.4.
It's already on Maven Central and should be soon in the Gradle Plugin Repository.
Let me know if you run into problems.

Okay, so it looks like it works now? Except when I use the 'runOsgi' task, it gives me:
Feb 14, 2022 12:13:34 PM org.jline.utils.Log logr
WARNING: Unable to create a system terminal, creating a dumb terminal (enable debug logging for more information)

And then when I try the various commands in terminal, it doesn't seem to respond? But if I run 'stop 0' then a couple seconds later it will give some kind of error and shut down, so maybe it's getting the commands and responding to them, but just not giving me any output from them? :/
I found this other issue with the same error message, which was fixed by this commit, which updated the version of jline, so maybe that's the issue?

Oh, should I make a new issue for this?

Please try running it like this:
./gradlew createOsgi

chmod +x build/osgi/run.sh
build/osgi/run.sh

It seems like your terminal was not detected when running the run task... so try running it directly as shown above.
If you still have issues, send me the contents of your osgi/bundle directory.

Also, what OS are you using? And what terminal?

Huh, yeah, okay, it works like that. I'm using Xubuntu 20.04 (focal), running the thing via netbeans. Works fine when I run it via xfce4-terminal. So, it must be a problem due to netbeans, good to know. :/

Huh, wait though. When I do './gradlew runOsgi' in xfce4-terminal, I get the same issue. So it's not just netbeans. :/

I created a new issue: https://github.com/renatoathaydes/osgi-run/issues/71


Change from xlsx to writexl

Hi -
I have changed the dependency from xlsx to writexl.
xlsx depends on java and I understand that Oracle have changed the license terms for Java so it is very expensive for large organisations.  Writexl (and there are others) don't depend on java.
My company has removed java from our computers because of the cost and so I couldn't run the original package.

Great! Thank you for highlighting and changing this.


⚠ Secret Website 3 has degraded performance

In 3f7433e, Secret Website 3 ($SECRET_WEBSITE_3) experienced degraded performance:

HTTP code: 200
Response time: 21072 ms

Resolved: Secret Website 3 performance has improved in 73cde9c.


Support multibinding - set

Support multibinding for set

resolved: https://github.com/Park-SM/HiltBinder/pull/15


feat: apkam spec changes and keys verb impl

- What I did

Implemented keys verb , changes to apkam spec, refactoring
- How I did it
Renamed enrollApprovalId to enrollmentId for consistency
Removed EnrollNamespace class and replaced with Map<String,String> (key- namespace, value-access) for ease of retrieval while checking authorisation for enrollments
Added AtEnrollmentException to GlobalExceptionHandler
In AbstractVerbHandler, added restriction for __global and __manage namespace. only keys verb will have access to these namespaces
Changes to EnrollVerbHandler

store encrypted default encryption private and encrypted default self encryption
For first client approval, store apkam public key in existing key name -  AT_PKAM_PUBLIC_KEY. For other approvals store apkam public key in pkams.public_keys format.

For approval notification, stored encrypted apkam symmetric key in notification value.
Implemented KeysVerbHandler with put, get and delete params
Made changes in scan verb handler to not display __manage keys
added unit and functional tests for keys verb, namespace access and enroll verb

- How to verify it

run the unit tests enroll_data_store_value_test.dart, keys_verb_test.dart
run the functional tests enroll_namespace_access_test.dart, enroll_verb_test.dart, keys_verb_test.dart

- Description for the changelog

Nearly there! Please resolve the conflicts and confirm the answers to my questions in the unresolved comments above

The changes related to the Lookup verb are addressed in the PR: https://github.com/atsign-foundation/at_server/pull/1505

See my most recent comment above #1496 (comment)

@gkc: The ticket for notify verb is created: https://github.com/atsign-foundation/at_server/issues/1444
A test in scan_verb_test.dart is updated, and also the tests in scan_verb_test.dart updated and the changes are available in the following PR: https://github.com/atsign-foundation/at_server/pull/1505


Validating value type properties with [Required] attribute

Because value types are non-nullable, they are assigned a default value during model binding. So for instance, if you have a Birthday property of type DateTime which is required on your model, and a date isn't supplied in the data posted to the controller, it will be assigned the value of default(DateTime).
At that point, it's too late for model validation to enforce the [Required] attribute on that property, because even default(DateTime) counts as a value and therefore passes the [Required] validator.
One way to handle this is to run model binding in two phases, one pre-phase which only resolves where in the model each posted property will go (along with the validation attributes on the model at that destination), then a post-phase which does the actual assignment. In between, you can run the model validation on the posted object directly, with the validation information coming from the model based on the pre-phase.

Use the framework-specific [BindRequired] attribute to ensure a value was included in the submission. (MVC pretty much calls IsValid()on ValidationAttributes and does not reinterpret the results.)
FYI the System.ComponentModel.DataAnnotations namespace is part of CoreFx. You can find relevant sources at https://github.com/dotnet/corefx/tree/master/src/System.ComponentModel.Annotations/src/System/ComponentModel/DataAnnotations or file issues at https://github.com/dotnet/corefx/issues.

Is that supposed to be used in conjunction with the regular '[Required]attribute? Or does this replace it? FWIW, I just tried using the[BindRequired]attribute, and theTryValidateModel` still returns true.
What might be happening is that since I'm posting the object from C# in an integration test, and in the C# class it's a value type, it gets submitted with a default value in the request. Which means for C# integration tests at least, there's no way to catch it with the built-in attributes.
Let me know if I'm missing something here.

[BindRequired] affects model binding only and that comes before validation. So nothing will change for your integration test around TryValidateModel(). However a real action will see errors in the ModelStateDictionary if a property with [BindRequired] is not bound (aka no data is provided for it).

used in conjunction with the regular [Required] attribute? Or does this replace it?

The two attributes serve different purposes and should be used however your application requires. If the application needs to confirm data was submitted, use [BindRequired]. If the application needs to confirm a value is not null, use [Required].

Nope, my hypothesis is false. I just tried posting an object of anonymous type with only a subset of the model's fields, so there's no way it could have filled in defaults for them. On the receiving end, TryValidateModel still returned true.

TryValidateModel() does not model bind and [BindRequired] is not relevant if that's all you're doing. Either add a parameter to your action with the type i.e. do model binding automatically or call TryUpdateModelAsync()

I believe my action already has a parameter with the type, so model binding should be happening automatically. Something like this:
[HttpPost]
public async Task Post(MyModel m) { ... }

If I omit the properties of MyModel which are marked with the [BindRequired] attribute, the action is still invoked and m has defaults filled in.
I think I'm missing something here...

In the scenario you're describing, there's no need to call TryValidateModel(). Check ModelState.IsValid and it should be false if no data is provided for the [BindRequired] properties.

Aha, perfect, that was exactly my problem. Thank you!

I swear I saw it work before I closed the ticket, but now this doesn't work. I posted an object with [BindRequired] value properties where those properties weren't set in the client (C# client, specifically), and ModelState.IsValid returns true in the controller. Something like this:
public class A { [BindRequired] Guid G { get; set; } }

await Client.PostAsync("some_url", new A().ToJsonContent()); // custom extension method which does what you think

// controller's ModelState.IsValid returns true when the request arrives

I still think it has something to do with the fact that when C# serializes an instance of A, because G is a value type, it gets assigned a default value (the generated JSON confirms this). This fools the [BindRequired] attribute. But I know for a fact that I've seen it work, and I was surprised that it did work, because that would imply the `Client

The only workaround I can think of is to make all value types nullable with ?, then just use the [Required] attribute. This works 100% of the time, without any ambiguity.

Is the action using [FromBody]? If yes, you must use Json.NET to control errors it adds. [BindRequired] doesn't apply in this new scenario.

It does, so that's likely my problem. I don't quite understand the fix, would you mind illustrating?

So why would this not be a bug / design flaw? It doesn't seem right that changing where in the request the object comes from (or changing the JSON library you're using) should allow the object to suddenly start bypassing the [BindRequired] attribute.
I suspect I know why it worked just fine earlier, and that's because the framework automatically decided that the object should be pulled from the body, without the [FromBody] attribution, and at that point the [BindRequired] attribute was working fine. I guess I made some kind of change, and suddenly the model parameter was coming in as null. I fixed it by adding the [FromBody] attribute, but then [BindRequired] stopped working. To me, that's a bug. Or why is that the expected behavior? It was certainly unexpected to me.

[BindRequired] customizes the MVC model binding system . That's its purpose and it's working as designed.
[FromBody] switches the affected property or parameter into the different world of input formatting. Each input formatter (e.g. Json.NET and a small MVC-specific wrapper) can be considered a separate system with its own customization. The model binding system has no knowledge the details of JSON (or any other) deserialization.
In an ideal world, we would have a common set of attributes that work everywhere. Pragmatically that is not currently possible and, even if it were, every serializer / deserializer would still have its own additional attributes.

The only workaround I can think of is to make all value types nullable with ?, then just use the [Required] attribute. This works 100% of the time, without any ambiguity.

I also find this odd behavior. To make a value required, you have to make it nullable and then add Required attribute?
From users perspective it makes no sense, now you have nullable property which is guarenteed to have a value. It may make sense in a API level, but not when you are using the API.

How does [BindNever] work with value types? Does my type have to be nullable? Also I'm using [FromBody] as well and none of the attributes are working. What is a workaround? I have a simple POST action method that it needs to accept JSON. Do I have to remove the [FromBody] atribute or do I have to use key value url pairs for that? Sorry if that's a lot of questions

How does [BindNever] work with value types?

The model binding system will not attempt to bind the value.

Does my type have to be nullable?

No, it will be left at its default value.

What is a workaround?

Your problem is not clear. Please open a new issue describing what you're doing, what you expect, and and what is happening instead.


Couple of errors.

Hello - I am trying to rune a finetuning (potentially a useful model for the hub!) And ran into a couple of errors.

When testing out the log likelihood scores of the base foundational model on a multi site mutational library I received:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[<ipython-input-3-9e1c691e3ebe>](https://localhost:8080/#) in <cell line: 94>()
    110     # result_df['Sequence'] = dataset_df['Sequence']
    111     # result_df['mutation'] = dataset_df['mutation']
--> 112     dataset_df['score'] = results
    113
    114     output_path = OUTPUT_HOME / f"{timestamp}_prediction_output_{Path(dataset_csv_path).stem}.csv"

6 frames
[/usr/local/lib/python3.10/dist-packages/torch/_tensor.py](https://localhost:8080/#) in __array__(self, dtype)
    970             return self.numpy()
    971         else:
--> 972             return self.numpy().astype(dtype, copy=False)
    973
    974     # Wrap Numpy array again in a suitable tensor when done, to support e.g.

TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

results is a list of size 1 tensors. This was easily fixed by results = [r.cpu().item() for r in results] and then manually downloading.

When finetuning, post training, I received:

---------------------------------------------------------------------------
HFValidationError                         Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/peft/config.py](https://localhost:8080/#) in _get_peft_type(cls, model_id, **hf_hub_download_kwargs)
    196             try:
--> 197                 config_file = hf_hub_download(
    198                     model_id,

9 frames
[/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py](https://localhost:8080/#) in _inner_fn(*args, **kwargs)
    105             if arg_name in ["repo_id", "from_id", "to_id"]:
--> 106                 validate_repo_id(arg_value)
    107

[/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py](https://localhost:8080/#) in validate_repo_id(repo_id)
    153     if repo_id.count("/") > 1:
--> 154         raise HFValidationError(
    155             "Repo id must be in the form 'repo_name' or 'namespace/repo_name':"

HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/content/SaprotHub/adapters/regression/Local/Model-evc-35M'. Use `repo_type` argument if needed.

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
[<ipython-input-14-6d9f4a5baa00>](https://localhost:8080/#) in <cell line: 258>()
    256
    257 from saprot.scripts.training import finetune
--> 258 finetune(config)
    259
    260

[/usr/local/lib/python3.10/dist-packages/saprot/scripts/training.py](https://localhost:8080/#) in finetune(config)
     48                 config.model.kwargs.lora_kwargs.config_list = [{"lora_config_path": model.save_path}]
     49
---> 50             model = my_load_model(config.model)
     51
     52         else:

[/usr/local/lib/python3.10/dist-packages/saprot/utils/module_loader.py](https://localhost:8080/#) in my_load_model(config)
     35       if 'num_labels' in model_config: del model_config['num_labels']
     36       from model.saprot.saprot_regression_model import SaprotRegressionModel
---> 37       return SaprotRegressionModel(**model_config)
     38
     39     if model_type == "saprot/saprot_pair_classification_model":

[/usr/local/lib/python3.10/dist-packages/saprot/model/saprot/saprot_regression_model.py](https://localhost:8080/#) in __init__(self, test_result_path, **kwargs)
     16         """
     17         self.test_result_path = test_result_path
---> 18         super().__init__(task="regression", **kwargs)
     19
     20     def initialize_metrics(self, stage):

[/usr/local/lib/python3.10/dist-packages/saprot/model/saprot/base.py](https://localhost:8080/#) in __init__(self, task, config_path, extra_config, load_pretrained, freeze_backbone, gradient_checkpointing, lora_kwargs, **kwargs)
     67
     68             self.lora_kwargs = EasyDict(lora_kwargs)
---> 69             self._init_lora()
     70
     71         self.valid_metrics_list = {}

[/usr/local/lib/python3.10/dist-packages/saprot/model/saprot/base.py](https://localhost:8080/#) in _init_lora(self)
     94                 if i == 0:
     95                     # If i == 0, initialize a PEFT model
---> 96                     self.model = PeftModelForSequenceClassification.from_pretrained(self.model,
     97                                                                                lora_config_path,
     98                                                                                adapter_name=adapter_name,

[/usr/local/lib/python3.10/dist-packages/saprot/model/saprot/self_peft/peft_model.py](https://localhost:8080/#) in from_pretrained(cls, model, model_id, adapter_name, is_trainable, config, **kwargs)
    326         if config is None:
    327             config = PEFT_TYPE_TO_CONFIG_MAPPING[
--> 328                 PeftConfig._get_peft_type(
    329                     model_id,
    330                     subfolder=kwargs.get("subfolder", None),

[/usr/local/lib/python3.10/dist-packages/peft/config.py](https://localhost:8080/#) in _get_peft_type(cls, model_id, **hf_hub_download_kwargs)
    201                 )
    202             except Exception:
--> 203                 raise ValueError(f"Can't find '{CONFIG_NAME}' at '{model_id}'")
    204
    205         loaded_attributes = cls.from_json_file(config_file)

ValueError: Can't find 'adapter_config.json' at '/content/SaprotHub/adapters/regression/Local/Model-evc-35M'

This goes into the  importable finetune function. I do not currently have the time to go fix it and make a pull.
Thanks for your work in making PLMs zero shot and finetuning more accessible! Really great work.
Evan

Hello, thank you very much for providing sush useful feedback!
For the first error, as you said it can be easily fixed by moving all tensors from GPU to CPU. We have corrected this error and updated the notebook so it should work well now.
For the second error, could you provide more information about how you ran the training? It seems like the program didn't find the adapter_config.json file to initialize a lora module. If you have finished training and saved the model weights to a folder, it should contain a list of files like:

@LTEnjoy I was running a simple regression training! I lost the traceback because I did not copy the notebook. I will try to run it again and see if I can give something more informative.

Hi, can you try rerunning this on our latest colab link?
https://colab.research.google.com/github/westlake-repl/SaprotHub/blob/main/colab/SaprotHub.ipynb
We did lots of tests so it should not run into error.

@LTEnjoy sorry for tardiness, will give it a go maybe next week.

done


Redirect auth/failure to DSI login

Trello card URL:
https://trello.com/c/ikmTQ91m/769-fix-404-error-as-dsi-is-sometimes-redirecting-to-auth-failure-when-credentials-are-invalid
Changes in this PR:
When users use the back button after logging in, DfE signin redirects the user to /auth/failure on our app, which returns a 404. This redirects that path to the HiringStaff::IdentificationsController#create action, which then in turn prompts the user to log in again.

 Fix 404 error as DSI is sometimes redirecting to auth/failure when credentials are invalid

When testing this locally I get OmniAuth::Strategies::OpenIDConnect::CallbackError

That's after I've signed in successfully and I'm on /school, then I hit the back button - is that correct?

Yep, I see the same thing. Lemme see if there's a better way to handle this


Global module

This series of commits re-introdces ModuleDescriptor as one stop shop for all information about a module.

bors r+


Add the env name to logs

Is it worth it adding build_name to the logs too?

build name is added by the pod labels, so I think it is alright.


Enhancements to Compare Two Years Graph

Compare Two Years Graph Comments:
Bigger Dotes on Choropleth, esp for Texas with 254 counties!
Possible Enhancement: Allow the Choropleth to be adjustable to allow the user to zoom in and out.  I would like to zoom in or select a box of data to highlight and investigate further.
Surprisingly, I was able to plot both States at the same time. but when I clicked on the points, I did not get any of the site information. If we continue to maintain that more than one State can be on the graph, I would consider color coding dots by States (Shades of Blue for the first State and Shades of green for the second State, etc.). Also, allow for site information to be displayed with State and County names.
edit: moved some text to another issue...
When I have more than one State and select the legend, the graph gets really small.
image

Moveability Problem:
When I try to scroll down the left hand side to get to the Counties, change the County code to County name, or even access all the States, the program locks up and I can't scroll down below a State or County.
The first time it happened, I couldn't go farther than 01007. I got out of the program and it did it again, but this time I couldn't go below NC. I restarted my computer, and it would let me go below ME this time.
I noticed that that RStudio was starting the Explore Water Use program in Internet Explorer. I copied the url and put it in Google Chrome and tried it again. It also locked up. At first I could get down to the bottom. I tried changing the County code to County name, but the program wouldn't change. Then, once I scrolled back up, it stopped me from go past 39127.
County Name Button Problem:
It is very important to me that I can change the legend to the County Names from the County codes. I really think that this will be a time saver for everyone. Sometimes it worked for me and other times it didn’t.

The legend should scroll now.
I'm not sure what's happening with the scrolling problem. I always choose 1 state, then minimize that drop-down. Then, I'm able to scroll down all the county or huc's.
I would not suggest trying to select all states.

Could you describe when choosing "county names" did not work?

My scroll was working a little wonky but only when I chose 2 states and one of them was Texas.  It seemed like as long as I just used 1 state and let the program adjust (ie gave it a little time) and minimized the state legend, I was ok.


Validating user input in contact us section

Without putting the details in the form even then it submits the form

I have resolved the issue  in #66


Add SchemaUpdate logic to Iceberg-Rust

The general philosophy of any engine that implements Iceberg-Rust is that they should not care about things like field-IDs. Therefore we try to hide this logic behind APIs. To allow evolution of the schema, we have the UpdateSchema API in Python and the SchemaUpdate (implements UpdateSchema) in Java. It would be good to have this one in Rust as well :)

Hey @Fokko
I would like to give this a shot

@vinjai That would be great! Feel free to ping me if you run into anything


master, and previous versions

I would like to have master and the previous version in the documentation. I think if we do something like this would be great.
@SooluThomas could you look at this and make this a high priority as then as the team improves the documentation in the master and we set a build every day (?) then this would allow us to see how it changes.

@mtreinish and @SooluThomas and @nonhermitian could we discuss this and how we can do it all.

I think we need to have tagging system enabled to tag the commits to their respective versions, which I think we have that. The only thing that we need now is a button or a means to display the latest version tag and toggle between the versions.

We should investigate this: https://docs.readthedocs.io/en/stable/versions.html.  This would be a good choice for theme as well.  It is already used by everyone.

Yes. Using RTD is one of the best options.

Im good with that as well but lets start another issue for theme.

Not sure if this is one the roadmap or not anymore.  Namely we host our docs at qiskit.org/documentation/* where as what we really need is qiskit.org/documentation/latest/* or qiskit.org/documentation/0.25/.  Closing now, can reopen if needed.

The previous releases half of this was just implemented in #1261 there should be a dropdown now and let you go to old versions. I went as far back as 0.24, but we can go even further back but it'll require someone to manually build and upload the old versions (and update the sphinx build conf accordingly). The builds from master/main already existed via CI (both here and for the project repos) where you can download the built documentation as artifacts from the CI build.


Random-access iterators over IVector/IVectorView

The built-in iterators for IVector and IVectorView (and more generally anything that has a GetAt method) now support random-access operations. They do not meet all the requirements of a random-access iterator, most notably:

reference is not value_type& or value_type const&.
operator-> is not supported.

Design notes:

The value_type had been misdeclared as T (e.g. IVector<Something>) rather than the element type (Something).
The reference is not actually a reference because the return value of operator* must be convertible to reference.
The element index is unsigned so that the relational operators like < produce correct results on large containers.
The difference_type is ptrdiff_t so that b-a and related operations work on large containers.
Return types for iterator operations are explicit to ensure they conform to iterator requirements.
We now rely on std::reverse_iterator to implement our reverse iterator.

/azp run


Provide automatic stacksafe interpretation for @tagless algebras

We should be able to wrap this style of interpreters https://github.com/47deg/freestyle/blob/master/freestyle-tagless/shared/src/test/scala/taglessTests.scala#L108 automatically if the user chooses to interpret a program in a stack safe fashion.
For example program[Option] is stack unsafe but if there is a way for us to lift the value unevaluated so we can do program[Option @@ Stacksafe] or program.interpret[Option] We could wrap the user defined handlers that are stack unsafe by default automatically into the context of Free without the overhead of the Free ADT currently expanded at MyAlgebra.StackSafe.
The goal is that the user won't need to manually create a handler for MyAlgebra.Handler[FreeS[Option, ?]] but just provide one MyAlgebra.Handler[Option] and we do the lifting for them.

As of #310, you can write program[Free[Option, ?]].runTailRec or program[StackSafe[Option]#λ].runTailRec.


Only able to apply colour to 1 word / sentence

Hi,
I can select some text and mark that text with a colour, for example yellow like on the screenshot below.

After that when I select another piece of text the colour on the first one will disappear as soon as I click on the colour select button (see screenshots above and below).

Hi @royhermans, finding trouble replicating this issues. See the attached GIF. Could you please replicated this in a simple test project and send me the mpk.

Hi @edwinmp , it took me a little while to be able to replicate the issue myself in a test project but I managed to do it. I have zipped the mpk file.
RichText_Test.zip

RichText_Test_2.zip
The problem seems to be caused by the sanize string action. In our project (and the testproject as well) we use use an OC save mf which also sanitizes the string.
In this testproject I removed the java action from the MF so I get the exact same string out as I put in and then the problem doesn't happen.

Hi @royhermans, thanks for the reproduction. It led me to another bug partly responsible for the above behaviour. To adhere to the existing behavior of the Mendix input widgets, the onChange microflow is only triggered on blur (if a change was made). Unfortunately, clicking on the editor toolbar was being taken as a blur event & would cause the editor to lose focus.
This shall be fixed in the next release.
As to what's happening in your projects, the only solution is as you've realised ... removing the sanitize action.

The selection issue has been fixed in version 1.0.2.


Add MediaCodecInfo to MediaCodecRenderer::onReadyToInitializeCodec

This adds the MediaCodecInfo to the onReadyToInitializeCodec to allow for renderers to know which codec is being initialize, not only which format the codec will be initialized for. With multiple codecs supporting a format, knowing which one is being used would be helpful.

I'm going to send this for internal review now. You may see some more commits being added as I make changes in response to review feedback. Please refrain from pushing any more substantive changes as it will complicate the internal review - thanks!


immutable map in EO

implement the simplest possible immutable map, which maps strings to EO objects.
Speaking it terms of scala, we need something like
abstract class eomap() {
// (no exceptions!)
def find(key : String) : Option[Object]
// if the key already exists in the map then remove it
def add(key : String, value : Object) : eomap
def remove(key : String) : eomap
}
Use the most simple linear implementaiton (presumably, with arrays).
See https://github.com/cqfn/eo/tree/master/sandbox/hse/eo before coding, maybe it is at least partially done there

Implementing Scala's Option[Object] with object of two attributes: .valid and .value. First attribute is set to true if the value is found.


Added default timeout env variable

It would be nice to pass timeout variable via system properties, so can configure for egxample debug environment with long/infite timeouts


Nothing gets created and Java files don't show in File Finder window

Doesn't work. Am on a Mac.

The problem that did not run on Windows has been fixed.

Has this fix been published?  Latest version as of today v0.0.9 still generates nothing on my Mac.
More Info
Published
5/28/2022, 15:03:04
Last released
6/25/2023, 03:55:15
Last updated
6/26/2023, 09:31:47
Identifier
oh318.j-diagram

P.S. The changelog in the extension is not being maintained - that would be good place to note any fixes IMO.

Hello, I am Kim Jin-il in charge of J-diagram maintenance.
Sorry for the late reply.
Could you send me the Draw.io file for testing in a different environment?
-----Original Message-----
From: "Andy @.>
To: @.>;
Cc: "Jinil @.>; "State @.>;
Sent: 2023-06-26 (월) 08:37:31 (GMT+09:00)
Subject: Re: [OH318/J-Diagram] Nothing gets created and Java files don't show in File Finder window (Issue #6)
Has this fix been published? Latest version as of today v0.0.9 still generates nothing on my Mac.
More Info Published 5/28/2022, 15:03:04 Last released 6/25/2023, 03:55:15 Last updated 6/26/2023, 09:31:47 Identifier oh318.j-diagram
P.S. The changelog in the extension is not being maintained - that would be good place to note any fixes IMO.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you modified the open/close state.Message ID: @.***>

Where is the generated draw.io diagram file supposed to be? As I said in the issue this extension "generates nothing on my Mac."

The draw.io file is generated in the directory path placed by user.
The problem can taken place when the access modifer of member variable is default. Because "org.jboss.forge.roaster.Roaster" only support private, protected, and public.
In addtion, the java interface source code does not support. we are going to update the interface function.
We could not consider the all cases when developing the j-diagram. If you need the other function, tell me about it. I going to update the j-diagram.
-----Original Message-----
From: "Andy @.>
To: @.>;
Cc: "Jinil @.>; "State @.>;
Sent: 2023-07-31 (월) 08:57:15 (GMT+09:00)
Subject: Re: [OH318/J-Diagram] Nothing gets created and Java files don't show in File Finder window (Issue #6)
Where is the generated draw.io diagram file supposed to be? As I said in the issue this extension "generates nothing on my Mac."
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you modified the open/close state.Message ID: @.***>

I finally got some ouput! The extension's "Current directory" output option is misleading and probably causing most users of this extension to think it is broken.

I just installed the extension fresh.
I created a simple java file

package com.aaa.bbb;

public class Simple {
    public static void main(String[] args) {
        System.out.println("Hello, World!");
    }

    // method that adds two numbers
    public static int add(int a, int b) {
        return a + b;
    }
}

I select the command "J Diagram: Convert java sources to draw.io file"
I select "open file finder" and select the Simple.java file
I select output to "Current directory"
I look for the draw.io file in the same directory as theSimple.java file but cannot find it.

Update: turns out that "Current directory" output option puts the file in
target/classes/com/aaa/bbb/j-diagram.drawio where nobody will find it.
The better option to use when "Open File Finder" when asked "Choose where to where to make draw.io" diagram and selecting a better location within your project.
P.S. Also nothing is generated when the draw.io file already exists.


Missing packages to rescue data

RH OpenShift uses rsync and tar(info) to copy data to and from a pod we need to add those packages in case of an emergency we can get access to the data inside.
https://github.com/cloud-bulldozer/ocm-api-load/blob/3b797f1e2a040f155667596cbfb48ca3f2da2cdb/Dockerfile#L5

@chentex are there any other packages that may be useful in the case where you might need to get into the pod to debug?

I cannot think of others, do you have any suggestions @dry923 ?


Added PEP 8 formatting and some minor fixes

please do merge as per the compatibility.

Thank you for you contribution to the repo and the community
On 12 Feb 2017, at 08:14, Yash Ladha<EMAIL_ADDRESS>wrote:

please do merge as per the compatibility.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


clear TextInput value on invalid value selected

Clears TextInput on invalid value

setValue(undefined) to clear TextInput

Hello @fateh999 ,
May I know if you have abandon this project? The issue mentioned here in been fixed by @KeaganStevens for a while, and waiting to be merged. This issue is a show stopper for someone who wants to reset the dropdown value through react hook form.
Appreciate your response. Thanks.


[BUG] Build fails on Flutter 3.16 for iOS or Windows

Describe the bug
The app can't be build because of the following error: Error (Xcode): Target native_assets failed: Exception: Package(s) x require the native assets feature to be enabled. Enable using flutter config --enable-native-assets.
To Reproduce
Upgrade to Flutter 3.16, and try to build the app for iOS or Windows.
Library version used:
Latest version
Additional context
This could be a bug in Flutter, since this seems due to a feature added in Flutter 3.16. See this Flutter issue for more information.

Hi @juliansteenbakker , did you tried to execute flutter config --enable-native-assets?

It has been confirmed that this issue is caused by some new feature of flutter, which is also using build.dart, colliding with the build.dart from this package. The --enable-native-assets is only available on master, and not on stable release.
See https://github.com/flutter/flutter/issues/138727#issuecomment-1820606396
The solution to this fix as proposed is to make the build.dart file name configurable, so it does not collide. This is also proposed for the other colliding feature of Flutter. See https://github.com/dart-lang/sdk/issues/54111#issue-2003911503

@juliansteenbakker  do you have such problem only with this package?

This is the only package that requires a build.dart file to be in the root of my project, which causes this issue.

@juliansteenbakker  you can also use build.yaml file on freezed or json_serializable packages. Could you please check it?
For example for freezed you can do next:
targets:
  $default:
    builders:
      freezed:
        options:
          union_key: type
          union_value_case: pascal

I don't have such problem on my PC, so can not fix it. Possible it's not our package issue

@juliansteenbakker, the same, 3.16+ version. upgraded today. No errors from my side :(

So, issue should be raised to Flutter maybe. It's not an issue of Swagger Generator. I'm closing the issue.

It happend to me after a flutter clean and flutter pub get

Also, it only happens on iOS or Windows, not on building Android.

It's strange that you have error during app build, and not during code generation... It's very interesting

It's strange that you have error during app build, and not during code generation... It's very interesting. Because package works only when you generating code

This is because a toplevel build.dart is now interpreted as a script which has to be run during Flutter app compilation to compile C code.
We'd like to reserve the toplevel build.dart for that use case:

https://github.com/dart-lang/sdk/issues/54334

Would it be possible to use tool/build.dart for the swagger code generator instead, according to the https://dart.dev/tools/pub/package-layout ?


Library Panel: Fix "Get By Name" endpoint response

Adds the schema fix implemented here: https://github.com/grafana/grafana/pull/78221
I also added comments for each schema modification so that we can remove them when the fixes are released in grafana/grafana

Is it the plan to clean up these 'modify' fixes at some point?

Yes. When 10.3 is released, the current main branch goes into a 10.2 branch and the main client is generated from 10.3 with all the modifys removed


Selecting an option in Autocomplete should not set the display string

https://github.com/flutter/flutter/blob/82b63ff27d7a15581837c84ca9304d3023934545/packages/flutter/lib/src/widgets/autocomplete.dart#L385
I understand the displayStringForOption function to only be for displaying the options, not modifying the option when setting the field value.

Thanks for the report @vsaase
Can you please elaborate by providing current behavior as compared to expected ? A code snippet would also be useful highlighting current behavior.


Choose a monitor?

This may just be a GFE limitation.
However, I use a headless dummy HDMI adapter (1 of 3 monitor identities).
Finding Gamestream only streams to the "1" monitor identity, regardless of what has been marked as "main display".
It even forces it back on when you disconnect it.
Thanks for the awesome tool!

This sounds like a GFE limitation however I'm unsure that I understand. What's the purpose the dummy adapter when you have 2 other displays? Are you unable to swap the ports around so that the dummy is in slot 1?

Ah, I'm probably overcomplicating it. I'll give that a go - thank you.

So also, if you can find a way to solve the problem within a session via command prompt, you could put that command in a .bat script and use that script as one of your background program within this tool and it'll run every session. Maybe something with DisplaySwitch.exe would help per this?

Great thinking! Will certainly keep it in mind.
Looking forward to diving into your tool. Thanks for putting it together.


Revive mpool commands

Summary of changes

Changes introduced in this pull request:

Add back mpool pending with same Lotus flags
Add back mpool stat

You can test them using shell command (most of the time mpool is empty on calibnet):
$ repeat 1000 ./target/release/forest-cli mpool pending && echo "---" && sleep 1
---
---
---
---
---
---
{
  "Message": {
    "Version": 0,
    "To": "t01491",
    "From": "t3sg27lp6xgz3fb7db7t4x4lhmsf3dgu73mj5sodkshh64ftr6dzkrfxrowroon2cr2f3vkumsi4schkpfyvea",
    "Nonce": 14704,
    "Value": "0",
    "GasLimit": 31073678,
    "GasFeeCap": "100507",
    "GasPremium": "99453",
    "Method": 6,
    "Params": "iggZG3DYKlgpAAGC4gOB6AIgRRHJtEnHDx51h/M46ebVUjTD1kowbg+8uWOSrQnQYWwaAAaPXIAaAB45DvQAAAA=",
    "CID": {
      "/": "bafy2bzaceacz2f5k5pcjhzvodhpgin2phycgk2magezaxxp7wcqrjvobbtj5w"
    }
  },
  "Signature": {
    "Type": 2,
    "Data": "hcBY3OATkjMBRly96aViP2CR0R68dqnmlB1k6g2C2EXfe7+AsCN7bF4+M5bA6SecCsP2Fx+NwYkpGBi1CFGon5U9bqilMIiXxuK0mIrNO0d6UocCBGi/IVZwW2K4hT9N"
  },
  "CID": {
    "/": "bafy2bzaceacz2f5k5pcjhzvodhpgin2phycgk2magezaxxp7wcqrjvobbtj5w"
  }
}

$ ./target/release/forest-cli mpool stat
t13phw6spj7uode7n5zm4tr4v5jp3rh2ff3byrcxq: Nonce past: 1, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 63370305
t1u2htzdz77nhhrial6cfdlekcou5cbhtmtbrzoaq: Nonce past: 1, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 42586957
t410fxkwkdni3fjjmweyaarxq2mipbd54p4e5vzaeygq: Nonce past: 1, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 7803847
-----
total: Nonce past: 3, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 113761109

Reference issue to close (if applicable)

Closes #2728
Other information and links

Change checklist

[ ] I have performed a self-review of my own code,
[ ] I have made corresponding changes to the documentation,
[ ] I have added tests that prove my fix is effective or that my feature works
(if possible),
[ ] I have made sure the
CHANGELOG is
up-to-date. All user-facing changes should be reflected in this document.

@elmattic is this ready for review? The Change checklist indicates it's not but the green Open suggests it is. :)

@elmattic is this ready for review? The Change checklist indicates it's not but the green Open suggests it is. :)

Putting it back to draft. I think this PR would still needs:

Proper documentation (mpool stat output is a little obscure)
Testing
Code coverage

Maybe we can skip the testing part.

Note: This PR will be abandoned and replaced by smaller PRs more easy to review (first is #3020).

Closing it for aforementioned reason.


Clarification for similarity measures

Dear Authors,
It was exciting to read about your methods and results for molecule generation applications! I have a few questions below and hope that you can provide more details.
In the article, you mentioned that the top 1000 molecules' mean pairwise Tanimoto similarities were calculated.

Could you please specify the type of fingerprint(s) that are used for the Tanimoto similarity calculation?

In addition, does the mean pairwise similarity mean taking each molecule in the set and calculating its similarity to all the other molecules in the set, repeating this procedure for all the other molecules then taking the mean of all the means?

I wasn't able to locate any code/comments relating to this part specifically.  I would greatly appreciate it if you can point me to the codes as well!
Regards,
Haote

Hi Haote,

We used the default fingerprinting available in RDKit, Chem.RDKFingerprint, although you should get similar numbers using the standard ECFP4 fingerprints.
Yes that is correct, this can be achieved efficiently using rdkit.DataStructs.BulkTanimotoSimilarity.

We indeed did not upload the analysis scripts, they were embedded in a mess of notebooks and we did not take the time to extract them out. Let us know if you have additional questions.

Thank you very much! That resolves all my questions!


Implement workaround for pasting html content in IE11

Reopen after an unintended merged on #3

@kevindew as per our offline review I:

rebased and restructure the code to place the IE11 related function in a separate file
re-tested with supported browsers

Now the thing is, I don't see another way to test the code in html-from-hidden-element without exporting those functions.
Feel free to push commits on this branch if you see an option.

@alex-ju sounds good, I've just put a test on it.


new(proposals): driver kernel testing framework

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area API-version

/area build

/area CI

/area driver-kmod

/area driver-bpf

/area driver-modern-bpf

/area libscap-engine-bpf

/area libscap-engine-gvisor

/area libscap-engine-kmod

/area libscap-engine-modern-bpf

/area libscap-engine-nodriver

/area libscap-engine-noop

/area libscap-engine-source-plugin

/area libscap-engine-savefile

/area libscap-engine-udig

/area libscap

/area libpman

/area libsinsp

/area tests

/area proposals

Does this PR require a change in the driver versions?

/version driver-API-version-major

/version driver-API-version-minor

/version driver-API-version-patch

/version driver-SCHEMA-version-major

/version driver-SCHEMA-version-minor

/version driver-SCHEMA-version-patch

What this PR does / why we need it:
Formalize a driver kernel testing framework.
@falcosecurity/libs-maintainers
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

NONE

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: incertum
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [incertum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/milestone 0.12.0

Love this proposal, @incertum, thank you for this great work!

Love this!  Thank you so much for pulling this together!

Love this! Thank you so much for pulling this together!

Thank you @TheFoxAtWork!

@falcosecurity/libs-maintainers I have moved this PR out of WIP, it is now ready for final review. Thanks in advance!

A minor question: are we talking about adding nightly tests? I think we do not want to run these tests on every PR/master change. Perhaps weekly tests can be enough too. Thank you again btw!

where I mentioned it to be "on demand" initially I added "(such as nightly tests)"

What needs to happen for us to move forward with this?

What needs to happen for us to move forward with this?

@TheFoxAtWork as a first step as suggested we created a new dedicated CNCF Service Desk ticket https://cncfservicedesk.atlassian.net/servicedesk/customer/portal/1/CNCFSD-1822. Thank you!

LGTM label has been added.  Git tree hash: d98ddee4f8accbc223b8c4c0b7eaef6d92b873e9

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: incertum, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [incertum,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, incertum, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,incertum,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/unhold


Disable the static token kubeconfig in the shoot template used for TM integration tests

How to categorize this PR?

/area testing
/kind cleanup
What this PR does / why we need it:
In this way we let the gardener-apiserver default the .spec.kubernetes.enableStaticTokenKubeconfig field. For Shoots with K8s < 1.26, it will be defaulted to true, for Shoots with K8s >= 1.26 - to false. The plan is to lock this field to false for Shoots with K8s >= 1.27.
Integration tests in gardener/gardener should be already adapted to do not rely on the static token kubeconfig. Integration tests outside of gardener/gardener should use the kubeconfig provided under $TM_KUBECONFIG_PATH/shoot.config or should obtain kubeconfig from the shoot/adminkubeconfig subresource.
Which issue(s) this PR fixes:
Part of https://github.com/gardener/gardener/issues/6773
Special notes for your reviewer:

~✅ This PR depends on https://github.com/gardener/gardener/pull/7495, hence it is in draft state.~

Release note:

The Shoot creation integration test does no longer sets `.spec.kubernetes.enableStaticTokenKubeconfig=true` in the default Shoot template. Instead, the gardener-apiserver will default the field.

/retest-required

/retest-required


Publish doc updates.

Docs Build status updates of commit aa7369d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/design/using-office-ui-fabric-react.md
:white_check_mark:Succeeded

docs/develop/convert-javascript-to-typescript.md
:white_check_mark:Succeeded

docs/excel/custom-functions-architecture.md
:white_check_mark:Succeeded

docs/excel/custom-functions-authentication.md
:white_check_mark:Succeeded

docs/excel/custom-functions-batching.md
:white_check_mark:Succeeded

docs/excel/custom-functions-best-practices.md
:white_check_mark:Succeeded

docs/excel/custom-functions-debugging.md
:white_check_mark:Succeeded

docs/excel/custom-functions-errors.md
:white_check_mark:Succeeded

docs/excel/custom-functions-json-autogeneration.md
:white_check_mark:Succeeded

docs/excel/custom-functions-json.md
:white_check_mark:Succeeded

docs/excel/custom-functions-localize.md
:white_check_mark:Succeeded

docs/excel/custom-functions-naming.md
:white_check_mark:Succeeded

docs/excel/custom-functions-overview.md
:white_check_mark:Succeeded

docs/excel/custom-functions-parameter-options.md
:white_check_mark:Succeeded

docs/excel/custom-functions-requirement-sets.md
:white_check_mark:Succeeded

docs/excel/custom-functions-requirements.md
:white_check_mark:Succeeded

docs/excel/custom-functions-runtime.md
:white_check_mark:Succeeded

docs/excel/custom-functions-save-state.md
:white_check_mark:Succeeded

docs/excel/custom-functions-troubleshooting.md
:white_check_mark:Succeeded

docs/excel/custom-functions-volatile.md
:white_check_mark:Succeeded

docs/excel/custom-functions-web-reqs.md
:white_check_mark:Succeeded

docs/excel/excel-add-ins-advanced-concepts.md
:white_check_mark:Succeeded

docs/excel/excel-add-ins-charts.md
:white_check_mark:Succeeded

docs/excel/excel-add-ins-shapes.md
:white_check_mark:Succeeded

docs/excel/excel-add-ins-tables.md
:white_check_mark:Succeeded

docs/excel/make-custom-functions-compatible-with-xll-udf.md
:white_check_mark:Succeeded

docs/images/operands.png
:white_check_mark:Succeeded

docs/images/word-task-pane-with-react-component.png
:white_check_mark:Succeeded

docs/images/word-task-pane-yo-default.png
:white_check_mark:Succeeded

docs/images/yo-office-word-react.png
:white_check_mark:Succeeded

docs/includes/excel-custom-functions-note.md
:white_check_mark:Succeeded

docs/includes/file-get-the-whole-document-from-an-add-in-for-powerpoint-or-word.md
:white_check_mark:Succeeded

docs/includes/quickstart-choose-editor.md
:white_check_mark:Succeeded

docs/includes/quickstart-yo-start-server-excel.md
:white_check_mark:Succeeded

docs/includes/using-excel-preview-apis.md
:white_check_mark:Succeeded

docs/overview/explore-with-script-lab.md
:white_check_mark:Succeeded

docs/overview/office-add-in-availability.md
:white_check_mark:Succeeded

docs/overview/schemas/content/OfficeAppBasicTypesV1_0.xsd
:white_check_mark:Succeeded

docs/overview/schemas/content/OfficeAppManifestV1_1.xsd
:white_check_mark:Succeeded

docs/overview/schemas/mail/OfficeAppBasicTypesV1_0.xsd
:white_check_mark:Succeeded

docs/overview/schemas/mail/OfficeAppManifestV1_1.xsd
:white_check_mark:Succeeded

docs/overview/schemas/taskpane/OfficeAppBasicTypesV1_0.xsd
:white_check_mark:Succeeded

docs/overview/schemas/taskpane/OfficeAppManifestV1_1.xsd
:white_check_mark:Succeeded

docs/quickstarts/excel-custom-functions-quickstart.md
:white_check_mark:Succeeded

docs/quickstarts/excel-quickstart-angular.md
:white_check_mark:Succeeded

docs/quickstarts/excel-quickstart-jquery.md
:white_check_mark:Succeeded

docs/quickstarts/excel-quickstart-react.md
:white_check_mark:Succeeded

docs/quickstarts/onenote-quickstart.md
:white_check_mark:Succeeded

docs/quickstarts/powerpoint-quickstart.md
:white_check_mark:Succeeded

docs/quickstarts/project-quickstart.md
:white_check_mark:Succeeded

This comment lists only the first 50 files in the pull request.
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Codicon: add an icon that resembles the VS Code logo

For the new welcomeBanner support in VSCode web I need a Codicon that is the VSCode logo (doesn't really need to be different for insiders vs stable I would say). I couldn't find it in our set currently.
Thanks!
//cc @bamurtaugh @digitarald

Is there a reason why we can't use the svgs that get swapped in our builds? If it's a codicon then it won't have the same styling as our logo since it has the gradients/shadows.

That would require support for icons in the welcome banner. @lszomoru I think created that workbench piece. Currently it only allows for codicons.

I think I'd be more in favor of doing that since the logo has complex gradients and it would look better than a codicon with a single color that we don't often use.

Ok then I think we can close this.

Do we need a separate issue for the banner to support svgs?

https://github.com/microsoft/vscode/issues/131977


Implement a push notification bouncer service for Android

The Google Cloud Messaging platform for Android push notifications does not support multiple mutually untrusted servers sending push notifications to the same app (in particular, the server has to have a certain secret key associated with the app).
In order to support push notifications for arbitrary Zulip servers, we need to build (or deploy) a push notification bouncer service that can redirect push notifications from registered Zulip servers to the relevant Android clients.
https://github.com/mattermost/push-proxy might solve this problem?  Needs testing.

https://github.com/zulip/zulip/pull/3092 is partial work towards implementing this.

This was just completed!  Huge thanks to @umairwaheed for finishing this effort.


Rehabilitate train from_json

It is very hard to retrain a job generated by the random search without this tool.
It should be rehabilitated or replaced by something else for this purpose.

Actually, we could also allow --config_file to be either a TOML file or a JSON file.
What do you think @ravih18?


Call the prepare method

Otherwise html responses may get incorrectly rendered as text due to the missing content-type header. Indeed, I have seen this issue. ;)

Doesn't the Illuminate\Routing\Router do this?

No. I tried this with a fresh Laravel app, and 404 pages didn't have their HTML content type set. After adding this code, it was correctly set.

Okey. I've never tested with 404 pages..
Thank you

:shipit:


Flask Session not working after latest itsdangerous package upgrade

Hello,
Today I had a problem while deploying a project which is based on flask and Flask-Session.
I received this error at server startup:
  File "/opt/gl/sso/env/lib/python3.6/site-packages/flask_session/__init__.py", line 16, in <module>
    from .sessions import NullSessionInterface, RedisSessionInterface, \
  File "/opt/gl/sso/env/lib/python3.6/site-packages/flask_session/sessions.py", line 23, in <module>
    from itsdangerous import Signer, BadSignature, want_bytes
ImportError: cannot import name 'want_bytes'

Digging on this issue I found that the package itsdangerous released version 1.0.0 which it has no want_bytes function anymore.
Forcing my requirements.txt with itsdangerous>=0.24,<1.0.0 solved the problem but I think you should refactor the package since itsdangerous is a main dependency of flask.

:+1:

The file structure of the itsdangerous package has changed. The function resides in the encoding.py file. Changing the import statements from:
from itsdangerous import Signer, BadSignature, want_bytes
to:
from itsdangerous import Signer, BadSignature
from itsdangerous.encoding import want_bytes
works for me.

itsdangerous 1.1.0 was just released and contains a fix for this. Flask-Session should still be fixed so it doesn't rely on this, but it won't fail for now.


Text and Resource codecs should be configured explicitly

To avoid a feature sometimes perceived by users as a bug, see SPR-17080 and SPR-15842.
These preconfigured codecs should be replaced by WebFluxCodecsModule member functions.

Should the same change also be mirrored in the WebFluxClientModule?

I started with some refactoring see https://github.com/spring-projects/spring-fu/pull/92

@matoatoa Thanks for your help. I chose to create by default String and Resource codecs when no codecs { } block is defined, and configure no default codecs when there is ssuch block defined. Is it fine for you?
@ciferkey I did the same for the client and server


LFPW datasets

Hello,
It is not clearly a bug, neither a missing feature so i did not know where to post this.
I used deeplake library to download LFPW dataset and it works well.
This dataset is composed of images and keypoints for each image. However, in the official website of this dataset (https://neerajkumar.org/databases/lfpw/), there are 105 keypoints (starting with 'left_eyebrow_out_x', 'left_eyebrow_out_y', ...)
When i download this dataset with deeplake, i have 204 keypoints per data without the explanation of each keypoint. So my question is what are those 204 keypoints ?
--
The code is very simple:
import deeplake
ds = deeplake.load("hub://activeloop/LFPW-test")
for data in ds:
keypoints = data['keypoints'].numpy().flatten # Array of 204 int values, instead of 105
--
Thanks in advance for your help

Hey @Lunnor, thanks for raising the issue. It looks like there are mistakes in the upload of this dataset. Let us re-upload the dataset correctly and get back to you.

Thanks a lot for your quick answer.

@Lunnor I have reuploaded the datasets at hub://activeloop/LFPW-train and hub://activeloop/LFPW-test. Note that many of the links in the dataset are now obsolete. Please check out the datasets and let me know if it works for you.

I tried again the aboved code.
Instead of loading the datasets with labels of 204 keypoints, it now raises an error at the line
ds = deeplake.load("hub://activeloop/LFPW-test")
Error:
deeplake.util.exceptions.UserNotLoggedInException: Please log in through the CLI in order to load this dataset, or create an API token in the UI and pass it to this method using the ‘token’ parameter. The CLI commands are ‘activeloop login’ and ‘activeloop register’.
--
I created an API token and gave it:
ds = deeplake.load("hub://activeloop/LFPW-test", token="<token_value>")
I received another error:
deeplake.util.exceptions.TokenPermissionError: A dataset does not exist at the specified path, or you do not have sufficient permissions to load or create one. Please check the dataset path and make sure that you have sufficient permissions to the path.

I tried again the aboved code.
Instead of loading the datasets with labels of 204 keypoints, it now raises an error at the line
ds = deeplake.load("hub://activeloop/LFPW-test")
-- Error:
deeplake.util.exceptions.UserNotLoggedInException: Please log in through the CLI in order to load this dataset, or create an API token in the UI and pass it to this method using the ‘token’ parameter. The CLI commands are ‘activeloop login’ and ‘activeloop register’.
--
I created an API token and gave it:
ds = deeplake.load("hub://activeloop/LFPW-test", token="<token_value>")
I received another error:
deeplake.util.exceptions.TokenPermissionError: A dataset does not exist at the specified path, or you do not have sufficient permissions to load or create one. Please check the dataset path and make sure that you have sufficient permissions to the path.

Hi @Lunnor can you please try to load the dataset with the command above
ds = deeplake.load("hub://activeloop/LFPW-test", token="<token_value>", read_only=True)

No, I still have: deeplake.util.exceptions.TokenPermissionError. As if my token was incorrect.
I created my account in https://app.activeloop.ai/ with my github profile
I created an API token in https://app.activeloop.ai/profile/lunnor/apitoken
Then i copy paste that token in a string.
import deeplake
ds = deeplake.load("hub://activeloop/LFPW-test", token="aTokenContainingLettersNumbersAndSymbols", read_only=True)

Hey @Lunnor Apologies for the issue, we forgot to make the dataset public earlier, which is now fixed. Can you pls try to load it again, no need to pass token.

Ok so as the dataset was only composed of the url of the images, many of them are now missing.
I managed to download

353 images from LFPW-train
96 images from LFPW-test

Some of them are full dark, or another image indicating that the original image is no longer available.
Anyway, thanks for your support.

Hey @Lunnor, what about the following datasets:

Celeba Dataset
300w Dataset
WFLW dataset
LFW Funneled Dataset

I'm sorry you've run into this issue with the LFPW dataset. The original dataset's links are down, so sadly, we cannot really do much about it.

Thanks for your message, i will look into these datasets.
I am looking for a dataset indicating if the person is wearing glasses, has an eye hidden, smilling or the face is not looking at the camera. Ideally a dataset with a label indicating if the photo would be valid for an ID card.

@Lunnor , CelebA should fit the bill then, it has a tensor named eyeglasses.  To the best of my knowledge we don't have a dataset with labels indicating if the photo would be valid for an ID card.

closing this thread as the issue at hand was addressed.


Improve benchmark settings to speed up benchmarking

*More detailed description of your change,
This PR makes the ACL Authorizer significantly faster.
// before
Benchmark                                        (aclCount)  (authorizerType)  (denyPercentage)  (resourceCount)  Mode  Cnt    Score     Error  Units
AuthorizerBenchmark.testAclsIterator                    500               ACL                50             5000  avgt    7  278.920 ±  28.651  ms/op
AuthorizerBenchmark.testAuthorizeByResourceType         500               ACL                50             5000  avgt    7    4.614 ±   1.076  ms/op
AuthorizerBenchmark.testAuthorizer                      500               ACL                50             5000  avgt    7    0.185 ±   0.045  ms/op
AuthorizerBenchmark.testUpdateCache                     500               ACL                50             5000  avgt    7  641.337 ± 123.786  ms/op

// after
Benchmark                                        (aclCount)  (authorizerType)  (denyPercentage)  (resourceCount)  Mode  Cnt    Score    Error  Units
AuthorizerBenchmark.testAclsIterator                    500               ACL                50             5000  avgt    7  243.929 ± 32.787  ms/op
AuthorizerBenchmark.testAuthorizeByResourceType         500               ACL                50             5000  avgt    7    3.833 ±  0.375  ms/op
AuthorizerBenchmark.testAuthorizer                      500               ACL                50             5000  avgt    7    0.033 ±  0.002  ms/op
AuthorizerBenchmark.testUpdateCache                     500               ACL                50             5000  avgt    7  595.364 ± 25.375  ms/op
JMH benchmarks done

*Summary of testing strategy (including rationale)
Since the code is equivalent the no extra tests should be added.
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

Thanks for the PR. There are a number of changes in this PR - which ones actually made a difference to the numbers? Some of them are a bit questionable.

Thanks for the PR. There are a number of changes in this PR - which ones actually made a difference to the numbers? Some of them are a bit questionable.

The main performance boost was from the A.hashcode() != B.hashcode() check.
Then the acl change to an array significantly improved the performance since it avoided boolean boxing/unboxing and the nested Seq iterator.
Then the Set change in the AclAuthorizer.scala which was called often in the inner loop.

Then the performance became harder to gain and this results in a bit more questionable optimizations. Which actually came out of the benchmarks.

enum comparison speed had a memory alignment issue and comparing by code was just faster in general.
improving the string compare performance for USER_TYPE by sharing the same reference. this might not have been that significant.

I only committed changes which actually resulted in performance improvements. I tried to change the enum to a 'flag' type with the values 1, 2, 8, 16, 32 but this didn't make any difference.