CIP-0??? | Plutus v1 compatible script references

Plutus v1 scripts are unable to use the CIP-0031 / CIP-0033 reference inputs / script references to reduce transaction size. As a result, between 80% and 90% of every block is occupied by repeating the same script bytes over and over.
This problem will not go away even as new protocols that do support Plutus v2 are released: these protocols exist forever, and in many cases, will hold hundreds of thousands in liquidity practically forever.  As long as that is the case, they will represent an arbitrage opportunity that someone somewhere will want to take advantage of.
This proposal suggests a very simple (albeit perhaps aesthetically uncouth), backwards compatible way to support script references for Plutus v1 scripts.
Rendered

Loving the intent here, but not sure if it remains optional if it's still a viable solution as a malicious user can always deploy and use/abuse a "traditional" Plutus v1 script to cause chain congestion and bloat the ledger size (although granted there is a cost to doing so and would be similar to spamming transactions with 15kb of gibberish in the metadata).
Obviously I don't know the impact this would have to the core node and ledger teams to make it all work, but I do agree that it is absolutely critical to solve this one and respect you bringing it up here as a topic of discussion. Hopefully we can hear from some of the core architects that can chime in on potential solutions.

Loving the intent here, but not sure if it remains optional if it's still a viable solution as a malicious user can always deploy and use/abuse a "traditional" Plutus v1 script to cause chain congestion and bloat the ledger size (although granted there is a cost to doing so and would be similar to spamming transactions with 15kb of gibberish in the metadata).

The difference is that v1 versions of the protocol have a financial incentive to them that will create a very long tail of people swapping against them.

Obviously I don't know the impact this would have to the core node and ledger teams to make it all work, but I do agree that it is absolutely critical to solve this one and respect you bringing it up here as a topic of discussion. Hopefully we can hear from some of the core architects that can chime in on potential solutions.

Yes, of course if there is a concrete reason not to do this, we'll have to look for alternatives, but I've never been able to get an answer besides "someone somewhere said it would be dangerous" and "we wanted to apply a policy about the script context consistently"

I also like the fact this CIP makes an effort to separate referenced scripts as inputs into the witness.
I've found it bizarre that the place we reference inputs values and datums is also where we provide script refs that essentially act as witnesses. The script ref inputs almost always take up extra space in reference inputs and are ignored by plutus execution.

@WhatisRT A birdie told me you in particular might be interested in this, and might be able to help me navigate the relevant updates to the ledger spec :D

Our old friend @JaredCorduan helped me track down (roughly) where in the code this change would be made (at least for babbage):
https://github.com/IntersectMBO/cardano-ledger/blob/c1b7f597f750e7684ca73f96943820019b23ef17/eras/babbage/impl/src/Cardano/Ledger/Babbage/UTxO.hs#L137-L141
with the conway implementation currently just forwarding to the babbage implementation:
https://github.com/IntersectMBO/cardano-ledger/blob/c1b7f597f750e7684ca73f96943820019b23ef17/eras/conway/impl/src/Cardano/Ledger/Conway/UTxO.hs#L86C29-L86C29

For brainstorming purposes: I wonder how this would be costed?
By introducing a way to reference from the witness set, node have to do extra calculation of retrieving the data.
Now for reference inputs, you pay for this, but you also gain that the full UTxO becomes available (so the address, datum, and value). Would your proposed solution be costed the same? And in both cases, how would this possible separation of cases impact the code?

I have a bunch of things to say about this:

There is no big difference between this proposal and simply allowing reference inputs to be used in transactions that use V1 scripts and not showing them. Those two options only behave differently in transactions that use both V1 and V2 scripts. Allowing reference inputs in transactions with V1 scripts is trivial, and doesn't even require CDDL changes, so it can be done with an intra-era HF.
We have considered this when introducing Babbage, and decided against it. Here are some facts:

You can make a really stupid V1 script that can tell if it's being run via reference input: make a pretty large script, and then check if the fees are 'too low'.
This means we cannot guarantee that script behaviour of V1 scripts will be exactly the same with this feature as before. Note: I believe it's unlikely that there is a script out there that would actually behave differently in a meaningful way, but there certainly could be.

This means that introducing such a feature carries a certain risk of breaking existing scripts. At the time, we weren't willing to take that risk, and I stand by that decision. If however the entire Cardano community decided that they would be willing to take that risk I see no issue with implementing such a feature.
To address some other questions:

@Quantumplation txwitscripts are the scripts that are included in txwits, while txscripts is defined in that figure. It looks like the Babbage spec is missing a note that txwitscripts is what used to be called txscripts in earlier specs.
@perturbing It wouldn't be costed any different than costing reference inputs. We have recently discussed that in a different context and there will be some small changes to the fee structure because of that.

I have a bunch of things to say about this:

There is no big difference between this proposal and simply allowing reference inputs to be used in transactions that use V1 scripts and not showing them. Those two options only behave differently in transactions that use both V1 and V2 scripts. Allowing reference inputs in transactions with V1 scripts is trivial, and doesn't even require CDDL changes, so it can be done with an intra-era HF.

I'd be totally fine with this, but my understanding was that we felt materially worse about "hiding" reference inputs, which may also include reference datums etc. Including them in the witness set was a way to morally satisfy the policy of not hiding things in the transaction body from the script context.
Personally, I feel like having them in the witness set is a lot cleaner; they're being included for the purpose of witnessing which script needs to be run. And to @MicroProofs's point above it has the extra benefit of allowing additional optimizations: when my plutus v2 script is scanning through the reference inputs looking for a particular datum, it isn't tripping over the script references.
That being said, I'm 100% ok with a tweak to this CIP that just allows reference inputs on a v1 transaction and is even simpler to adopt.

We have considered this when introducing Babbage, and decided against it. Here are some facts:

You can make a really stupid V1 script that can tell if it's being run via reference input: make a pretty large script, and then check if the fees are 'too low'.

How would this work in practice? The plutus v1 script has access to the fee paid, but not the minimum fee; and it doesn't have access to the witness set, so it can't calculate what the fee "should have been"; so the only way I can think to achieve this would be a script with some hard-coded heuristic about what the fee should be.  Something like if feePaid < 13000000 + 12000 * length inputs.

This means we cannot guarantee that script behaviour of V1 scripts will be exactly the same with this feature as before. Note: I believe it's unlikely that there is a script out there that would actually behave differently in a meaningful way, but there certainly could be.

This means that introducing such a feature carries a certain risk of breaking existing scripts. At the time, we weren't willing to take that risk, and I stand by that decision. If however the entire Cardano community decided that they would be willing to take that risk I see no issue with implementing such a feature.

To think through the possible outcomes here, there are only three possible outcomes to this change: a script that should be spendable becomes unspendable, and a script that should be unspendable becomes spendable, and a script maintains the same behavior, but with different execution units.
The first one isn't really a concern: if for some bizarre reason this was the case, users could continue to provide the script in the witness, and the semantics would remain unchanged, and they could continue to spend the UTXO like normal. Obviously not ideal, since the point is to reduce tx size, but also not impacting the vast majority of scripts (if any at all), so there's still a huge benefit to this change.
The second one is worth more careful thought, as releasing funds that should stay locked is bad... but I legitimately can't come up with an even semi-reasonable construct where this would happen. Someone would have to be checking explicitly if the fee was lower than some threshold, and using that to execute a different code path. Like you said, certainly possible, but it seems like it'd almost have to be designed with this change in mind :sweat_smile:
The third one is not really worth worrying about. The outcome is the same, but the fee changes slightly.
If we're really worried about this, a messy solution would be to add the size of any plutusv1 scripts to the size of the transaction, regardless of where they're sourced from. The reduction in fees from saved space is way less important than the actual reduction in space.

To address some other questions:

@Quantumplation txwitscripts are the scripts that are included in txwits, while txscripts is defined in that figure. It looks like the Babbage spec is missing a note that txwitscripts is what used to be called txscripts in earlier specs.

Ok, that's what I thought, thanks :)
Overally, it's hard to express how valuable some change like this would be (either the version I suggested, or the version you suggested), so I appreciate you having an open mind and talking through the different implications! :)

Why not just change the on disk storage so it does a by-hash lookup. The bloat on the wire doesn't really matter then. Also does not require a hard fork.
We might see the same scripts in every block, but save them to disk once.

I think that this cip should also be implemented, I just think the storage fix is a quick win.

Why not just change the on disk storage so it does a by-hash lookup. The bloat on the wire doesn't really matter then. Also does not require a hard fork.
We might see the same scripts in every block, but save them to disk once.

I think that's an orthogonal and less urgent concern.
60gb of disk space a year is unpleasant, but tolerable. Certainly not the disk-guzzler that other chains are.
The reason it's so painful is because of the wire bloat. The block propagation times are super sensitive to block size because of the data sent over the wire.
So the point is to cut out the wire bloat. It's equivalent to raising our block size to 125kb without impacting propagation times, or something like that.
Also, the change to the on-disk format may not actually be easier. IIUC the immutable db stuff is some pretty gnarly haskell code, and I could see this ultimately being a huge change.
Ultimately I think that would be worthwhile as well, but the bulk of the benefit, and the minority of the effort, surprisingly.

I think:

The policy of not omitting information when constructing the script context is a good one, and the risks from violating it are real, and not the sort of thing we should generally be doing with a system like this that promises very strong backwards compatibility.
But the benefits in this case are very high.

So overall I think we should do it. Time has shown that we were wrong about how big a deal this would be.

For sure! If I'm being honest, I think I have a bit of (hopefully understandable) frustration born of dealing with the impacts of this over the last year plus, and that frustration bleeds a little into the CIP :sweat_smile:. But I've also always tried to say that I wasn't in the hot-seat and ultimately responsible for the decision, so there's a very big "benefit of the doubt" blanket that I apply. It's frustration with the state of affairs, not frustration with you or anyone on the team personally.
I agree that having a very strong policy of backwards compatibility is good, sensible, and even essential. I agree that omitting things from the transaction definition can have very real risks, and should be weighed carefully, and that the burden of justification is heavily on the proposer. So long as it doesn't turn into a blanket policy, applied unilaterally regardless of the magnitude of the real world benefits or the slimness of the risks on a case by case basis, then I'm happy.
With regards to the simpler approach you and WhatIsRT suggest, I'd be thrilled with that approach. My only reason for proposing this one is because I thought it was off the table, given the previous commitment to the aforementioned policy and discussions we've had in the past.
I'll make a new revision of this CIP today with the suggested updates.

@michaelpj let me know if the latest draft is sufficient; it includes the historical note and a reworked rationale section that outlines the risks as well as the benefits. The specification section is pretty bare, as I'm not familiar with the exact details (ex: which parts of the spec would need to be updated, etc.), and I could use some recommendations on the "Implementation Plan" if you have a moment!

@Quantumplation Here is a relevant Ledger ticket, which contains some interesting analysis: https://github.com/IntersectMBO/cardano-ledger/issues/3965

which parts of the spec would need to be updated

It would have to be Agda spec for Conway that would need to be updated. @WhatisRT is the person that can answer this. From what I know Alonzo related functionality has been merged recently, so it might be in a state that we can actually implement it in the spec. Although since Babbage functionality (that introduced reference scripts) is still not implemented we might not be able to reflect it in the spec yet.

Implementation Plan

Implementation for this is surprisingly simple. All we need to do is enable Reference scripts for PlutusV1. For example, this is the only thing that is preventing reference scripts being used in Babbage for PlutusV1:
  when (isSJust (txOut ^. referenceScriptTxOutL)) $
    Left $
      inject $
        ReferenceScriptsNotSupported txOutSource

https://github.com/IntersectMBO/cardano-ledger/blob/fdc366df654fc02b1668012342732d41eaa099fe/eras/babbage/impl/src/Cardano/Ledger/Babbage/TxInfo.hs#L94-L97

@rphair Thanks! Sorry, I didn't realize the meeting was yesterday or I would have tried to attend heh

As per this comment by @lehins, implementation of this CIP has been officially added to the Conway hard-fork backlog!
https://github.com/IntersectMBO/cardano-ledger/issues/3965#issuecomment-1908212420


Path string parsing, needs /

If you try to use plot or enjoy with a directory filepath that doesn't end in /, you get the following error.
# command
python -m bevodevo.enjoy -n BackAndForthEnv-v0 -f results/GeneticPopulation_MLPBodyPolicy_p64_g100_s13_w3 -pi MLPBodyPolicy -a 1 -e 1 -s 1

# output
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "bevodevo/bevodevo/enjoy.py", line 227, in <module>
    enjoy(args)
  File "bevodevo/bevodevo/enjoy.py", line 35, in enjoy
    my_dir = os.listdir(argv.file_path)
FileNotFoundError: [Errno 2] No such file or directory: 'results/GeneticPopulation_MLPBodyPolicy_p64_g100_s13_w3'

Fix added in PR #14


I got error like Unsupported operation while dowloading small pdf file (below 1 mb).

I got error like Unsupported operation while dowloading small pdf file (below 1 mb).

Can you provide the link to the file please?

I tried both http and https link.
https://www.projectmadurai.org/pm_etexts/pdf/pm0001.pdf
http://www.projectmadurai.org/pm_etexts/pdf/pm0001.pdf

Can you post the error form console also.
also maybe the block of code that's throwing the error?
Thanks

I/flutter ( 8509): /storage/emulated/0/நூல்கள்/திருக்குறள்.pdf
════════ Exception caught by widgets library ═══════════════════════════════════════════════════════
The following UnsupportedError was thrown building DownloadAlert(dirty, dependencies: [_InheritedTheme, _LocalizationsScope-[GlobalKey#1606c]], state: _DownloadAlertState#5f8c9):
Unsupported operation: Infinity or NaN toInt
The relevant error-causing widget was:
DownloadAlert file:///C:/Users/Kabilan/AndroidStudioProjects/books/lib/views/details.dart:354:29
When the exception was thrown, this was the stack:
#0      double.toInt (dart:core-patch/double.dart:193:36)
#1      double.floor (dart:core-patch/double.dart:163:34)
#2      Constants.formatBytes (package:books/util/consts.dart:75:35)
#3      _DownloadAlertState.build (package:books/components/download_alert.dart:106:41)
#4      StatefulElement.build (package:flutter/src/widgets/framework.dart:4619:28)
...
════════════════════════════════════════════════════════════════════════════════════════════════════

did you tried downlod small pdf file (below 1 mb) file?

@balgithhub The app was made for ePub files. So i didn't download any pdf files at all

Where are you getting the pdf files from?
did you modify the app?
A little information will help.

no i downloded lot of pdf files using with your app, but those are above 10 MB

I am not modifying anything on app level, on API file i used pdf file name

just chenged the pdf viewer instead of epub kitty

Okay, now that's some useful information.

I'll look into the problem and give you a response.
Thanks!

I changed these two files only
download.dart
onTap: () async{
String path = dl['path'];
Navigator.push(context,
MaterialPageRoute(builder: (context) => PDFScreen(path)),
);
//pageChannel.receiveBroadcastStream().listen((Object event) {
//print('page:$event');
//}, onError: null);
},
Deatails.dart
startDownload(BuildContext context, String url, String filename) async{
Directory appDocDir = Platform.isAndroid
? await getExternalStorageDirectory()
: await getApplicationSupportDirectory();
if(Platform.isAndroid){
Directory(appDocDir.path.split("Android")[0]+"${Constants.appName}").create();
}
String path = Platform.isIOS
    ? appDocDir.path+"/$filename.pdf"
    : appDocDir.path.split("Android")[0]+"${Constants.appName}/$filename.pdf";
print(path);
File file = File(path);
if(!await file.exists()){
  await file.create();
}else{
  await file.delete();
  await file.create();
}

@balgithhub I tried using this https://file-examples.com/wp-content/uploads/2017/10/file-example_PDF_500_kB.pdf. It's below (1 mb) and it works fine but your pdf didnt work
I'm still looking into the problem but if you have any other helpful information, it'll be appreciated

For me also its working fine https://file-examples.com/wp-content/uploads/2017/10/file-example_PDF_500_kB.pdf.
Could you please try below 200KB file?

i tried below 200kb also.
https://file-examples.com/wp-content/uploads/2017/10/file-sample_150kB.pdf
and it worked

For me also its working fine https://file-examples.com/wp-content/uploads/2017/10/file-sample_150kB.pdf. So size is not a problem. Something wrong on website access.
http://www.projectmadurai.org/pm_etexts/pdf/pm0001.pdf

Maybe you should look into that and tell me if you find something.
Thanks

What is this error?
E/flutter ( 8509): [ERROR:flutter/lib/ui/ui_dart_state.cc(157)] Unhandled Exception: DioError [DioErrorType.DEFAULT]: SocketException: Failed host lookup: 'catalog.projectmadurai.org' (OS Error: No address associated with hostname, errno = 7)
E/flutter ( 8509): #0      DioMixin._dispatchRequest (package:dio/src/dio.dart:963:7)
E/flutter ( 8509):
E/flutter ( 8509): #1      DioMixin._request._interceptorWrapper... (package:dio/src/dio.dart:849:37)
E/flutter ( 8509): #2      DioMixin.checkIfNeedEnqueue (package:dio/src/dio.dart:1117:22)
E/flutter ( 8509): #3      DioMixin._request._interceptorWrapper.. (package:dio/src/dio.dart:846:22)
E/flutter ( 8509): #4      new Future. (dart:async/future.dart:176:37)
E/flutter ( 8509): #5      _rootRun (dart:async/zone.dart:1180:38)
E/flutter ( 8509): #6      _CustomZone.run (dart:async/zone.dart:1077:19)
E/flutter ( 8509): #7      _CustomZone.runGuarded (dart:async/zone.dart:979:7)
E/flutter ( 8509): #8      _CustomZone.bindCallbackGuarded. (dart:async/zone.dart:1019:23)
E/flutter ( 8509): #9      _rootRun (dart:async/zone.dart:1184:13)
E/flutter ( 8509): #10     _CustomZone.run (dart:async/zone.dart:1077:19)
E/flutter ( 8509): #11     _CustomZone.bindCallback. (dart:async/zone.dart:1003:23)
E/flutter ( 8509): #12     Timer._createTimer. (dart:async-patch/timer_patch.dart:23:15)
E/flutter ( 8509): #13     _Timer._runTimers (dart:isolate-patch/timer_impl.dart:398:19)
E/flutter ( 8509): #14     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:429:5)
E/flutter ( 8509): #15     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)
E/flutter ( 8509):

The address (catalog.projectmadurai.org) can't be reach maybe because of internet issues or your server is down

No website is fine. I used your API file, just rename the site name.

Can you post the link to the file here?

http://www.projectmadurai.org/pm_etexts/pdf/pm0001.pdf

The link you just posted says "projectmadurai.org" but the error says "catalog.projectmadurai.org"
seems you added catalog and that caused the error

That is just dummy link. I hope you not change anything while download the new small pdf file.

I am changing only pdf file name on your API file.

Changing file name on my API?
i dont understand. Do you have access to the API Backend?
How are you changing the name?

I used mockable.io site and created new API file.

new API files? or new PDF files?

new API file and just i put my pdf link.

Okay, can you post the link again? and also the error?

http://www.projectmadurai.org/pm_etexts/pdf/pm0001.pdf

I used your API file and just change the pdf file name. Got this error.
════════ Exception caught by widgets library ═══════════════════════════════════════════════════════
The following UnsupportedError was thrown building DownloadAlert(dirty, dependencies: [_InheritedTheme, _LocalizationsScope-[GlobalKey#a05c9]], state: _DownloadAlertState#8afa2):
Unsupported operation: Infinity or NaN toInt
The relevant error-causing widget was:
DownloadAlert file:///C:/Users/Kabilan/AndroidStudioProjects/books/lib/views/details.dart:354:29
When the exception was thrown, this was the stack:
#0      double.toInt (dart:core-patch/double.dart:193:36)
#1      double.floor (dart:core-patch/double.dart:163:34)
#2      Constants.formatBytes (package:books/util/consts.dart:75:35)
#3      _DownloadAlertState.build (package:books/components/download_alert.dart:106:41)
#4      StatefulElement.build (package:flutter/src/widgets/framework.dart:4619:28)
...

I really do not understand how you changed the files in my API because, the files from the App comes from Feedbooks API.
so you must be using your own API and your own files

Did you get any info after checking your website access?

Its not my website, its public website. Its my own API only. It's working fine for other sites only this site not working.

Did you add your website address anywhere in coding (except API file)?

No. It's only in the API file

Ok thanks. I will update if i resolved this issue.


Boot configuration (some wifi connection) get problems (Ubuntu)

Router1: b'NG-AP-ST', b'P\xe66\xc1\x0e\x10', 1, -71, 7, False
Router2: b'Freifunk', b'\xd2\x95\x05I\xd48', 9, -86, 0, False
Testsequence:
I start with empty Boot config:

Set up connection Router2 via Network or Connect Wi-Fi:
Store icon appears (rotate clockwise) but nothing happens/stored !
The store icon stays permanent with clockwise move.
But connection established (can get NTP)

I close the connection of Router2 and setup Router1
The Store icon appears and data can be stored !!!

After successful storing I can also (after close R1 and new setup Router2) store data ?!
But it seems be not the data of Router2.
When I start jamaBootCfg manually I can see always “Connecting Wi-Fi to "NG-AP-ST"... OK”,

Ubuntu 22.04, github source code

Hi @ghvau and thank you for your feedback!
It was a bug and I fixed it right then and there.
It works now, thanks again. 👍🏻

Thanks, solved!


Devon4Node MyThaiStar Backend in NestJS

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


x/pkgsite: Service Unavailable

What is the URL of the page with the issue?
https://pkg.go.dev
What is your user agent?
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36
Screenshot

and

What did you do?
Just access the documentation
What did you expect to see?
The documentation website
What did you see instead?
Service Unavailable, title 500 Server Error

Closing as it's back, after 5 min downtime.

Yes, this was a bad rollout. We were paged, and rolled back. Should be fixed now.
Thanks very much for the prompt issue!


[Gally]: master <- dev

Automatically created by Git-Ally

:tada: This PR is included in version 1.10.156 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Fixing some buggy data handling, add some logging

I'm fixing a few things in this plugin:

Wrapping a bunch of the SHOW ENGINE INNODB STATUS; parsing code in try/except blocks. This change is a little on the blind side - we were seeing the plugin get unloaded with this in the collectd logs:[2016-12-19 10:33:41] mysql plugin: Sending value: response_time_count/13=0
[2016-12-19 10:33:41] Unhandled python exception in read callback: ValueError: invalid literal for int() with base 10: 'ROLLING'
[2016-12-19 10:33:41] read-function of plugin `python.mysql' failed. Will suspend it for 60.000 seconds.

The output from SHOW ENGINE INNODB STATUS; is a bit unpredictable and difficult to parse, so I'm making the assumption that defensive coding here is the way to go.
2. not 0 == True, so not value will drop any values that are 0 on the floor. Check that value is not 0 before doing that return.
3. Add a little more useful verbose logging.

Would love to see this merged.

@cswingler: would you mind addressing the conflicts in this and we'll get it down? If you don't have the time, I can take a stab.

finally got around to doing this :)


Add kernel32!SetHandleInformation and kernel32!GetHandleInformation

P/Invokes for kernel32!SetHandleInformation and kernel32!GetHandleInformation
HandleFlags enum

@AArnott if you have any ideas on how to test these, please let me know

Regarding how to test this (which seems like a good idea), you could create a ManualResetEvent in the unit test and then obtain its Handle property, and call the 2 new methods on that handle, just to see if no exception is thrown.

Thanks for the feedback! Hope I understood you correctly. Also, I'd like to point out two things:

Is it okay that from  GetHandleInformation we may get a result that's not defined in HandleFlags, e.g. 0x00000003, which is HANDLE_FLAG_INHERIT | HANDLE_FLAG_PROTECT_FROM_CLOSE? Honestly, I don't quite understand that.
Also I don't know if the methods I added are store-banned, because I have no clue how to check that. Could you please explain?

You can check for store-banned status from the Microsoft API docs for the method:

So that's a good point: we need to move these methods to the storebanned folder so they don't cause store app rejection.

Is it okay that from GetHandleInformation we may get a result that's not defined in HandleFlags

Yes: that's the nature of [Flags] enums. Folks would test for individual flags.

Thanks for the help! Hope now it's all OK.


Fix: Ktor 2 not supporting iOS 12

Issue
Ktor 2.x introduced some changes to the Darwin engine which require min. iOS 13.
Ktor 2.1.1 introduced the LegacyDarwin engine as a fix.
Fix
Allow providing a custom engine.

LGTM!


Add option EBPF_JIT_ALWAYS_ON to permit building EC with no interpreter

Resolves: #737
Note:
If nothing referenced the uBPF code, then the linker doesn't include it in the final binary.
Signed-off-by: Alan Jowett<EMAIL_ADDRESS>

To build with interpreter disabled:
msbuild /m /p:Configuration=Release /p:Platform=x64 ebpf-for-windows.sln /p:DefineConstants=EBPF_JIT_ALWAYS_ON

@dthaler what would you recommend?

Debug build has both JIT and interpet mode and release build has only JIT.
Add another dimension to the build configuration (creating 4 possible options).

@dthaler what would you recommend?

Debug build has both JIT and interpet mode and release build has only JIT.
Add another dimension to the build configuration (creating 4 possible options).

1 sounds ok to me for now.

Can we rename EBPF_JIT_ALWAYS_ON to CONFIG_BPF_JIT_ALWAYS_ON?
I'd like anyone who searches for the latter (due to Linux docs) to be able to find this.

Can we rename EBPF_JIT_ALWAYS_ON to CONFIG_BPF_JIT_ALWAYS_ON? I'd like anyone who searches for the latter (due to Linux docs) to be able to find this.

Done.


Image upload fails when an object is uploaded to a newly created COS instance.

What happened:
When an image is uploaded to a newly created COS instance, an issue is observed with the error :
I0506 19:55:23.505155   63494 client.go:296] Resource service InstanceID :12db7e41-80af-4b23-b4fd-010242182638
Error: unknown error occurred, err: AccessDenied: Access Denied
	status code: 403, request id: aed6aa58-c541-4a72-8a29-221347bfb4f4, host id:
Usage:
  pvsadm image upload [flags]

./pvsadm image upload -b kishen-centos -f pvsadm
What you expected to happen:
Image upload should work without an issue for newly created COS instances.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:

pvsadm version (use pvsadm version):
Environment - On-Prem or IBM PowerVS:
OS (e.g: cat /etc/os-release):
Kernel (e.g. uname -a):
Others:

/assign

Fixed through #612


Not working on i3wm.

executing ./paperview train 5 just seems to refresh my monitor and doesn't change the wallpaper. Clip here
. Its a Thinkpad T450 with an integrated intel GPU if that helps.

Make sure to add a slash to the end of train, like train/

I'm having the same issue. I'm using the train/ syntax, but even after disabling feh it still does nothing.
I'm running an i7 8700K with GTX960 and Intel graphics disabled.

Alas it seems to be hit or miss. As its hardware / platform dependent it may be tricky to debug. Luck so far has been with Arch Linux and I3 / DWM window managers. I am curious if this setup is only applicable to Arch Linux

That's what I'm running. Arch linux with i3-gaps.
I am also using a liteDM for a display manager. Could this be the issue?

LiteDM most likely is the cause. I know desktop managers tend to overwrite the root X11 window. Worth a shot, maybe, but possible try it in a virtual machine first

It was actually picom that was the issue. After disabling it, it worked really well! Now all that's left is finding the cause of that issue...

From the reddit comments, many people were mentioning having no luck with picom. I'm glad you were you able to confirm. My best guess is that picom writes to the X11 root window after paperview does. The X11 is free real estate, after all. Why not draw to it if nothing else is? :P

Anywho, the latest push has some new bitmaps. Here's a video of it: https://www.youtube.com/watch?v=6ZTiA885bWM.
If you're interested in generating your own desktop images, pick a pretty gif from here: https://imgur.com/gallery/F9DAH, and save it to your machine. You can convert the gif into an array of bitmaps with imagemagick's convert tool:
convert -coalesce your.gif out.bmp
Just pop the bitmaps into a folder, and it will compatible with paperview.

Is there a way to convert parts of a video into an array of bitmaps? I think it'd be really cool to take clips of views from the ISS and play them through. Or possibly the videos on Apple TV's screen savers.

There are gif editors online that can probably shorten gifs. Any standard video editor on linux can do the same I imagine. Converting with imagemagick is just the final step. The less BMP wrangling the better.
Ha, I mean, if nothing is drawing to the wallpaper, and you're able to draw to the wallpaper as a picom developer, why not do so? Its a gateway for some special effects. Its just a shame, because leaves secondary hackers in the blue.

Oh, I'm not a Picom developer. I just use it since it makes my desktop look amazing! I was just messing around with the config file by doing things like turning shadows on/off to see if there was a setting causing the issue. No luck though...
Thanks for the help! I really like this simple easy-to-use background. I'm currently trying to see if I can add a feature to set the program to rotate through different scenes when the user enters a third argument to set the amount of time between transitions. Would this be something you'd be interested in?

It's probably related to something along these lines: https://github.com/chjj/compton/issues/162

Only those wallpaper setters providing the property _XROOTPMAP_ID or _XSETROOT_ID are supported.

It's probably related to something along these lines: chjj/compton#162

Only those wallpaper setters providing the property _XROOTPMAP_ID or _XSETROOT_ID are supported.

So fo fix the problem we need to set the background in the same way it does hsetroot
https://github.com/himdel/hsetroot

Oh, I'm not a Picom developer. I just use it since it makes my desktop look amazing! I was just messing around with the config file by doing things like turning shadows on/off to see if there was a setting causing the issue. No luck though...
Thanks for the help! I really like this simple easy-to-use background. I'm currently trying to see if I can add a feature to set the program to rotate through different scenes when the user enters a third argument to set the amount of time between transitions. Would this be something you'd be interested in?

Possibly, but likely not at this time. I prefer extreme simplicity over feature richness (even the code doesnt even handle a single error!) but I'll try out your fork for sure

So to fix this issue we need to find a way to make SDL render directly on the pixmap of the root window
Snippet to obtain the pixmap:
int screen = DefaultScreen(self.x11d);
Pixmap pixmap = XCreatePixmap(self.x11d,
                                RootWindow(self.x11d, screen),
                                DefaultDepth(self.x11d, screen),
                                DisplayWidth(self.x11d, screen),
                                DisplayHeight(self.x11d, screen));

And the after every render we need to reset the propertyies (XROOTPMAP_ID, _XSETROOT_ID)

It's probably related to something along these lines: chjj/compton#162

Only those wallpaper setters providing the property _XROOTPMAP_ID or _XSETROOT_ID are supported.

And this code will do the work:
int set_root_atoms(const Video *video) {
    int scr = DefaultScreen(video->x11d);
    Atom xroot = XInternAtom(video->x11d, "_XROOTPMAP_ID", True);
    Atom eroot = XInternAtom(video->x11d, "ESETROOT_PMAP_ID", True);

    // Clean old wallpaper
    if (xroot != None && eroot != None) {
        Atom type;
        int format;
        unsigned long length, after;
        unsigned char *data_xroot;
        XGetWindowProperty(video->x11d, RootWindow(video->x11d, scr), xroot, 0L, 1L,
                           False, AnyPropertyType, &type, &format, &length, &after,
                           &data_xroot);
        if (type == XA_PIXMAP) {
            unsigned char *data_eroot;
            XGetWindowProperty(video->x11d, RootWindow(video->x11d, scr), eroot, 0L, 1L,
                               False, AnyPropertyType, &type, &format, &length,
                               &after, &data_eroot);
            if (data_xroot && data_eroot && type == XA_PIXMAP &&
                *((Pixmap *) data_xroot) == *((Pixmap *) data_eroot)) {
                XKillClient(video->x11d, *((Pixmap *) data_xroot));
            }
        }
    }

    xroot = XInternAtom(video->x11d, "_XROOTPMAP_ID", False);
    eroot = XInternAtom(video->x11d, "ESETROOT_PMAP_ID", False);

    XChangeProperty(video->x11d, RootWindow(video->x11d, scr), xroot, XA_PIXMAP, 32,
                    PropModeReplace, (unsigned char *) &video->pixmap, 1);
    XChangeProperty(video->x11d, RootWindow(video->x11d, scr), eroot, XA_PIXMAP, 32,
                    PropModeReplace, (unsigned char *) &video->pixmap, 1);
    return 0;
}

(I've added a pixmap field in the struct video)
I've tested and it seems to work but i couldn't figure out how to render on the pixmap directly from SDL

I have a proof of concept that works with picom 🎉🎉 but I used another library for the rendering since i don't know SLD if anyone is willing to help porting it to SDL I'm here
Source:
https://gist.github.com/AlecsFerra/ef1cc008990319f3b676eb2d8aa89903

Amazing work. I can link to this in my README?

I'm having the same issue. I'm using the train/ syntax, but even after disabling feh it still does nothing.
I'm running an i7 8700K with GTX960 and Intel graphics disabled.

try restarting it


Slow saving?

I trying to save the binned and chromatically corrected data as "Tiff Volumes" (no compression, no 8bit conversion). When I do this in BDP2 it takes approximately 5-10x longer than when I do it from the BDP (applying the same corrections and saving with (what appears to me) same settings, same no. of I/O threads). For example, the described correction and saving process took 5min for 100GB dataset when doing it from BDP compared to cca. 50min using BDP2.

Yeah....I did expect that the BDP2 is slower, because of how things are done internally, but I did not expect that it is that much. I will check whether I can speed things up. This is not a simple thing though...
Would you still use it? I guess the argument could be that once one configured all the processing steps one could just let it run over night. In other words, do you think saving speed is a critical issue?

Could you do me a favour and try whether this speed difference is specific to the binning?
In other words, if you only do the chromatic shift correction, is it as much difference?

Without the binning BDP is still cca. 3x faster than BDP2
sure the speed is not so critical when doing things over night on TB size data (although the speed difference is significant so it might affect those data volumes as well).
In our case we have 100-200GB size data which we processed in BDP (chromatic shift, binning, saving as tiff) and than analysed in Imaris. The entire workflow in BDP including the saving took only cca. 5min (we new the exact amount of binning and the chromatic correction because we looked at the same data using the "interactivity" of BDP2 just before). So it was very satisfying when we could work on the data in Imaris after only 5min. Compared with a gap of 30-40min that was needed for saving when using BDP2. You see what I mean - 5min one can wait and continue with the analysis, but what do you do for 30-40min? After 40 min I'm already doing another project and I forgot what I was doing there in the first place :)
...it is different when something takes 2h vs 6h, I would run both things over night and than I would not care about the speed difference but when I can choose between 5 min and 30 min than I go with 5.

makes sense!
I will see what I can do!

Hi @metavibor,

Your workflow is saving from Luxendo h5 files to Imaris, correct?
If so, could you please try again, I worked on it, and changed some things, hopefully it is faster now.
Could you also please check how setting the number of IO threads during saving to one compares with having more threads (the hdf5 library is inherently single threaded, so it is in fact not so clear whether multi-threading helps...)?
Could you please paste here the text from the log window from the trial with one thread?

Thanks a lot for the testing!

Hi @tischi,
Here are some answers:
1. Your workflow is saving from Luxendo h5 files to Imaris, correct?
yes
2. If so, could you please try again, I worked on it, and changed some things, hopefully it is faster now.
It is much faster now. BDP is still somewhat faster (needs 3min for the task vs 4min that BDP2 needs at I/O=5) but this does not matter. So - great!! Thank you!
3. Could you also please check how setting the number of IO threads during saving to one compares with having more threads (the hdf5 library is inherently single threaded, so it is in fact not so clear whether multi-threading helps...)?
The whole process scales nicely linearly with the number of I/O (my workstation has 10 cores (and 20 logical processors)).
I/O=1 -> 20min for the task
I/O=5 -> 4min for the task
I/O=10 -> 2min for the task
I/O=20 -> a bit less than 2min (runs the CPU at almost 100% (compared to cca 60-70% at I/O=10) but for some reason that did not translate into faster job execution... I could not see any obvious bottlenecks in the hardware. I was saving on the SSD... it does not matter, just a curiosity)
I decided to use I/O=10
4. Could you please paste here the text from the log window from the trial with one thread?
Here is the content of the log window with one thread trial:
"Searching files in folder: Z:\Data\SPIM_tests\Samy\190314\dense collagen\2019-03-14_14.30.33\stack_1_channel_0
Number of files in main folder matching the filter pattern: 60
Folder: Z:\Data\SPIM_tests\Samy\190314\dense collagen\2019-03-14_14.30.33\stack_1_channel_0
FileType: Hdf5
BitDepth: 16
nX: 2048
nY: 2048
nZ: 181
nC: 2
nT: 30
GB: 91.10028288
Image size without binning [GB]: 91.10028288
Binned view size [GB]: 30.38158848
Binned view size [GB]: 10.13214108
Saving: Started...
Saving: Configuring volume-wise reader...
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 1/60; time ( spent, left, task ) [ s ]: 17, 1054, 17; memory: 2054MB of 97951MB (2%)
[DEBUG] Load and process data cube [ s ]: 14
Saved file: 2/60; time ( spent, left, task ) [ s ]: 34, 1013, 17; memory: 3507MB of 97951MB (3%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 3/60; time ( spent, left, task ) [ s ]: 52, 997, 17; memory: 4942MB of 97951MB (5%)
[DEBUG] Load and process data cube [ s ]: 14
Saved file: 4/60; time ( spent, left, task ) [ s ]: 69, 974, 17; memory: 6392MB of 97951MB (6%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 5/60; time ( spent, left, task ) [ s ]: 87, 960, 17; memory: 7832MB of 97951MB (7%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 6/60; time ( spent, left, task ) [ s ]: 105, 945, 17; memory: 9290MB of 97951MB (9%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 7/60; time ( spent, left, task ) [ s ]: 122, 929, 17; memory: 10726MB of 97951MB (10%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 8/60; time ( spent, left, task ) [ s ]: 140, 911, 17; memory: 12175MB of 97951MB (12%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 9/60; time ( spent, left, task ) [ s ]: 158, 897, 17; memory: 13631MB of 97951MB (13%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 10/60; time ( spent, left, task ) [ s ]: 176, 882, 17; memory: 15079MB of 97951MB (15%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 11/60; time ( spent, left, task ) [ min ]: 3, 14, 0; memory: 16519MB of 97951MB (16%)
[DEBUG] Load and process data cube [ s ]: 14
Saved file: 12/60; time ( spent, left, task ) [ min ]: 3, 14, 0; memory: 17967MB of 97951MB (18%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 13/60; time ( spent, left, task ) [ min ]: 3, 13, 0; memory: 19415MB of 97951MB (19%)
[DEBUG] Load and process data cube [ s ]: 14
Saved file: 14/60; time ( spent, left, task ) [ min ]: 4, 13, 0; memory: 20864MB of 97951MB (21%)
[DEBUG] Load and process data cube [ s ]: 16
Saved file: 15/60; time ( spent, left, task ) [ min ]: 4, 13, 0; memory: 22315MB of 97951MB (22%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 16/60; time ( spent, left, task ) [ min ]: 4, 12, 0; memory: 23760MB of 97951MB (24%)
[DEBUG] Load and process data cube [ s ]: 16
Saved file: 17/60; time ( spent, left, task ) [ min ]: 5, 12, 0; memory: 25217MB of 97951MB (25%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 18/60; time ( spent, left, task ) [ min ]: 5, 12, 0; memory: 26665MB of 97951MB (27%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 19/60; time ( spent, left, task ) [ min ]: 5, 12, 0; memory: 28114MB of 97951MB (28%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 20/60; time ( spent, left, task ) [ min ]: 5, 11, 0; memory: 29551MB of 97951MB (30%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 21/60; time ( spent, left, task ) [ min ]: 6, 11, 0; memory: 30999MB of 97951MB (31%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 22/60; time ( spent, left, task ) [ min ]: 6, 11, 0; memory: 32447MB of 97951MB (33%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 23/60; time ( spent, left, task ) [ min ]: 6, 11, 0; memory: 33895MB of 97951MB (34%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 24/60; time ( spent, left, task ) [ min ]: 7, 10, 0; memory: 35343MB of 97951MB (36%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 25/60; time ( spent, left, task ) [ min ]: 7, 10, 0; memory: 36791MB of 97951MB (37%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 26/60; time ( spent, left, task ) [ min ]: 7, 10, 0; memory: 38239MB of 97951MB (39%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 27/60; time ( spent, left, task ) [ min ]: 8, 9, 0; memory: 39688MB of 97951MB (40%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 28/60; time ( spent, left, task ) [ min ]: 8, 9, 0; memory: 41136MB of 97951MB (41%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 29/60; time ( spent, left, task ) [ min ]: 8, 9, 0; memory: 42595MB of 97951MB (43%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 30/60; time ( spent, left, task ) [ min ]: 8, 8, 0; memory: 44032MB of 97951MB (44%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 31/60; time ( spent, left, task ) [ min ]: 9, 8, 0; memory: 45480MB of 97951MB (46%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 32/60; time ( spent, left, task ) [ min ]: 9, 8, 0; memory: 46928MB of 97951MB (47%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 33/60; time ( spent, left, task ) [ min ]: 9, 8, 0; memory: 48376MB of 97951MB (49%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 34/60; time ( spent, left, task ) [ min ]: 10, 7, 0; memory: 49824MB of 97951MB (50%)
[DEBUG] Load and process data cube [ s ]: 16
Saved file: 35/60; time ( spent, left, task ) [ min ]: 10, 7, 0; memory: 51272MB of 97951MB (52%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 36/60; time ( spent, left, task ) [ min ]: 10, 7, 0; memory: 52720MB of 97951MB (53%)
[DEBUG] Load and process data cube [ s ]: 16
Saved file: 37/60; time ( spent, left, task ) [ min ]: 11, 6, 0; memory: 54168MB of 97951MB (55%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 38/60; time ( spent, left, task ) [ min ]: 11, 6, 0; memory: 55616MB of 97951MB (56%)
[DEBUG] Load and process data cube [ s ]: 16
Saved file: 39/60; time ( spent, left, task ) [ min ]: 11, 6, 0; memory: 57065MB of 97951MB (58%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 40/60; time ( spent, left, task ) [ min ]: 12, 6, 0; memory: 58513MB of 97951MB (59%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 41/60; time ( spent, left, task ) [ min ]: 12, 5, 0; memory: 59962MB of 97951MB (61%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 42/60; time ( spent, left, task ) [ min ]: 12, 5, 0; memory: 61409MB of 97951MB (62%)
[DEBUG] Load and process data cube [ s ]: 16
Saved file: 43/60; time ( spent, left, task ) [ min ]: 12, 5, 0; memory: 62857MB of 97951MB (64%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 44/60; time ( spent, left, task ) [ min ]: 13, 4, 0; memory: 64305MB of 97951MB (65%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 45/60; time ( spent, left, task ) [ min ]: 13, 4, 0; memory: 65753MB of 97951MB (67%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 46/60; time ( spent, left, task ) [ min ]: 13, 4, 0; memory: 67201MB of 97951MB (68%)
[DEBUG] Load and process data cube [ s ]: 16
Saved file: 47/60; time ( spent, left, task ) [ min ]: 14, 3, 0; memory: 68650MB of 97951MB (70%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 48/60; time ( spent, left, task ) [ min ]: 14, 3, 0; memory: 70097MB of 97951MB (71%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 49/60; time ( spent, left, task ) [ min ]: 14, 3, 0; memory: 71546MB of 97951MB (73%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 50/60; time ( spent, left, task ) [ min ]: 15, 3, 0; memory: 72994MB of 97951MB (74%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 51/60; time ( spent, left, task ) [ min ]: 15, 2, 0; memory: 74442MB of 97951MB (75%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 52/60; time ( spent, left, task ) [ min ]: 15, 2, 0; memory: 75890MB of 97951MB (77%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 53/60; time ( spent, left, task ) [ min ]: 16, 2, 0; memory: 77338MB of 97951MB (78%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 54/60; time ( spent, left, task ) [ min ]: 16, 1, 0; memory: 78786MB of 97951MB (80%)
[DEBUG] Load and process data cube [ s ]: 16
Saved file: 55/60; time ( spent, left, task ) [ min ]: 16, 1, 0; memory: 80234MB of 97951MB (81%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 56/60; time ( spent, left, task ) [ min ]: 16, 1, 0; memory: 81682MB of 97951MB (83%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 57/60; time ( spent, left, task ) [ min ]: 17, 0, 0; memory: 83130MB of 97951MB (84%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 58/60; time ( spent, left, task ) [ min ]: 17, 0, 0; memory: 84578MB of 97951MB (86%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 59/60; time ( spent, left, task ) [ min ]: 17, 0, 0; memory: 86026MB of 97951MB (87%)
[DEBUG] Load and process data cube [ s ]: 15
Saved file: 60/60; time ( spent, left, task ) [ min ]: 18, 0, 0; memory: 87474MB of 97951MB (89%)"

just a correction on question 1.
I'm saving from Luxendo h5 to "tiff volumes"

Great! Happy to hear that it improved that much!!

Could you please update your Fiji and test it again. Maybe it is even faster now....
You will be able to play with IO threads and processing threads separately.
Why do you save as Tiff and not directly as Imaris?

will do. If you could do for the "Bin" function the same thing you did for the "ChromaticShiftCorrection" that would be great (#54 in case you forgot)
I didn't see the benefit of saving in Imaris format since Imaris can open tiff volumes nicely. I'm not familiar with the Imaris format and I thought with tiffs I'm more flexible if I need to go to another software at some point. What is the benefit of saving directly in Imaris format?

Imaris will forbid soon to open Tiff, because the performance of Imaris is not good when working on Tiff based format.


Icon request: Sigh-In, Sign-Out icons

It would be great to have sign-in, sign-out icons here.

any news?

Hi, new icons are not created on this repo. It only contains the icon font and required css for self hosting the icons.
From the available icons, login icon looks close to what you want:
login

If you need more icons please try requesting on the official repo.


Sentinel-2 pair picking can exceed the maximum lambda runtime of 900 s (15 min)

In its-live-monitoring-test, processing the sentinel2-valid-2,json integration test message (shown below) pulls so many potential secondary pairs that the lambda is timing out even when setting the maximum timeout of 15 minutes.
{
  "name": "S2B_MSIL1C_20240430T142739_N0510_R139_T24VUR_20240430T162937",
  "id": "bb62af19-0e23-4f90-87be-a1e936a117db",
  "path": "products/2024/4/30/S2B_MSIL1C_20240430T142739_N0510_R139_T24VUR_20240430T162937",
  "timestamp": "2024-04-30T14:27:39.024Z",
  "datatakeIdentifier": "GS2B_20240430T142739_037345_N05.10",
  "sciHubIngestion": "2024-04-30T17:19:38.004Z",
  "s3Ingestion": "2024-04-30T18:14:57.807Z",
  "tiles": [
    {
      "path": "tiles/24/V/UR/2024/4/30/0",
      "timestamp": "2024-04-30T14:34:42.769Z",
      "utmZone": 24,
      "latitudeBand": "V",
      "gridSquare": "UR",
      "datastrip": {
        "id": "S2B_OPER_MSI_L1C_DS_2BPS_20240430T162937_S20240430T143400_N05.10",
        "path": "products/2024/4/30/S2B_MSIL1C_20240430T142739_N0510_R139_T24VUR_20240430T162937/datastrip/0"
      }
    }
  ],
  "datastrips": [
    {
      "id": "S2B_OPER_MSI_L1C_DS_2BPS_20240430T162937_S20240430T143400_N05.10",
      "path": "products/2024/4/30/S2B_MSIL1C_20240430T142739_N0510_R139_T24VUR_20240430T162937/datastrip/0"
    }
  ]
}

The test stack is much more verbose, setting the logging level to DEBUG -- it's possible that setting logging to INFO would keep us under the timeout...
But I think that this is currently a blocking issue.

Switching from DEBUG to LOG logging level in the test environment brought this down to just over 5 minutes, and sentinel2-valid-1.json ran in 1.5 minutes.
I will leave this open to monitor over time as we'll probably want to address it, but this is not a blocking issue anymore. We can monitor and determine how often this is a real problem.
Another strategy would be to reduce the search bounding box so that it doesn't overlap with neighbor tiles, significantly reducing the number of results to check.


cashexplorer.bitcoin.com was taken down, replace with api.bitcore.io?

Some progress here: https://github.com/teran-mckinney/bitcash

I don't thing cashexplorer.bitcoin.com was ever a thing (it's explorer.bitcoin.com).
But now the API is located at rest.bitcoin.com.
I added bitcore support in #46. I would like implement rest.bitcoin.com support for redundancy :)

Looks like this API blocks Tor users so does cause issues that way.

This issue should be fixed by #47

Looks like this API blocks Tor users so does cause issues that way.
Both rest.bitcoin.com and api.bitcore.io work for me on Tor Browser. Are you still experiencing this issue?

Ok, off topic:
urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): trest.bitcoin.com:443
urllib3.connectionpool: DEBUG: https://trest.bitcoin.com:443 "GET /v2//rawtransactions/sendRawTransaction/01000000016450c5c7fb706f5c0cdf1b3fbb1d98b66f8d049770c61fd8763c8e8cda712b83010000006a4730440220528f7ac6565916f1ad2e4e8e8f453efa47afa349203b08a2de0e9af78d19229802202a4a00cdda40c4dfbe03ee9208ea0f6b2898a4271013e6a014c4902fa50461e64121033d5c2875c9bd116875a71a5db64cffcb13396b163d039b1d9327824891804334ffffffff02d0070000000000001976a914e7c1345fc8f87c68170b3aa798a956c2fe6a9eff88ac64621b00000000001976a914990ef60d63b5b5964a1c2282061af45123e93fcb88ac00000000 HTTP/1.1" 404 36

Is it really a GET request for sending a TX? Not a POST? Also gives me a 404, but over Tor. Not sure over clearnet.
Looks like Tor Browser works with https://rest.bitcoin.com/v2/address/utxo/bitcoincash:qzfyvx77v2pmgc0vulwlfkl3uzjgh5gnmqk5hhyaa6 but the library does not. If you curl it, it blocks it (torsocks curl). So rest.bitcoin.com needs to configure Cloudflare to not filter by user agent.

Is it really a GET request for sending a TX? Not a POST?

Yes, for a single transaction it is a GET.
You can also use POST for multiple transactions (as per the docs on rest.bitcoin.com).
Regarding Tor, I will ask the Bitcoin.com team if they can expedite this.

Gotcha. Thank you!

@teran-mckinney could you close this? 👍

@merc1er thanks for finding this issue! https://github.com/Bitcoin-com/rest.bitcoin.com/issues/333
Good to know we have an answer now.


The file was completely deleted!

🚨 The issue tracker is not for questions 🚨
The same file was deleted twice and the other files were deleted the second time. Can you retrieve it?Note is delete completely!

@binperson can you be more specific of what happened?


My Chart needs touch show more Data

** Don't make a duplicate issue.
You can search in issues to make sure there isn't any already opened issue with your concern.
fl_chart : 0.55
flutter :
[✓] Flutter (Channel beta, 3.3.0-0.5.pre, on macOS 13.2 22D49 darwin-arm, locale zh-Hans-CN)
[!] Android toolchain - develop for Android devices (Android SDK version 33.0.1)
✗ Android license status unknown.
Run flutter doctor --android-licenses to accept the SDK licenses.
See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.
[!] Xcode - develop for iOS and macOS (Xcode 14.2)
✗ CocoaPods not installed.
CocoaPods is used to retrieve the iOS and macOS platform side's plugin code that responds to your plugin usage on the Dart side.
Without CocoaPods, plugins will not work on iOS or macOS.
For more info, see https://flutter.dev/platform-plugins
To install see https://guides.cocoapods.org/using/getting-started.html#installation for instructions.
[✓] Chrome - develop for the web
[!] Android Studio (not installed)
[✓] VS Code (version 1.75.1)
[✓] Connected device (3 available)
[✓] HTTP Host Availability
Is your feature request relasted to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
I want to display some data other than coordinate value when the chart is displayed by touch, such as the name of the curve. How to achieve this?  includd,linechart,barchart.
thank you
Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Please check the BarChartSample1:
(You can do the same in LineChart)
Sample 1 (Source Code)

I want to display text information in the way of not wrapping, or in a certain width of internal wrapping, how to configure? thank you，as the question on the linechart

You need to increase the tooltip maxContentWidth property in the LineTouchTooltipData or BarTouchTooltipData.


add message field validation of contact us page

add message validation to the contact us form to avoid single letter submission which often leads to garbage response

okay add validation using zod there make PR to dev after complete and before making PR post video here

if you have any issue in setting up backend we are in vc now


Fix to_rgb function

I found that the demo code does not work well as is - especially for deep model, so I fixed some minor bugs - especially to_rgb function.
Anyway, the work is amazing and I was impressed! 👍

Thanks a lot for your contribution!


Export JVM metrics from the javaagent using micrometer

README changes will be added in a separate PR

Seems like the tests have passed (at least the linux ones...) -- @iNikem @kubawach @breedx-splk @jkwatson I've addressed all your comments, please re-review this PR 🙏


does this work in command mode with the calendar plugin at the same time?

oh nvm i figured it out

@simkimsia what did you figure out?


[1.11.0+] createLocalScreenTracks returns ended tracks

Describe the bug
I first discovered this while working with createLocalScreenTracks.  LocalTrack calls super(), which leads to setting this_mediaStreamTrack. Then it calls setMediaStreamTrack, which stops this._mediaStreamTrack if it is set.
1. LocalTrack.constructor(mediaStreamTrack) ->
Track.constructor(mediaStreamTrack) ->
this._mediaStreamTrack = mediaStreamTrack

2. LocalTrack.constructor(mediaStreamTrack) ->
LocalTrack.setMediaStreamTrack(mediaStreamTrack) ->
if (this._mediaStreamTrack) { this._mediaStreamTrack.stop() }

Reproduction
Use createLocalScreenTracks. Returned track readyState is already ended.
Logs
No response
System Info
Any os, any browser.

Severity
blocking all usage of LiveKit
Additional Information
No response

thanks for the report, fixed with https://github.com/livekit/client-sdk-js/pull/769


nvim => tmux not working: "zsh:1: command not found: tmux"

Followed the troubleshooting guide. I am on macOS 14.5. The output of TmuxNavigatorProcessList for me is zsh:1: command not found: tmux. Have messed with some of the shell settings on nvim and tmux to no avail and out of ideas. I am guessing it's related to the shell because tmux is certainly a command and installed.
Any ideas?

Same for me ... it works on Mac M1 pro but doesn't work on Mac M3 max (not sure if that's relevant)

This uses the system() vim command to execute tmux. It could be that vim is running in some kind of sandbox and does no have access to / doesn't see tmux. I encountered this when running vim from flatpak (flatpak run org.vim.Vim):
/bin/sh: line 1: tmux: command not found

for vim installed from apt (I'm using Pop OS) it works.

Okay so for me, it turned out to be the .zshenv file. I entirely deleted it and everything is back to normal.

I believe the issues here are unrelated the plugin -- if tmux: command not found then as noted, it sounds like an issue with vim, the shell vim uses (note, it might be different from your normal terminal shell -- see this thread for notes), and the $PATH configuration for where to find tmux.
I'm going to close this now based on those notes, but please feel free to comment or reopen if you think I'm missing something.


Feature/get file bug

Fixed by file deployment bug for milestone 0.1.0

Thanks, could you squash commits into one?

Ignored as new pull request with proper squashed commits submitted


Updated validation.py python script

Updated validations.py python script.
Fixed the behavior of validate_user function in validations.py.
Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

[ ] Tests pass
[ ] Appropriate changes to README are included in PR

"Files changed


Put server argument of tf.exe in quotation marks to allow for whitesp…

Using the task "Index Sources & Publish Symbols" v2, source retrieval breaks for TFVC URLs with spaces, e.g. "http://tfs.example.com:8080/tfs/MY REPO".
It seems the quotation marks, which are present in function New-TfsGitSrcSrvIniContent, are missing in function New-TfvcSrcSrvIniContent.

Adding @arunkm - could you take a look at this?

Closing due to age, please reopen if needed.


Feature Request: work with Central Management Server

SQL Operations Studio Version: 0.23.6

We have lot of servers group and a lot more servers in CMS, recreating them in new OPS will be big headache.

Completely agree. We have a couple hundred instances, to manually add them would be no good.

I think it will be possible to write custom script to convert CMS content to %APPDATA%\sqlops\User\Settings.json with help from dbatools module function Get-DbaRegisteredServer

@Zhunya As a one-time thing, yes, but that would mean breaking from actively used/maintained CMS which is unappealing.

@Zhunya we plan to fully support CMS integration in Object Explorer viewlet prior to GA.

@kburtram Do plans to "fully support CMS integration" include support for multi-server queries? In SSMS, that is one of the features I use CMS for the most--to run a query against a group of CMS servers, and get a single, consolidated result set.

I sure hope this will be included in the June release, as we plan to also start using Azure Data Studio a lot more next to SSMS18. But currently this is a blocking issue for us.

CMS is coming in June release and already available in our Insider Build. Closing issue.


Plots scaling

For some reason today my plots from QMLT started to look like this:

I wasn't able to fix it, so I hacked it - in plot.py, _plot method I just changed ax.set_ylabel(ylabel) into ax.set_title(ylabel).

Which actually is much easier to read than this rotated y_label - especially in the case of the long parameter names.

Hi @mstechly, that's a good solution - it didn't show up in our testing because our parameter names were never large enough! Since you have already made the change your end, feel free to submit a small pull request with the change.
Another thing that comes to mind - at the moment, all default variables will be plotted against the step on the x axis. Originally, the plotting module was written to be more general - hence why 'global_step' is used for all x-axis labels. Do you think it would still be clear if this axis label was removed?

Ok, I will do the PR - it will require some refactoring because this argument is called "y_label" and not "title" - I will come up with an appropriate solution.
Also, I wanted to point out, that it worked properly with such long names earlier - I have no idea why it broke just today.
I think it will be clear, however, I'm not 100% sure. I will try to ask someone not involved for the opinion.


Toolbar - Break with id in sub menu

I'm submitting a ...
[x ] bug report
[ ] feature request
[ ] question about the decisions made in the repository

What is the current behavior?
If you have a break with an id set inside an sub menu of the toolbar the break won´t be displayed correct.
It shows the text of the break id.

If the current behavior is a bug, please provide the steps to reproduce and if possible a screenshots.

What is the expected behavior?

What is the motivation / use case for changing the behavior?

Link to jsFiddle that demonstrates the problem:
@https://jsfiddle.net/so3dq3so/

Other information

Created a PR: #1695

Closing as resolved: https://jsfiddle.net/ghmfbd43/


What about fast_match and regex iterator?

Regex iterator in C++20 may be a coroutine-generator, isnt it better then returning a range?
What about fast_match (only checks match or not) ?

They may be coroutines, but I do support also C++17. What about fast_match?

What about fast_match
I do not see such possibility now, sorry if it exist


Add readme of autoware_version

Description

Review Procedure

Remarks

Pre-Review Checklist for the PR Author
PR Author should check the checkboxes below when creating the PR.

[ ] Read pull request guidelines
[ ] Code follows coding guidelines
[ ] Assign PR to reviewer

Checklist for the PR Reviewer
Reviewers should check the checkboxes below before approval.

[x] Commits are properly organized and messages are according to the guideline
[x] Code follows coding guidelines
[ ] (Optional) Unit tests have been written for new behavior
[x] PR title describes the changes

Post-Review Checklist for the PR Author
PR Author should check the checkboxes below before merging.

[ ] All open points are addressed and tracked via issues or tickets
[ ] Write release notes

CI Checks

Build and test for PR / build-and-test-pr: Required to pass before the merge.
Build and test for PR / clang-tidy-pr: NOT required to pass before the merge. It is up to the reviewer(s). Found false positives? See the guidelines.
Check spelling: NOT required to pass before the merge. It is up to the reviewer(s). See here if you want to add some words to the spell check dictionary.

なんだろうこれ :thinking:

It was resolved by re-run jobs. Maybe a temporary network problem? :thinking:

I see, thank you!


Streaminfo not accessible in async callback

When making async call inside onRequestHeader (and I think the other event handlers as well), pointers to streaminfo's data struct (e.g. request headers) will be reset after return from onRequestHeader, which makes those data structure not accessible in async call's callback. Seems like SetEffectiveContext does not have access to those data. We probably should keep those pointers if module event handlers return StopIteration.

What other aspects other than the headers do you need ?

I only need headers for the sample plugin. I think your PR already covers all other aspects that I can think of.


Flatbox bricked

Hey guys, just got a new Flatbox that had GP2040-CE installed. Tried to update the firmware and supposedly chose the wrong .uf2 (rev5 when i should have gone rev4 i guess) and kinda bricked my device... Buttons are shifted all around and I can't get into the web configurator to undo my damage... Have any idea how I can save that? Can't access web configurator or Bootsel...
Thanks in advance
Max
EDIT:
There was a button that allowed me to force Bootsel, I found the flash-nuke, but now none of the verisons (0.7.1 since that was the default) works correctly (the board inside this seems to be a custom pcb, named "FightingBox-XLL-rev1.1") and it has a little screen. It was this one from Ali https://de.aliexpress.com/item/1005005514703052.html?spm=a2g0o.order_list.order_list_main.5.21ef5c5f90dj4I&gatewayAdapt=glo2deu  the model with screen and led switches

download the flash nuke from the downloads section
https://gp2040-ce.info/#/download
Hold down the BOOTSEL button on your device and plug it in.
Drag the Nuke file into your device and it should erase your old firmware and allow you to start over.

hey Spoony! Thanks for your reply, I already was able to nuke that, but I can't figure out a firmware that is working... Any ideas? I messaged the seller on Ali to find out which firmware they used, but I am kinda pessimistic about that approach,..


Update AWSExecutor to become self-hosted compatible

When hosting Covalent on AWS care must be taken to ascertain how credentials are acquired and used when interfacing with other AWS services through the executors (Batch, ECS, EC2 ... ). When using Covalent locally, the AWSExecutor base class parses the shared credentials file (located under ~/.aws/credentials`) to parse the access key, secret key and the session token. However, when running inside an EC2 instance, this method will not work since the shared credentials file is not present and as a good practice no long lived credentials should ever be placed inside EC2 instances.
The solution is for Covalent to assume the IAM role attached to the EC2 virtual machine before making any API requests to other services such as Batch, ECS etc. To support this, the AWSExecutor needs to be modified slightly so that the self.execution_role attribute is used and assumed by boto3 to generate temporary credentials.
Action items

Update the AWSExecutor constructor to fetch temporary credentials when an execution role is specified.
Fall back to parsing the credentials from AWS_SHARED_CREDENTIALS_FILE if the above fails
Fail to execute if both of the above fail

Solution:

Update self.boto_session_options return value
if a user provided value for profile_name is found then add that entry into the session_options dictionary
if the profile_name is default then do not add that to the session_options dictionary

this will enable boto3 to get temporary credentials from Ec2InstanceMetadata
When running locally, where there is not instance metadata, boto3 will read the profile from the credentials file


top_k and top_p transposed in vertexai

Fix transposed properties in vertexai model
Models

@hwchase17
@agola11

@khallbobo - you might be interested in this since you just fixed something related. :)

Added a test to cover that args are sent to the library as expected.
The call against the LLM is mocked, but I couldn't figure out how to mock out the environment check.
So it does make the call to validate your credentials.

Wow.  This will certainly change results!  Without my change, these are ignored anyway.  With my change, we will start seeing very different results.

Google is now returning an error if you are using an older version of langchain. For anyone else wondering why their project has suddenly broken when nothing has changed their end (and they may not even be setting top_p), you may be seeing this message:
google.api_core.exceptions.InvalidArgument: 400 40.000000 is out of supported range [0, 1];  for top_p.

@mheguy-stingray to be clear this change fixes the issue I was seeing. I just wanted to highlight the error message so others can discover this issue and find a resolution quickly.
I was using 0.188 which was working for for me up until yesterday. I then started seeing the error message I've referenced above and everything stopped working. I assumed it was my code - but even rolling back changes and deleting indexes didn't fix the issue. I spent a lot of time scratching my head and trying to work out why it suddenly broke for me.
My assumption is that the VertexAI was previously ignoring an invalid top_p setting, hence everything was previously working. It is an assumption, but I see no other explanation. Either way, I thought it might be useful to highlight the error that the API returns in case others come across it.

@mheguy-stingray I've looked at #5566 but that appears to only apply to chat. Your fix applies to all VertexAI calls as _default_params is used in _predict.


[1 of 2: DCOS-12561] Page Header Breadcrumbs

This is a refactor of the ServicesBreadcrumbs components. It adds some new features:

If the user is navigating through groups, only the last two groups will appear in the breadcrumb. All other breadcrumbs get "collapsed" into the ellipsis. Clicking the ellipsis navigates one level up.
When there is not enough horizontal space to display the service health, it gets removed from the DOM.

I also introduced four new breadcrumb components:

Breadcrumb: this wraps the rest of the Breadcrumb components
BreadcrumbCaret: this will render a caret
BreadcrumbSupplementalContent: this content will not shrink by default, and is meant for icons or other supplemental content
BreadcrumbTextContent: this is the primary content; it will shrink and display an ellipsis when truncated

A follow-up PR will implement these new components in the rest of the app. Until then the breadcrumb styling will incorrect.
This styling is intentional for this PR:

Checklist

[x] Did you add a JIRA issue in a commit message or as part of the branch name?
[ ] Did you add new unit tests?
[ ] Did you add new integration tests?
[ ] If this is a regression, did you write a test to catch this in the future?

Hey @jfurrow , is something like this expected with this PR?

Sorry @wavesoft the screenshot in the description showed something quite different... that is correct, and https://github.com/dcos/dcos-ui/pull/1935 will fix it to look pretty again. Perhaps we shouldn't split up the PRs at all?

I was thinking that every PR that targets master should not break functionality. I am not sure how this can be achieved here, possibly by splitting this into a different place, such that each part is not changing anything on master?
But personally, I am also OK if we merge this and the next one in the same day, but I don't know how we can enforce this. Therefore it gets my label 👍


test fix for app insights metric alert

/networking

/networking

/networking

Hey, @eosho!
:rocket: Deploying network is beginning! :rocket:

/infra

Hey, @eosho!
:rocket: Deploying infra to 'wbademo-RG' is beginning! :rocket:


Hot reload issue

Hot restart and hot reload not working properly ion android studio

Hi @asfaqueahmedsakkar
please follow up on https://github.com/flutter/flutter/issues/57796 and comment there,
closing current issue as duplicate


Update kubectl drain docs

change docs/user-guide/kubectl/kubectl_drain.md and docs/man/man1/kubectl-drain.1 text so that they mention that eviction is used when the apiserver supports it.
Link to [http://kubernetes.io/docs/admin/disruptions/]
Say that it waits for graceful termination
Say that you should not operate on the machine until the command completes (duh?)
push the updated docs to kubernetes/kubernetes.io

@pwittrock @ymqytw

This is documenting https://github.com/kubernetes/kubernetes/pull/35483

We run this script to update imported docs, including kubectl docs https://github.com/kubernetes/kubernetes.github.io/blob/master/update-imported-docs.sh
So @ymqytw only needs to update kubectl drain code, and running this script will automatically sync those updates

Thanks @janetkuo
Created kubernetes/kubernetes#37321

@janetkuo I think we can close this, since #37321 has been merged.


Show all commits for all metrics and highlight failed metrics

To "align" the metrics for catching spikes and regressions visually, we show all the commits for all the metrics.
Metrics that don't have a value for the latest commit are treated as failed commits.

Edit: Adding a screenshot

Additionally, this PR also has a couple of other bug fixes.

Fix React warnings about missing  key
Fix bug with marking metrics as old metrics

I also worked on making graphs using full width and aligning them vertically and increasing the visual density (#315) , since these changes seemed related to the other work of displaying all the commit IDs on the x-axis for all the metrics.

This looks very good! What do you think of the full width graphs, are we loosing too much information density?

This looks very good! What do you think of the full width graphs, are we loosing too much information density?

I like the full width graphs, and having the commits aligned vertically. I think we could potentially move the title, subtitle and legend/last value to a column next to the graph to use some more of the vertical space. But, in general I like the idea of moving towards vertically aligned graphs without having to move our eyes horizontally across the page to compare things.

This looks very good! What do you think of the full width graphs, are we loosing too much information density?

I like the full width graphs, and having the commits aligned vertically. I think we could potentially move the title, subtitle and legend/last value to a column next to the graph to use some more of the vertical space. But, in general I like the idea of moving towards vertically aligned graphs without having to move our eyes horizontally across the page to compare things.

With #323, it should help with saving more vertical space on overlaid graphs.

This looks very good, thanks!


Add back in cell_path support

Found a FIXME comment on line 96 of file crates/nu-command/src/conversions/into/binary.rs

I think this has been fixed, let's close it.


Ask some questions

Dear Dr.Zhang, If there are no labels in my trainset.Can I remove the cls_token? If  do, will it influence the training process?

Hi, the cls_token represents the global information of the input image. It is called cls_token (aka class token) since it was first used to do classification.
For the MAE pretraining, I don't think the cls_token is that important. Also, the MAE algorithm itself doesn't require you to have labels in your dataset. Remove it just make it different with the most ViT models in the field.
For the classifier training, a cls_token is recommended to have. If you remove that, you need to different way to get the representation for the classifier head. Maybe a mean-pooling of the image tokens can do the job?

To be honest, in my opinion ,I just regard MAE as an embedding method for my tasks.  I will add a linear layer at the end of model to output the final embedding, then I use KNN algorithms with the final embedding to get a adjacency matrix followed by clustering( using some clustering algorithms such as Louvain, Leiden...) But I have a question, if I get the classifier results from MAE model, Will this result be different from the one I got through the method above?
BTW, as for my input data. just two same shape matrix( rows represent samples, columns represent features), I regard two matrixs as two channels of pictures, but the ptaches are one dim in mydataset. Do you have some suggestions or hints for me? Thanks!

In addition, my goal is to get the labels of input example and visualize it by umap or TSNE. I think the features used for visualization are very important, otherwise the distinction between each community is not going to be very clear in the results of the visualization.

Regarding your setting, if I am understanding correctly, you are training with some tabular data with a 2d vector feature for each entry. I think the tabular setting will generaly be a challenge for MAE, since one core assumption is that the information in the input data (images) is redundant. That is why do can reconstruct the data with a high masking rate of 75%. For tabular data, I presume it is more independent between each entry? In this case, you should not use high masking rate in general. Or maybe your data is intercorrelated? Then you can try with higher masking rate.
I cannot tell which option is better (using the cls_token as embedding or mean pooling the tokens) for your dataset. If I were you, I will just try both and see which one works better.

Thank you so much for your reply, I will do it. The data is interconnected in a sense. Thanks.

Excuse me, I'm sorry to bother you. I have several questions want to ask you.Because I add a layer in the end of decoder, and use the ouputs to cluster and visualize, but the cluster figures are not well.Many points are not clustered together. I suspect the set of  layers of encoder and decoder and head numbers influence results. To be honest, I'm not sure how to decide the models's paras such as layer numbers and head numbers. Maybe should I try different combination?  Second , I see run 20 epoches,the loss is decreased to 0.02, maybe overfitting, but when I adjust the maskratio and dropout,the effect on the results is not very clear. In addition, the cluster figures are not well. Is it possible that I extracted the output in the wrong place? Added codes attached

I am not sure what you intend to do. Are you looking for some representation vector for your data? Then you should take the output of the encoder as the representation instead of the decoder.

Get it. Thanks for your reply.


Support multiline commands

Working with JMESPath queries and Kusto queries (in the Resource Graph extension) results in some long commands.
This PR makes it possible to split long commands across several lines, add comments, and execute a selected substring of a longer command.

@chrmarti / @chrisdias  - I can't see the build error; can you take a look and let me know what's wrong?

@chrmarti / @chrisdias - I set up an Azure DevOps build pipeline using the azure-pipelines.yml at the root of the project and the branch for this PR (in my fork) builds without any errors.
I you can let me know what the error is I'd love to fix it and get this merged.

@chrmarti / @chrisdias - it's been a while since I checked on this PR. I can't see the build error but if you let me know what it is I'd love to get it fixed. If y'all don't think this PR is worthwhile just let me know and I can close it.

@chrmarti - I see all the issues as being resolved; please let me know if there are still some changes I need to make.

Looks good. I've added 'line' to the settings id to make it render nicely in the settings editor:

Thanks!

@mburleigh I have published this feature with version 0.5.0 of the extension. As usual you will be mentioned in the VS Code (1.42) release notes for this PR. 👍


Automated cherry pick of #11245: fix: get apigateway auth/regions with error of illegal mix of collations

Cherry pick of #11245 on release/3.6.
#11245: fix: get apigateway auth/regions with error of illegal mix of collations

Codecov Report

Merging #11247 (7f0f640) into release/3.6 (b9d30c0) will increase coverage by 2.07%.
The diff coverage is 4.00%.

@@               Coverage Diff                @@
##           release/3.6   #11247       +/-   ##
================================================
+ Coverage         5.86%    7.94%    +2.07%
================================================
  Files              828      107      -721
  Lines           129133    13240   -115893
================================================
- Hits              7577     1052     -6525
+ Misses          120876    12068   -108808
+ Partials           680      120      -560

Flag
Coverage Δ

aFlag
7.94% <4.00%> (+2.07%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/apigateway/handler/auth.go
0.00% <0.00%> (ø)

pkg/apigateway/handler/auth_totp.go
0.00% <0.00%> (ø)

pkg/apigateway/handler/idp.go
1.84% <0.00%> (ø)

pkg/apigateway/handler/misc.go
0.00% <0.00%> (ø)

pkg/apigateway/handler/resource.go
0.00% <0.00%> (ø)

pkg/baremetal/cronjobs.go
0.00% <0.00%> (ø)

pkg/baremetal/manager.go
0.49% <0.00%> (-0.01%)
:arrow_down:

pkg/cloudcommon/db/lockman/inmemory.go
64.55% <20.00%> (+0.08%)
:arrow_up:

pkg/monitor/alerting/eval_context.go
34.25% <0.00%> (-3.89%)
:arrow_down:

pkg/util/imagetools/imagetools.go
40.50% <0.00%> (-2.74%)
:arrow_down:

... and 647 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 19c0825...7f0f640. Read the comment docs.


Unopar

Student of Unopar

@Maridh src/mariana.kt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

Hello!
I already fixed can you check for me please?
Em qua., 15 de jun. de 2022 às 06:28, Philip Torchinsky <
@.***> escreveu:

@Maridh https://github.com/Maridh src/mariana.kt: this request has an
incorrect file path format, please ensure that you are familiar with this
repository rules located at the bottom of this page
https://github.com/JetBrains/swot and change your request accordingly.
If you don't know how to change request, close it and open a new one. If
you are not able to make the pull request according the instructions,
please send us an email to @.*** Please mention this
request URL in the message.
—
Reply to this email directly, view it on GitHub
https://github.com/JetBrains/swot/pull/14383#issuecomment-1156230202,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANGWVSVBPOL4V4PI2LEOT4LVPGO4HANCNFSM5YZBHNKA
.
You are receiving this because you were mentioned.Message ID:
@.***>


Add Pod Distribution metric for kube-burner

Signed-off-by: Sai Sindhur Malleni<EMAIL_ADDRESS>Description
Fixes

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@da344e8). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #745   +/-   ##
=========================================
  Coverage          ?   53.82%
=========================================
  Files             ?        8
  Lines             ?      327
  Branches          ?        0
=========================================
  Hits              ?      176
  Misses            ?      151
  Partials          ?        0

Flag
Coverage Δ

gha
53.82% <0.00%> (?)

python-3.9
53.82% <0.00%> (?)

unit
53.82% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update da344e8...debd6b2. Read the comment docs.