ODH-ADR-Operator-0005: Whitelist component fields for user customizations

Description
This ADR is a subset of feature requests defined in #23 .
How Has This Been Tested?

Merge criteria:

[ ] The commits are squashed in a cohesive manner and have meaningful messages.
[ ] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[ ] The developer has manually tested the changes and verified that the changes work

/cc @israel-hdez @danielezonca @etirelli @LaVLaS

After reading this ADR, the main question I have is:
Why do we reconcile everything back to the default state?
Shouldn't we treat the components' manifests as a sort of "reasonable" default and allow users to adjust them according to their needs? KfDef version (v1) might have been too liberal in this regard, as it merely checked if a given resource already existed and skipped applying if that was the case. I understand that we want to have control over certain aspects, but this proposal suggests we might have gone too far with this approach.
Perhaps we should start considering what changes users can make (or define what is not changeable) to the resources we create and allow them to do so.

I do agree with letting users update the resources, I am just not sure how we can limit the scope of the updates. So if we do not reconcile to the desired state, there can be usecase when users do multiple config updates and result in an error state.  Maybe we can add a way for user to return to defaults if they are in such error state?
Kfdef actually provided no way to customize, it used to reconcile everything back to the default state.

I do agree with letting users update the resources, I am just not sure how we can limit the scope of the updates

Maybe we could find a simple way to define paths/fields of a particular resource that we will skip during reconciliation. We can for example remove such a field from the patch operation when applying the original, "desired" state.

So if we do not reconcile to the desired state, there can be usecase when users do multiple config updates and result in an error state. Maybe we can add a way for user to return to defaults if they are in such error state?

I am not sure about this one. My line of thinking is more of what kind of changes would result in "unsupported configuration". For example, we might want to make sure our images are always reconciled. But things like resources are very dependent on the target env and usage patterns so this could be skipped from reconciliation.

Kfdef actually provided no way to customize, it used to reconcile everything back to the default state.

I have only worked on the plugin part of KfDef and in there when you were creating so-called "raw resources" it was skipping apply during reconciliation when the resource already existed. So I stand corrected :)
OTOH it lets you provide your own overlays to customize the kustomize :) I am not saying this is the way to go, but it was possible without blowing up the KfDef API.

Why do we reconcile everything back to the default state?
Shouldn't we treat the components' manifests as a sort of "reasonable" default and allow users to adjust them according to their needs?

I know we can go this way to be quite flexible. However, I'm more on the side that it is the operator CRDs that should expose the needed settings, rather than instructing the users to do it themselves.
IMHO, the more we allow for such flexibility, the more the operator becomes less valuable -- i.e. why not just provide the manifests to the user?
Personally, I'm quite sold to the explanation in operator framework website:

With Operators you can stop treating an application as a collection of primitives like Pods, Deployments, Services or ConfigMaps, but instead as a single object that only exposes the knobs that make sense for the application.

So, the general idea that this ADR is proposing is good to me.

Why do we reconcile everything back to the default state?
Shouldn't we treat the components' manifests as a sort of "reasonable" default and allow users to adjust them according to their needs?

I know we can go this way to be quite flexible. However, I'm more on the side that it is the operator CRDs that should expose the needed settings, rather than instructing the users to do it themselves.
IMHO, the more we allow for such flexibility, the more the operator becomes less valuable -- i.e. why not just provide the manifests to the user?
Personally, I'm quite sold to the explanation in operator framework website:

With Operators you can stop treating an application as a collection of primitives like Pods, Deployments, Services or ConfigMaps, but instead as a single object that only exposes the knobs that make sense for the application.

So, the general idea that this ADR is proposing is good to me.

Treating a component as a whole sounds like the right thing to do, I am not opposing that, nor do I suggest going back to "using primitives".  I fear that CRD can become bloated with different levels of abstraction. That's why I started thinking that perhaps we can open up certain aspects of our building blocks for the users to fine-tune instead of exposing those in our CRDs.
Is the suggested approach based on the assumption that the "user" is someone not knowledgable in the domain space, so instead of letting them fine-tune certain "low-level" settings, we are going to expose those low-level settings in some other place? I wonder if by doing so aren't we exactly exposing knobs?

Maybe, a middle ground is to expose in the CRD a high level setting like kserve.resources.expectedDeployedModels and similarly modelmesh.resources.expectedDeployedModels so that the operator does some math and adjusts the low-level settings of all impacted components (kserve, modelmesh, odh-model-controller, ¿etc?). But I don't think we have data to do this.

Will this still require us to expose component specific details?
Should we consider components to have a minimal and default config(overlays). This will not be limited to only resources.
spec.components.Kserve.config: minimal
OR
spec.components.Kserve.config: default

@israel-hdez However I think this will require us to have data for minimal resources required by a component.

@VaishnaviHire

Will this still require us to expose component specific details? Should we consider components to have a minimal and default config(overlays). This will not be limited to only resources.

My rationale tells me yes.

Should we consider components to have a minimal and default config(overlays). This will not be limited to only resources.
spec.components.Kserve.config: minimal
OR
spec.components.Kserve.config: default

I'm not sure overlays are suitable, because resources are something can need fine-tuning, and we already had got this feedback from the field.
This ADR is proposing to use the DataScienceCluster for that, and I think it is fine. Then, my understanding is that @bartoszmajsak is suggesting that it is better to whitelist fields of the resources so that the user can manually change those fields... this is also fine! My personal preference would be to do it in the DSC just because my understanding is that centralized configs is one of the added benefits of using operators (and this is what this ADR is proposing, so this is looking fine to me!).
In model serving we just need a mechanism to configure the resources. Whether this ADR is refined, or dropped in favor of whitelisting fields, or something else... that's more up to you :-)

I will update the ADR to reflect the discussion to provide list of fields to whitelist across all components. See PR for Kserve specific changes.

I have updated the adr to reflect implementation to whitelist fields across components
cc: @bartoszmajsak

@VaishnaviHire is this ready for merging?

Hi Yes, Just need lgtms from component owners


Publish to exchange

WHAT is this pull request doing?
Move the logic of publishing a message to an exchange from Vhost to the exchange.
The idea came because of the need for the exchange to handle the messages differently for MQTT support
but it has additional benefits like @queue_bindings and @exchange_bindings can have different types and as you may see I've used only one "bindings" for both queue and exchange. for example for FanoutExchange it's just a @bindings : Set(Destination)
There is probably some more things that should move down to each subclass to make the code clearer and to support the different types that @bindings has now.
HOW can this pull request be tested?
specs

@spuun @carlhoerberg any thoughts about always including "default" exchange when getting bindings for a queue heere: https://github.com/cloudamqp/lavinmq/blob/publish-to-exchange/src/lavinmq/vhost.cr#L286
instead of adding the default in the http controllers

Im fine with that

@spuun agree, early return is nicer. Fixed all now I thinkg

Specs fails because of this: https://github.com/cloudamqp/amqp-client.cr/pull/48


Reverse index

Would it be possible to add a "reverse index" so that I could lookup the tailwind classes using this docset? It would be helpful to be able to be able to search for "flex-col" and get the documentation like you can on the Tailwind site.

pretty sure it would be possible yeah. the TOC is generated here: https://github.com/godbout/tailwindcss-dash-docset/blob/c846336b4b46143110b470874e394885436301f9/app/Docsets/TailwindCSS.php#L240
so you would just need to grab whatever selector(s) for the attributes that you're looking for.
but:

that will pollute the index/search capabilities
i have zero time and mental capacity to do it and test, etc. currently

Got it. Thanks for the quick reply. And thanks for making this docset.

let's keep this issue open so that when/if i have time, i have a look.

Hi @jacobat,
If you are still interested in the functionalities, the DocSet available on Dash's user_contrib repository has the indexes you mentioned. See here for the details.

I'm no longer a Dash user. Thanks though.

docset has been taken over: https://github.com/godbout/tailwindcss-dash-docset/issues/34


Benchmark: messages which size is bigger than 50k

The benchmark report: http://emq-xmeter-benchmark-en.readthedocs.io/en/latest/


Add support for ELB attributes.

Enable support for retrieving and modifying ELB attributes such as connection draining, cross zone load balancing, idle timeout and s3 access logs.

This is great. Thank you!


Something about NFC

I read these TLV bytes from my credit card.
7081B49F4681B0A1C70DB39980B4E79F81D8730E6A511E306109DA40F3F07BC25EB55783C2783BE5CB6857599F0E5B9EEBB4BDDD3FB079D4645073708B54AE104C854442A17D4B4619D79AD6E4C8AD9641CC010FE145E538343F30668C8A28D6D7BA83661E4EAA5D668DA3DC6B6684BE5717E1DB9DC9D9E110F14937329B110B8C42E4FA2EFAB299C5D4B95F001CD4F93C3D9059ED971AE2DB038F6DEF92F5FC50A1864DDC0D92292F9F6618FB437078898CD55610EA2D9000
but got an out-of-range exception when extracting the TLV Data.
I find that the code doesn't recognize multi-bytes length field. So I try to fixed it.
And I implemented some function in TLVEntry and ApdbCommand.

@bin456789,
Thanks for your contribution.
To ensure that the project team has proper rights to use your work, please complete the Contribution License Agreement at https://cla.microsoft.com.
It will cover your contributions to all Microsoft-managed open source projects.
Thanks,
Microsoft Pull Request Bot

@bin456789, thanks for signing the contribution license agreement. We will now validate the agreement and then the pull request.
Thanks, Microsoft Pull Request Bot

Thanks. I've asked the NFC team to review your contribution.


AMICA 15 plugin on Debian 10

Transfered from https://github.com/sccn/eeglab/issues/136
Description
When using the following command to call AMICA, an error occurs on Debian 10:
[W,S,mods] = runamica15(EEG.data(:,:),'max_threads',1500,'max_iter',7);
eeglab14_1_1b/plugins/AMICA1.5.1/amica15c /Code/amicaouttmp/input.param: Segmentation fault
Something went wrong...
No gm present, setting num_models to 1
No W present, exiting
Reference to non-existent field 'W'.

Error in runamica15 (line 892)
    weights = mods.W(:,:,1);

The same error has occurred on 2 different computers with Debian 10. It does not happen when I use the exact same EEG data and the same command on Windows 10. Thus, it is possibly the binfile amica15c has not been updated for the Debian 10 OS.
Versions

OS version
Debian10

Matlab version
R2019a

EEGLAB version
14.1.1b

You might want to try to recompile AMICA for your platform (Debian). @japalmer29 might know more.
Arno

Hi, this error probably occurs because you are trying to use 1500 threads.
Maybe you meant max_iter? Anyway, could you try without a max_threads
argument and see if it runs?
Best,
Jason
On Tue, Mar 3, 2020, 12:53 AM Arnaud Delorme<EMAIL_ADDRESS>wrote:

Transfered from sccn/eeglab#136
https://github.com/sccn/eeglab/issues/136
Description
When using the following command to call AMICA, an error occurs on Debian
10:
[W,S,mods] = runamica15(EEG.data(:,:),'max_threads',1500,'max_iter',7);
eeglab14_1_1b/plugins/AMICA1.5.1/amica15c /Code/amicaouttmp/input.param: Segmentation fault
Something went wrong...
No gm present, setting num_models to 1
No W present, exiting
Reference to non-existent field 'W'.
Error in runamica15 (line 892)
weights = mods.W(:,:,1);
The same error has occurred on 2 different computers with Debian 10. It
does not happen when I use the exact same EEG data and the same command on
Windows 10. Thus, it is possibly the binfile amica15c has not been updated
for the Debian 10 OS.
Versions
OS version Debian10
Matlab version R2019a
EEGLAB version 14.1.1b
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/japalmer29/amica/issues/21?email_source=notifications&email_token=ACRBESRHWNWQLHVBC4UNEFLRFPI7ZA5CNFSM4K7XDGNKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IRXTDJA,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACRBESTOMIFCQOAR7YI6XF3RFPI7ZANCNFSM4K7XDGNA
.

Hi @japalmer29
I am sorry, that was a typo. I was actually doing:
[W,S,mods] = runamica15(EEG.data(:,:),'max_threads',7,'max_iter',1500);
The error was there when I ran this command.

I see. It is possible the the compiled Ubuntu version doesn't work in your
environment. But could you try running it from the command line using the
Memorize.fdt and amicadef.param files as described at the bottom of the
webpage http://sccn.ucsd.edu/~jason/amica_web.html ?
On Tue, Mar 3, 2020, 9:44 AM heidichiu<EMAIL_ADDRESS>wrote:

Hi @japalmer29 https://github.com/japalmer29
I am sorry, that was a typo. I was actually doing:
[W,S,mods] = runamica15(EEG.data(:,:),'max_threads',7,'max_iter',1500);
The error was there when I ran this command.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/japalmer29/amica/issues/21?email_source=notifications&email_token=ACRBESXFONYPOJ3XKAKNCBDRFRHHRA5CNFSM4K7XDGNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENRUDBY#issuecomment-593707399,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACRBESW7OPSD7FTW2WJWBKTRFRHHRANCNFSM4K7XDGNA
.

I have downloaded Memorize.fdt and the amicadefs.param.
In the terminal,
cd path/to/download/folder  path/to/amica15mac ./amicadefs.param
It returned "Segmentation fault".

Did you try amica15ubu, the Ubuntu binary version? Amica15mac is for Mac.
From: heidichiu<EMAIL_ADDRESS>Sent: Tuesday, March 3, 2020 4:04 PM
To: japalmer29/amica<EMAIL_ADDRESS>Cc: Jason Palmer<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [japalmer29/amica] AMICA 15 plugin on Debian 10 (#21)
I have downloaded Memorize.fdt and the amicadefs.param.
In the terminal,
cd path/to/download/folder path/to/amica15mac ./amicadefs.param
It returned "Segmentation fault".
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/japalmer29/amica/issues/21?email_source=notifications&email_token=ACRBESRVXVTU6Y2T7CM4QPTRFSTWXA5CNFSM4K7XDGNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENSKXFY#issuecomment-593800087 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACRBESQJRI5OVTYLZ6CD5NDRFSTWXANCNFSM4K7XDGNA .  https://github.com/notifications/beacon/ACRBESWLEOFW7BMO6DFPBL3RFSTWXA5CNFSM4K7XDGNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENSKXFY.gif

Sorry about that.
I have tried amica15ub and it works now.
However, in the runamica15.m, it uses the bin file amica15c, which does not work for me.
The original code in runamica15.m:
if ispc
    binfile = 'amica15mkl.exe'; % on computing
elseif ismac
    binfile = 'amica15mac';
else
    binfile = 'amica15c;
end

After I changed it to the following, the command runamica15()  works.
if ispc
    binfile = 'amica15mkl.exe'; % on computing
elseif ismac
    binfile = 'amica15mac';
else
    binfile = 'amica15ub;
end

So, is this the solution? Will there be any difference between amica15c and amica15ub?

Oh, ok the amica15c was compiled on Fedora Linux cluster maybe with incompatible options. I’m not sure how to distinguish Ubuntu/Debian, but we can at least make a help note to try amica15ub if amica15c doesn’t run.
Thanks,
Jason
From: heidichiu<EMAIL_ADDRESS>Sent: Tuesday, March 3, 2020 4:24 PM
To: japalmer29/amica<EMAIL_ADDRESS>Cc: Jason Palmer<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [japalmer29/amica] AMICA 15 plugin on Debian 10 (#21)
Sorry about that.
I have tried amica15ub and it works now.
However, in the runamica15.m, it uses the bin file amica15c, which does not work for me.
The original code in runamica15.m:
if ispc
binfile = 'amica15mkl.exe'; % on computing
elseif ismac
binfile = 'amica15mac';
else
binfile = 'amica15c;
end
After I changed it to the following, the command runamica15() works.
if ispc
binfile = 'amica15mkl.exe'; % on computing
elseif ismac
binfile = 'amica15mac';
else
binfile = 'amica15ub;
end
So, is this the solution? Will there be any difference between amica15c and amica15ub?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/japalmer29/amica/issues/21?email_source=notifications&email_token=ACRBESVZDXIUEO6S6WEHP7LRFSWCHA5CNFSM4K7XDGNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENSMGDQ#issuecomment-593806094 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACRBESQ2I53TIS4HXV4OQVTRFSWCHANCNFSM4K7XDGNA .  https://github.com/notifications/beacon/ACRBESSDGUVYI4SATGQBEKLRFSWCHA5CNFSM4K7XDGNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENSMGDQ.gif

Thank you very much for the help!

@japalmer29
/etc/os-release seems to be present on Centos/Ubuntu/Debian (and world readable) so perhaps inspecting that file is somewhat safe way to determine the OS of the Linux environment. (ID="centos").
Older non systemd environments might not have it but it appears even non-systemd debian has it. So it should be at least better than guessing at this point.

Hi
I've posted an issue I'm having when trying to use AMICA through EEGLAB (windows 10, 64-bit computer).
The post is in the following link:
https://github.com/sccn/eeglab/issues/217
would it be possible for anyone to help me?
Thanks in advance,
Anderson


Test and fix to cater for Windows/crlf format view files

Changed the whitespace regex to look like this.
MULTI_DENT = /^(?:\r?\n[^\r\n\S]*)+/
WHITESPACE = /^[^\r\n\S]+/
Added some tests parsing unix and windows files.

Nice, thanks!


Public APIs for chisel object names

From discussion with @chick, @ucbjrl, and @azidar, this change provides chisel obejct names for FIRRTL annotation and chisel testers. Chisel's internal HasId now extends Signald having four public methods:

signalName: returns the CHIRRTL name of the signal
pathName: returns the full path name of the signal from the top module
parentPathName: returns the full path of the signal's parent module instance from the top module
parentModName: returns the signal's parent module(not instance) name. This is not final until CHIRRTL is actually emitted because duplication checks are done in this stage.

To annotate chisel objects and pass them to FIRRTL, users may have containers of SignalId and create annotation for signals with signalName and parentModName after CHIRRTL is emitted by Chisel Driver.  Also, path names are useful for various cases including chisel testers.

retest this please

One concern I have with the general annotation strategy (which pathName and parentPathName in particular expose as a concern) is a superposition problem. This is related but more complex than simple aliasing. It is related (but not solely caused by) to the fact that annotations make it so that you can't rely on just the graph description to see if two modules are equal (even though many portions of chisel assume so.)
Annotations of the form parentModName.signalName are fine and simplest to handle directly. For a given named instance of something (wire, reg, module, etc.), it is intuitive to know to look at that entry in the map for that thing. The issue demonstrates for handling annotations of the form (parent.parentModName).(parent.signalName).signalName (and so on for parent.parent until you reach some ModuleDef). pathName is the special case of needing to reach the 'top' ModuleDef. Since the same ModuleDef can be used to describe multiple instances of a module, the annotations of specific nodes can be thought of as being 'in superposition.'
e.g. Circuit Top has 4 modules: Top, A, B, C that instantiate like a diamond
Both A and B instantiate a C and Top instantiates both A and B
Annotations refer to some range analysis:
  C.in says in is known to be <16
  Top.instA.instC.in says in is known to be >= 8
  Top.instB.instC.in says in is known to be < 8

Passes really need to know to 'decompress' the module definition listing into the actual module instances and analyze those separate instances with the proper annotations. Note that some annotations may apply to all, some, or only one instance depending on where the annotation is rooted. A parentModName.signalName always applies to all. A pathName applies to one unless the top has been modified). And, some other nested reference rooted in non-Top may refer to say half the instances.
I particularly label this as a concern because FIRRTL already has bugs where it fails to handle deduplicated modules properly in the width inferer. Going back to the above example, if C has an input with unknown width, width constraints from its use in both A and B are merged. (This actually leads to frightening results when that input then drives an output of unknown width. Now, one of those module instances is going to experience a silent/implicit truncation that is not obvious from local inspection.)
Actually, the symmetry between the problems can be further illustrated by doing width inference in two passes that transfer information via the annotations. The first pass analyzes the circuit and adds annotations to each node representing constraints. You expect most the annotations to be on/refer to parentModName.signalName.[signalPart] (for internal nodes) or (parent.parentModName).(parent.signalName).[IOpart] (for a submodule IO). Then, the second pass reads all those and resolves them. The bug is that it is 'flattening' all the latter references to just parentModName.signalName, even when the parent.parentModName are different.
Even if you legislate that all annotations must be rooted at the top definition (or add an analysis that replicates annotations such that they all are rooted at the top), this only solves an aliasing problem. You CAN further solve the superposition problem by replicate and renaming module instances until no annotated thing is in superposition. However, this could result in a lot of repeated work (and to some degree means Chisel shouldn't have bothered deduplicating in the first place). Also, since any pass could introduce new annotations (that could cause a superposition issue) or adjust module instantiations, you would need to run this analysis after every pass.

In writing up examples in my prior comment I noted an oddity in naming:
Given that Module <: HasId and therefore SignalId, Module has a use for signalName (it is in fact the name of it as the instance). However, it is somewhat odd for a Module to have a signalName. Instead I propose:
signalName -> instanceName
SignalId -> InstanceId.
It makes sense for wires and regs to have an instanceName as they are merely 'instances' of hardware registers or wires (like how in general software values could be considered instances of some type).


Add xiao-ble-battery binding and device tree objects

to to use your great library more in a zephyr specific way.
With these changes it is possible to configure your library without touching the library itself.
I hope you will like it
You have done a great work btw!

Yeah, it's definitely a typo 😅
I will fix it on Monday when I am back at work 😉
I am glad that you liked my changes!


Temperature comparison work

Hi @bpbond,
This is the code (not cleaned up yet) for comparing temperature data, Tgav specifically, from CMIP6 to the Hector run outputs.
This amends the run_hector() function to extract the big four Hector outputs in the run loop and return a new data frame called results in addition to the tracking data frame called output.
I'm having an issue with the graph itself. I read in CMIP6 data and can get a nice geom_ribbon() graph for it. But the Hector data won't plot as a ribbon (even though the structure of the two data sets is the same). The code currently binds the two together, but even when I tried the Hector data on its own, the plot was just blank. Any idea what might be going wrong? Thanks!

Thanks for your comments! I made the above adjustments.
Hmm, I think something is going wrong when I do group_by(run_number, year). The minimum and maximum are the same in each year within each run number. Do I need to group by run number? I thought yes at first but now I'm thinking maybe not, since we don't really care about the individual runs but the spread as a whole. What do you think?

^^ without grouping by run_number

😍  woo that is beautiful! Agreed, grouping by run_number doesn't make sense, and whoever told you to do that should be punished forthwith :)

Check is failing with message
processing file: trackingC.Rmd
[13](https://github.com/JGCRI/trackingC/runs/5660757484?check_suite_focus=true#step:7:13)
(*) NOTE: I saw chunk options "cmip, fig.cap = " Global temperature anomaly data from CMIP6 and Hector runs"
[14](https://github.com/JGCRI/trackingC/runs/5660757484?check_suite_focus=true#step:7:14)
 please go to https://yihui.org/knitr/options
[15](https://github.com/JGCRI/trackingC/runs/5660757484?check_suite_focus=true#step:7:15)
 (it is likely that you forgot to quote "character" options)

Yep, I forgot to close my figure caption 🤷♀ . Just waiting for a 1000x run to finish running (curious what the graph will look like) and then will fix that and merge.

Nice! Yes as one would expect. COOL.  @kdorheim 👀 ☝

I agree, I think that would be a good thing to add - is this too confusing of a graph color-wise, or can you tell what's happening?

@leeyap Oh, interesting. It does up the complexity but really nice!

That's really cool because it shows the Hector ±1 sd falls within the CMIP ±1 sd, even as the model runs a little cooler.

@bpbond do we want to tweak anything here, or is this okay to merge?

Looks great to me @leeyap ! Merge away :)


Feature request: add support for charset property

Description
Add logic to map charset property to ESLint rule.
As of right now, there is no implementation for the rule that checks file encoding but after some discussion on the issue in the tslint-to-eslint-config repo, some may be added soon. So I decided to create the FR a bit ahead of time just to not forget it.

Hi @Kydyralin, thanks for the issue!
Would you please expand on your proposal? How do you envision charset in editor configs mapping to an ESLint rule?
Note that at this point, I'm not sure I want to add rules to this plugin itself: the plugin's current mission is to export an ESLint core rule configuration which enforces JavaScript according to the editor config settings. If a core ESLint rule is created for enforcing character sets somehow, then I am happy to add support for that rule.

Hi!
Sorry for the late response. About your question: as mentioned in the description of this FR, we had a discussion on another repo about encoding tslint rule which, as of right now, has no implementation. Mr @criskrzysiu has answered that my issue created on the tslint-to-eslint-config repo probably isn't the best place for that and maybe I should create it here (since this repo suits well with charset config and encoding rule). Also, Mr @criskrzysiu said that maybe he could try to write the ESLint implementation for the rule so it would make sense to have FR to support it in this repo.
Hopefully, you get the idea of what we are trying to do. If not, feel free to ask anything and I'll try to explain it better.
Again, this FR is about adding support for the encoding rule only when the ESLint version will be implemented.

Sounds good @Kydyralin, that works for me.
If core ESLint declines a charset rule, let me know and we can consider implementing it here. Thanks!

Hi, I would suggest adding the rule here as it nicely fit here. Charset is
usually configured in the .editorconfig file and adding it in second place
like rule configuration could lead to errors. I think implementing it in
one rule that enforce editorconfig settings would be much better solution.
If You don’t have time to implement it Yourself I could try to create PR
basing of the charset rule in tslint.

Hi @criskrzysiu, the purpose of this plugin is to automatically generate a configuration of core ESLint rules based on an .editorconfig file.
So, even if the rule is implemented in core ESLint, this plugin will allow you to set your charset only in .editorconfig without also having to set it explicitly in your ESLint config.
Let me know if there's anything I can clarify further.


Use LazyLock over lazy_static

This would need an MSRV of 1.80.0 so it might have to wait a bit.
Sorry for the formatting changes, if you'd consider merging this I'm happy to revert those.
EDIT: it's also fine to only use the docs + test (as they already work perfectly)

@atezet Thanks for the PR!
Yes, could you please revert the formatting changes in the README and docs?
LazyLock is the long term way to go I love having it in, but I'd like to keep lazy_static and once_cell::sync::Lazy for a while, to avoid breaking changes for now and to give some time window for projects to migrate.

I reverted the formatting changes and split the PR in logical commits that only changes docs and examples and adds a test for LazyLock. In #164 I also included a commit that uses LazyLock in nutype_macros, which I think would be acceptable to include as I think it is still possible to use lazy_static or once_cell if one would want to depend on a lower Rust version, but feel free to leave that for later.

@atezet Thanks you!

@greyblake, @atezet You can actually start using it today already. Just feature gate on Rust version. Use 1.80 for new LazyCell and LazyLock and fallback to lazy_static to prior versions of Rust.

@zekefast We cannot silently fallback on lazy_static, it needs to be explicitly within dependencies of the project that uses nutype.
It probably works out of the box,and swapping lazy_static with LazyLock does not require anything, but I just want to have tests in place, to make sure that  lazy_static still works if some users use it.

@zekefast We cannot silently fallback on lazy_static, it needs to be explicitly within dependencies of the project that uses nutype.

That's true.

@greyblake The tests cover that now, right? So #164 would be mergeable too?


Replace get_pdb with get_file in readme (v2 API change)

The example in the readme was out of date. I just replaced it with what's in the lib.rs doc comment

Oops, thanks


fix LabelLayout async init used UILabel().

If we initialize the LabelLayout asynchronously with an NSAttributedString, most of the time we don't have to set the font, but the default access to UILabel(). Font, this is not safe.I don't quite understand why this is designed in this way.

@ghmake1y Thanks for your PR. Could you elaborate why UILabel() isn't safe? We want to match LabelLayout font the same as UILabel by default

@ghmake1y Does the test you added fail without your change?

@nicksnyder Yes,the test function named 'testAsyncAttributedLabel' is failed if i don't change default font.

@jingwei-huang1 You can see code of function testAsyncAttributedLabel in LabelLayoutTests.swit.If async call UILabel() will be crashed,it will be trigger Main Thread Checker: UI API called on a background.

@jingwei-huang1 UILabel() is not safe because LayoutKit can call it in background queue. When use LabelLayout in ReloadableViewLayoutAdapter, I saw Xcode complaint Main Thread Checker: UI API called on a background thread: -[UIView init] PID: 26099, TID: 2008031, Thread name: (none), Queue name: ReloadableViewLayoutAdapter (QOS: USER_INITIATED), QoS: 25.
You can reproduce this issue by running the LayoutKitSampleApp and open the "FeedCollectionViewController" example.

If using system font is not ideal, maybe we can replace it with UIFont.preferredFont(forTextStyle: .body)?


personal config: names for peers

So that names as known by a peer could be specified.

Use my x package

new commands:
peer export
peer import

ignored: translation system was too complex to use


Severity is blank in the report

Describe the bug
In the report (HTML or JSON), the severity is blank. So the vulnerabilities for a given severity are hard to find.
To Reproduce
Steps to reproduce the behavior:

Scan a project (Tested with Java)
Read the report.html, in the severity field for each vulnerability.

Expected behavior
Severity should be filled.
Screenshots

Desktop (please complete the following information):

OS: Linux
Version 3.0.0 (Release insider_3.0.0_linux_x86_64.tar.gz)

I have observed the very same behaviour on my end.


✨ [Feature request]: Play statistics: View Count

Describe the solution you'd like
One of the best ways to show play engagement is through "view count". Good to have the stat implemented. Later point of time this data can be used for sorting mechanism
Describe alternatives you've considered
NA
Additional context
Need a UI suggestion before starting the imeplementation.

Its a good one. So prereq is to get the meta in DB?

@atapas me and @vasantisuthar would love to take it up, as it's a part of what we are building

Sure, take it up


Why med dao section

Implements the new WhyMedDAO section

Looks like this so far.
Related todos on Mobile:

[ ] padding spacing, colors - some weird behaviour between slick js and chakra
[ ] add the fancy animated dots
[ ] font styles for the cards

Unrelated issues

[ ] overflow issues with the MainColumn


I'm fine going right into master and integrating to dev


chore: Remove chance lib

This PR is part of a series where we refactor and remove unused libs from the codebase.
Type of change

Refactor

How Has This Been Tested?

Don't know to test this

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Test coverage results :test_tube:
:white_circle: Total coverage has not changed
No changes to code coverage between the base branch and the head branch

/ok-to-test sha=d9fd1d5

@jsartisan I think we need to update the yarn.lock to remove this as well. Could you push the updated yarn.lock file as well?

/ok-to-test sha=60f19ea

/ok-to-test sha=60f19ea

/ok-to-test sha=60f19ea


Can't get c++ semantic completion.

@abingham @ptrv
My flags in .ycm_extra_conf.py.

flags = [
'-Wall',
'-Wextra',
'-Werror',
'-fexceptions',
'-DNDEBUG',
'-std=c++11',
'-x',
'c++',
'-isystem',
'd:/appdata/Microsoft Visual Studio 10.0/VC/include',
]

I put it  in E:\download\P2P-master\P2P-master\P2PDemo.
And my test project is in E:\download\P2P-master\P2P-master\P2PDemo\P2PClient.
In my init-file.
(set-variable 'ycmd-global-config
			  "D:/appdata/emacs/extraBin/ycmd/cpp/ycmd/.ycm_extra_conf.py")
(set-variable 'ycmd-extra-conf-whitelist
			  '("e:/*"))
(setq ycmd-extra-conf-handler 'load)

In a cpp file of this project, in which I included stdio.h, I tried to type sca and expected scanf, but it failed.
There are something different in c file. I created test.c in this project and tested. As you see in the screenshot.

I was happy! I used sca to test in this file soonly. Unfortunately, I failed again. Back to the cpp file, I used vector to test once more and get many macro completion instead of its member function.

And this maybe helpful.

If you enter "sca" and then run ycmd-display-completions, does this show "scanf" in the results?

((completion_start_column . 2)
(completions .
[])
(errors .
[]))

Which compiler are you using?

There are clang and clang++. I don't know which one that ycmd use.

You might check this issue with ycmd.


[Mobile] The size of "Preparing your wallet" pop-up doesn't match the design

Steps to reproduce:

Click the "Agree&continue with Google" button on the mobile device
Select the mailbox
Pay attention to the size of pop up

Actual result: The size of "Preparing your wallet" pop-up doesn't match the design
Expected result: The size of "Preparing your wallet" pop-up should match the design and be less
Checked on DEV V1.0.0
LG V30+ // Android 8.0 // Google Chrome 84.0.4147.111

@AnastasiiaOdnoshevna is this specific for android? how does it look on desktop and on iOS?

@sirpy  iPhone 11 | iOS 13.3

Desktop Windows 10

@AnastasiiaOdnoshevna so its android specific?

@AnastasiiaOdnoshevna for me on my android it looks fine. (gooddev)

@AnastasiiaOdnoshevna this spinner animation was removed as it caused the #2280 bug.
it was replaced by a simple spinner. please retest.

@sirpy the pop-up is displayed according to the requirement
Проверено на DEV 1.0.6-0 и NEXTorg V1.0.5
LG V30 + // Android 8.0 // Google Chrome 84.0.4147.111
iPhone 8 | iOS 13.3.1


Paths to files are not resolved with realpath before opening

 Environment data

Language Server version: 2023.7.10
OS and version: linux x64
Python version (and distribution if applicable, e.g. Anaconda): 3.11.4
python.analysis.indexing: true
python.analysis.typeCheckingMode: off

 Code Snippet
pyproject.toml
[tool.pyright]
extraPaths = ["development"]

a.py
def f():
    pass

b.py
from proj.a import f

folder structure
$ tree
.
├── a.py
├── b.py
├── development
│   └── proj -> ..    # this is a symlink to root project dir
└── pyproject.toml

 Repro Steps

Open a.py and b.py in VSCode
Press "Go to definition" on f in b.py

 Expected behavior
VSCode switches to previously opened a.py tab.
 Actual behavior
VSCode opens another tab for file development/proj/a.py. It is the same a.py but now you have it opened in two tabs and you can start to modify both at the same time and face troubles.

Thanks for the issue but I believe this is by design. VS code doesn't follow symlinks and neither does Pylance. We don't run any python code to analyze the file, so this is just node.js opening the file for us.
See this issue:
https://github.com/Microsoft/vscode/issues/57873
You might try the symlink follow extension.
You can repro this without pylance being involved by just opening the two files in VS code. Changes are not reflected in each until you save.

Okay, thanks, Symlink Follow extension with "symlink-follow.autoFollow": true setting solves my issue.

How is this expected though? If VS Code doesn't follow symlinks, then how would it open a symlink to the target file when you expect it to open the target file?

Actually, I think this is a bug. We're likely passing back the wrong URI for the file during goto defintion. See https://github.com/microsoft/pylance-release/issues/5136


Question about Size

Hi,
I want to make the signature pad a little bit bigger. I changed the width/height of the canvas, then I changed the CSS width/height. The size of the pad grows, but when i goto sign my name, I can only use like half of the canvas vertical instead of the whole things. Any idea on how I can make this work.
Thanks

What browser are you using? Also, how did you resize it?
I did this and it works fine under Android / Chrome / Firefox / Safari.
GOOD: <canvas class="pad" width="600" height="200" style="border: 1px solid black"></canvas>
I tried defining the dimensions with CSS and it is not working as intended.
NOT GOOD: <canvas class="pad" style="border: 1px solid black; height: 200px; width: 600px"></canvas>
Try setting it with html attributes instead of CSS, it works fine :)

Yeah, you'll need to set the dimensions in HTML:
<canvas class="pad" width="500" height="550"></canvas>

Then, if your using the template CSS, you will likely have to adjust the CSS so the wrapper fits properly:
Line 24:
.sigPad { width: 502px; }
Line 149:
.sigWrapper { height: 550px; }


seata1.3 AT 模式性能测试 - 分支事务一直尝试回滚一直提示回滚失败

[ ] I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description
测试环境：
Ⅱ. Describe what happened
If there is an exception, please attach the exception trace:
Just paste your stack trace here!

Ⅲ. Describe what you expected to happen
Ⅳ. How to reproduce it (as minimally and precisely as possible)

xxx
xxx
xxx

Ⅴ. Anything else we need to know?
Ⅵ. Environment:

JDK version : jdk 1.8
OS : macos 10.15
Others: springcloud:Hoxton.SR3   springboot:2.2.5.RELEASE. spring-cloud-alibaba.version: 2.2.1.RELEASE

鹅


ETA on the code?

Very excited to see how this is implemented. The paper does not cover enough detail to replicate in my opinion :)
When will the code be released?

Hi — thanks for the interest! We're working on releasing the code, and should have it out in two weeks or sooner.


Setting OperationTelemetry.Success to false without setting ResponseCode causes the service to silently drop your events

Runtime version netcoreapp3.1, however this appears to be service-side
Hosting environment: Windows

Describe the bug
When using the TelemetryClient.StartOperation/StopOperation() pattern, if you set the operationTelemetry.Success = false, without setting operationTelemetry.ResponseCode, the event is successfully uploaded to Azure Application Insights, but then it is silently dropped from metrics and log analytics.  If you set operationTelemetry.ResponseCode to a non-null value, then the events are available.
This is problematic because many of the code samples show setting Success == false without setting any ResponseCode, and this requirement is not called out in the SDK documentation.
Perhaps the SDK could automatically set this to a non-null value when Success is set to false.
Note: Using _telemetry.TrackException() does not set Success or ResponseCode.  Perhaps operation should have its own TrackException, or there should be a StopOperation(Operation op, Exception ex) overload?
To Reproduce
using var operation = telemetryClient.StartOperation("Example");
operation.Telemetry.Success = false;
telemetryClient.StopOperation(operation);
telemetryClient.Flush();

https://github.com/microsoft/ApplicationInsights-dotnet/issues/2082 possible duplicate

Seeing the same issue in a similar pipeline with .NET 4.7.2 (AI 2.19.0).
In my case, I'm using a telemetry initializer to set the success property of RequestTelemetry items to false based on context information. For the local pipeline (console tracing), the request shows up correctly and with the correct result. For the remote pipeline (server telemetry channel), the request disappears (its children can still be found under the parent operation).
Setting a response code, e.g. "500", appears to resolve the problem.


Custom headers looking weird when using Romanian characters

Hello, it seems that the current font doesn't support some characters from the Romanian language. When using characters like ă, â, ș etc. the headers look like the following:

I'm running the latest (as of 16 of november with the 2.0.0 update) version of Flame using the multiarch tag from docker. I've tested this on Firefox 94 on Windows and Linux (Mozilla/5.0 (X11; Linux x86_64; rv:94.0) Gecko/20100101 Firefox/94.0), also on Chromium 96 (Version 96.0.4664.45 (Official Build) Arch Linux (64-bit)).
I think this could be fixed by having an option to change the font for the headers or by replacing the default font with something else.

You can set custom font using CSS:
* {
  font-family: "Comic Sans MS", sans-serif !important;
}

Here is the list of web safe fonts:
Arial
Arial Black
Verdana
Tahoma
Trebuchet MS
Impact
Times New Roman
Didot
Georgia
American Typewriter
Andalé Mono
Courier
Lucida Console
Monaco
Bradley Hand
Brush Script MT
Luminari
Comic Sans MS

All right, I guess using custom CSS works too. Thanks for the help!


Integrate labeling algorithms

This means, ability to generate our own atlases...our desikan atlases currently come from JIST via long-deprecated code. Not sure this is high priority, but a small capability gap. Basically involves registering labeled brain(s) with T1 to MNI space. -Will

migrated.


libmproxy proxy.ProxyConfig - ignore_hosts

its not possible to add only one host in proxy.ProxyConfig
libmproxy/proxy/config.py line 55:
ignore_hosts=tuple('.*')

libmproxy/proxy/config.py line 21:
HostMatcher
self.patterns = list(patterns)

when cast tuple to list
the result is ['.', '*']
instead ['.*']

workaround is set '.*' twice in ignore_hosts but its ugly...

Thanks for the report - this is an error on your end though:
>>> tuple('.*')
('.', '*')

The correct way to specify this would be ('.*',) or tuple(['.*']).
libmproxy/proxy/config.py#L55 currently contains an empty tuple, which works fine.
            ignore_hosts=tuple(),

Alternatively, you can just pass a list here, you just avoid that as a the default argument in Python because it's mutable (see http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments).


Add save predictions and measurements to documentation

Closing PR as there is newer and improved documentation on this topic


Return TableProviderFilterPushDown::Exact when Parquet Pushdown Enabled

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and why for this feature, in addition to  the what)
Currently even when parquet predicate pushdown is enabled, and the predicate can be fully pushed down, the physical plan still contains a FilterExec when using ListingTable
| physical_plan | ProjectionExec: expr=[service@0 as service, host@1 as host, pod@2 as pod, container@3 as container, image@4 as image, time@5 as time, client_addr@6 as client_addr, request_duration_ns@7 as request_duration_ns, request_user_agent@8 as request_user_agent, request_method@9 as request_method, request_host@10 as request_host, request_bytes@11 as request_bytes, response_bytes@12 as response_bytes, response_status@13 as response_status]                            |
|               |   CoalesceBatchesExec: target_batch_size=4096                                                                                |
|               |     FilterExec: container@3 = backend_container_0 OR pod@2 = aqcathnxqsphdhgjtgvxsfyiwbmhlmg                                                                                |
|               |       RepartitionExec: partitioning=RoundRobinBatch(8)                                                                                |
|               |         ParquetExec: limit=None, partitions=[home/raphael/Downloads/data.parquet], predicate=container_min@0 <= backend_container_0 AND backend_container_0 <= container_max@1 OR pod_min@2 <= aqcathnxqsphdhgjtgvxsfyiwbmhlmg AND aqcathnxqsphdhgjtgvxsfyiwbmhlmg <= pod_max@3, projection=[service, host, pod, container, image, time, client_addr, request_duration_ns, request_user_agent, request_method, request_host, request_bytes, response_bytes, response_status] |
|               |

Describe the solution you'd like
ListingTable::supports_filter_pushdown should return TableProviderFilterPushDown::Exact when

Parquet predicate pushdown is enabled
The FileFormat is parquet
The predicate is fully pushed down by ParquetExec (not all predicates are supported)

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

This is a good proposal I think -- it would skip unecessary filtering and likely make plans faster. It will become more useful (maybe even necessary) when predicate pushdown is enabled by default

I think it is a good idea. I confirm it would help improve performance by 30x in https://github.com/apache/arrow-datafusion/issues/5404#issuecomment-1501221831.

I am thinking to pick this up but seems the work is not as trivial as I thought. Following are my two cents' thoughts.
Related code to improve is following
https://github.com/apache/arrow-datafusion/blob/7545177001b9dc04951f0c1c2008509f3895de8e/datafusion/core/src/datasource/listing/table.rs#L736-L756

The FileFormat is parquet

This is trivial, we can match self.options.format with ParquetFormat

Parquet predicate pushdown is enabled

This setting could come from ParquetExec.pushdown_filters or ConfigOptions (more specifically, ConfigOptions.execution.parquet.pushdown_filters)
https://github.com/apache/arrow-datafusion/blob/7545177001b9dc04951f0c1c2008509f3895de8e/datafusion/core/src/physical_plan/file_format/parquet.rs#L393
However, when ListingTable.supports_filter_pushdown is called, ParquetExec is not created yet, nor does SessionState being passed as input.
Idea: shall we add state: &SessionState as input for ListingTable.supports_filter_pushdown, the same as ListingTable.scan ❓
https://github.com/apache/arrow-datafusion/blob/7545177001b9dc04951f0c1c2008509f3895de8e/datafusion/core/src/datasource/listing/table.rs#L668-L670

The predicate is fully pushed down by ParquetExec (not all predicates are supported)

is it making sure the row_filter is built from the predicate without error? If so, seems it requires converting a logical Expr to PhysicalExpr first.
https://github.com/apache/arrow-datafusion/blob/7545177001b9dc04951f0c1c2008509f3895de8e/datafusion/core/src/physical_plan/file_format/parquet.rs#L527-L547

I wonder if it might be possible to always return exact for Parquet files, and to just manually insert a FilterExec for any predicates that can't be pushed down
Tagging @alamb and @crepererum who are more familiar with this part of the codebase

I'll try and look into this in more detail tomorrow

I wonder if it might be possible to always return exact for Parquet files, and to just manually insert a FilterExec for any predicates that can't be pushed down

That's what we want to do for InfluxDB IOx as well: https://github.com/influxdata/influxdb_iox/issues/7408

I am going to find time sometime this week

Background Reading
Here is a background article about parquet predicate pushdown:  https://www.influxdata.com/blog/querying-parquet-millisecond-latency/#heading5
Specifically the section on Late materialization describes what is done in the RowFilter code
Code links for Listing Table
Supports filter pushdown: https://docs.rs/datafusion/latest/datafusion/datasource/listing/struct.ListingTable.html#impl-TableProvider-for-ListingTable
Conditions:
Here are the conditions @tustvold  lists above, and some links about what they mean

Parquet predicate pushdown is enabled

This refers to the datafusion.execution.parquet.pushdown_filters configuration setting (docs here)

The FileFormat is parquet

This should be relatively straightforward to determine

The predicate is fully pushed down by ParquetExec (not all predicates are supported)

"pushdown" refers to this code in ParquetExec (there is some background information in the parquet RowFilter API)
https://github.com/apache/datafusion/blob/16a3557325eb8f949f4a87ab90c0a0b174dc8d86/datafusion/core/src/datasource/physical_plan/parquet/row_filter.rs#L43-L70

I think there are likely two things needed for this PR:

Some API that ListingTable can use to determine if the predicates can be pushed down
Tests that show / confirm that using RowFilter really is an exact filtering

FWIW I think @itsjunetime  is looking into this ticket

Yes, sorry - I'm still somewhat new to this codebase, so I'm taking a good second to read through the associated docs and comments to figure it out first.

No need to rush. I think @alamb's comment was mostly meant as an assignment signal, so that nobody else starts to work on it and we end up wasting resources 🙂

No need to rush. I think @alamb's comment was mostly meant as an assignment signal, so that nobody else starts to work on it and we end up wasting resources 🙂

Yes this is what I meant. Sorry about not being clear.

I filed https://github.com/apache/datafusion/issues/12115 to track some additional testing that I would like to do for this feature.

There is a nice PR here for this feature: https://github.com/apache/datafusion/pull/12135


Advance User Search

User Story
Search Users

Epic Title
#1111 Search

T-shirt size: SML

User Story
As a Next, I should be able to search for users, and get back the roles  that match the  user's manager fields

Description
Search for managers rapidly to get users displayed.  This may require modification of the JSON and consideration on sooting or display in the front end.

Acceptance criteria
able to issue a search request and get back a correct search response for users.  Used in the Browse page.  Tests and Updated swagger associated

migrated to new tmobile/sawtooth-next-directory repo


Container App Jobs (preview) deployment fails with cancelled error message.

Please provide us with the following information:

This issue is a: (mark with an x)

[x] bug report -> please search issues before submitting
[ ] documentation issue or request
[ ] regression (a behavior that used to work and stopped in a new release)

Issue description
Deployment of ContainerApp Job via bicep/ARM fails to succeed but the job is created and functions as epxected.
Steps to reproduce
Example bicep file
@description('Location for all resources.')
param location string = resourceGroup().location

resource appEnvironment 'Microsoft.App/managedEnvironments@2022-10-01' existing = { name: 'my-env' }

resource job 'Microsoft.App/jobs@2022-11-01-preview' = {
  name: 'my-job'
  location: location
  properties: {
    environmentId: appEnvironment.id
    configuration: {
      scheduleTriggerConfig: {
        cronExpression: '*/15 * * * *'
        parallelism: 1
        replicaCompletionCount: 1
      }
      replicaRetryLimit: 1
      replicaTimeout: 60
      triggerType: 'Schedule'
    }
    template: {
      containers: [
        {
          image: 'mcr.microsoft.com/k8se/quickstart-jobs:latest'
          name: 'main'
          resources: {
            cpu: json('0.25')
            memory: '0.5Gi'
          }
        }
      ]
    }
  }
}

Deployment command:
az deployment group create --resource-group "<rg-name>" -f job.bicep --confirm-with-what-if

Expected behavior [What you expected to happen.]
The deployment should succeed since the job has been created and it functions as expected.
Actual behavior [What actually happened.]
The job resource is created but the deployment fails with this error message:

The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Canceled'
CorrelationID: ca496e80-fc5c-4a5b-a013-1471cca2da30

Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Ex. Did this issue occur in the CLI or the Portal?
This happens for when using ARM templates (json/bicep)

Hi @mburumaxwell ,
I'm having the same issue. If you have any update on this let me know.
Thanks

Hi, We found a bug in our ARM validation when invoked from Bicep. We are working on a fix.

Hi @vinisoto,  could you let me/us know if you rolled out the fix? Is anything necessary on the client side?

This is still not working (tried Friday and a few minutes ago).
Start time: 2023-06-13T11:16:13+03:00
Correlation ID: 52e378f2-36a0-41f7-b2c4-aad3a0ea910d

Thanks @mburumaxwell. Is there no way currently to deploy Azure Container App Jobs using Bicep, or does it fail under certain circumstances? cc @vinisoto

I also have another issue: if I try to deploy a job using az containerapp job create, it works as expected. But if I try to assign the resource id of an existing user-defined managed identity using the --user-assigned parameter, it fails with a strange error message as I have owner permissions on my Azure subscription, so I should be able to make assigments:
#!/bin/bash

# Variables
prefix="Pluto"
name="${prefix}SenderJob"
managedIdentityName="${prefix}WorkloadManagedIdentity"
resourceGroupName="${prefix}RG"
environmentName="${prefix}Environment"
triggerType="Manual"
imageName="paolosalvatori.azurecr.io/sbsender:v1"

# Retrieve the resource id of the user-assigned managed identity
echo "Retrieving the resource id for [$managedIdentityName] managed identity..."

I also have another issue: if I try to deploy a job using az containerapp job create, it works as expected. But if I try to assign the resource id of an existing user-defined managed identity using the --user-assigned parameter, it fails with a strange error message as I have owner permissions on my Azure subscription, so I should be able to make assigments:
#!/bin/bash

# Variables
prefix="Pluto"
name="${prefix}SenderJob"
managedIdentityName="${prefix}WorkloadManagedIdentity"
resourceGroupName="${prefix}RG"
environmentName="${prefix}Environment"
triggerType="Manual"
imageName="paolosalvatori.azurecr.io/sbsender:v1"

# Retrieve the resource id of the user-assigned managed identity
echo "Retrieving the resource id for [$managedIdentityName] managed identity..."

I solved by installing the latest update of the containerapp extension for Azure CLI that didn't return any error

I still get this error, anyone with an update?


fix tests

This PR applies the changes in https://github.com/nautiluslabsco/ergo/pull/49 to the http path, and makes src.Payload json serializable by way of a new src.serialization.serialize function.
It also adds a new failing test, and I'm not sure how to fix it.  This test publishes a message which is consumed by component dispatch_greetings, and the results are consumed by a second component greet. To conform to https://github.com/nautiluslabsco/ergo/pull/49, the test's original payload is a serialized dict that simply maps arguments to dispatch_greetings's params, without nesting them in a data key. But the amqp_invoker still nests its outbound data. Later on, ergo fails to parse the arguments to greet here, because this time Payload._data looks like {"data": "Alice"} instead of {"name": "Alice"}. One solution would be to ensure that function_invocable.invoke returns a Payload object, which we can update in-place here instead of yielding a new dict. But if the component function returns a scalar (or anything besides a dict), then it's not clear to me what we should do here to convert it into a new Payload. It seems like the only way to construct this Payload out of key-value pairs would be to depend on something like
returns:
    0: name

in a dispatch_greetings.ergo config.

One other point - I know that the outbound (returned) data is still being mapped into a data element and I will fix that in a subsequent PR.   I didn't want to include too much in the first PR - rather do it in bite sized chunks and keep them reviewable.   Also wanted to keep the scope of the change to the problem we were already discussing.   But the outbound data binding and a host of other changes are on the list for refactor very soon.

The primary upstream issue here is that where data_in is serialized in the exception block, it should not include config/context.   That has to be omitted from the outbound payload.

@mattian7741 I merged your last commit, and removed the deprecated serializer.serialize.


Need a way to opt-out of tag helper at pre-compile time.

If an HTML element is a tag helper it will be code genned as a tag helper no matter what.  We need to add a way for a tag helper to provide a way to "opt-out" of tag helperifying an html element at compile time.

my 2c this can happen post RTM.

This item is not possible until we have more concrete support in the DTH for TagHelpers/executing user code.

We shouldn't rely on user code in dth. Lets talk
-----Original Message-----
From: "N. Taylor Mullen"<EMAIL_ADDRESS>Sent: 1/26/2015 5:00 PM
To: "aspnet/Razor"<EMAIL_ADDRESS>Cc: "Yishai Galatzer"<EMAIL_ADDRESS>Subject: Re: [Razor] Need a way to opt-out of tag helper at pre-compile time.(#95)
This item is not possible until we have more concrete support in the DTH for TagHelpers/executing user code.
—
Reply to this email directly or view it on GitHub.=

Pinging this because we still need to talk :smile:

Talked in person, I think we can skip this for now.

Closing because there are no plans to implement this.


[TACHYON-732]Metrics BlocksEvicted is not correct

https://tachyon.atlassian.net/browse/TACHYON-732
Update metrics BlocksEvicted when a block is moved.

+1 LGTM   the check failure is due to failing to fetch remote repo.

Jenkins, test this please.

I guess this depends on the BlocksEvicted semantic, if it is demoted from a tier, does it count as evicted? It seems strange because it is still in Tachyon storage.

May be better to have a separate BlocksDemoted semantic?

Thanks! Adding a BlocksDemoted seems to be clear as I do want to know how many blocks are moved to next tiers. Ok, I'll close this one and leave 0.7 as is. I'll open a new PR to master branch.


Transport constructor should accept undefined "level" instead of defaulting to "info"

By default, we have no way to control the level of all transports of a logger through the logger's "level" property, because in transport.js, the Transport constructor defaults its "level" property to "info", causing the transport's level always to be used instead of logger's level.
This logic is implemented in the log function in logger.js as this:
  for (var i = 0, l = this._names.length; i < l; i++) {
    var transport = this.transports[this._names[i]];
    if ((transport.level && self.levels[transport.level] <= self.levels[level])
      || (!transport.level && self.levels[self.level] <= self.levels[level])) {
      transport.log(level, msg, meta, function (err) {
        self.emit('logging', transport, level, msg, meta);
      });
    }
  }

and in transport.js:
var Transport = exports.Transport = function (options) {
  events.EventEmitter.call(this);

  options               = options        || {};
  this.level            = options.level  || 'info';
  this.silent           = options.silent || false;
  this.handleExceptions = options.handleExceptions || false;
};

So I think it's better to leave transport's level to undefined in its constructor rather than defaulting it to 'info'.

@cavingdeep I think you're misreading the code:
 if ((transport.level && self.levels[transport.level] <= self.levels[level])
      || (!transport.level && self.levels[self.level] <= self.levels[level])) {

Reads as: "If the transport has a level and that level is below the level to be logged OR the transport does not have a level and the level set on the logger instance is below the level to be logged"
So if you set .level on all transport instances associated with a logger their levels will always be preferred. The two statements are mutually exclusive. If you could provide a failing test here I can elaborate more.


[compat] RATS MOD -  net.minecraft.crash.ReportedException: Rendering entity in world

I realize that this mod is still very much a work in progress and i very much appreciate all the hard work being done to bring morph to forge.
I want to help test as many mobs/morphs as possible to help with compatibility. I am working on adding morph abilities to the rats.. the rats:rat entity seems to work just fine although i have not tested variations.. this crash happened when trying to morph into a rats:plague_beast
crash-2021-08-03_10.49.29-client.txt
https://mclo.gs/tMy3tDD
You should make a discord server or something! this is outstanding work ❤
forge-36.2.0
bmorph-1.16.5-1.3.1.jar
rats-7.2.0-1.16.5.jar
citadel-1.7.2-1.16.5.jar

Alright, gonna have a look at this. Thanks for helping me developing and improving this mod.

Would like to add that this doesnt happen with just rats..but ocassionally other mobs..and if it does happen ,the player can no longer join the server until i wipe their playerdata....where are morphs stored?  what part of word/playerdata/<uuid>.dat has the list of morphs a player has?

I think it is in ForgeCaps/bmorph:morph_cap. You should be able to just spawn in a demorphed state by yeeting the integer "currentMorphIndex" and "currentMorphItem" (if present) out. (I hope this helped). This reminds me that I should probably fix this issue that you spawn in as a random mob when exiting and entering a world again...

I've also started working on some modded mob abilities with my friends here

This issue should be fixed now with the latest release on CurseForge.


Add project property file to Spring Monitor

This file could be used after a consensus has been reached on this specification PR: https://github.com/aep-health-and-standards/Telemetry-Collection-Spec/pull/286

API change check
API changes are not detected in this pull request.


swap order of configure commenting and notifications

Current Behavior
In the Project Settings, "Configure Commenting" is below "Configure Notifications."

Suggested Behavior
Because notifications are a result of commenting (at least at the present moment, the notifications are dependent on comments being left), they should be listed after the commenting configuration panel.

The suggested order change will be in effect in https://github.com/CenterForOpenScience/osf.io/pull/3397. See https://github.com/CenterForOpenScience/osf.io/pull/3397#issuecomment-133438729


Update header.png

Updated "service" to "software" on image text
Hey, I just made a Pull Request!

The header image on the software catalog page was out of date, stating "service definition" and "Backstage service catalog". I have updated the image to now state "software" in place of "service.
:heavy_check_mark: Checklist

[ ] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ x] Screenshots attached (for UI changes)
[ x] All your commits have a Signed-off-by line in the message. (more info)

@sevedkim Hi! Thanks. Remember to DCO sign all your commits - see the check details or the comment above.

Hi @sevedkim, is there any chance we could also update the image of Backstage on the Laptop? Those tabs under the main header aren't there any more, they've been gone for a long time now 😉


Explain why you need to choose an eth1 client

It's not intuitively obvious to someone staking on eth2 why they need to run an eth1 client.
Could do with a simple explanation at the top of the Select Client page.

Can't seem to self-assign myself, but will try and post something here today

@CarlBeek do you think it's worth mentioning Infura (and explaining the convenience vs decentralisation tradeoff)?

I don't think so. If someone is prepared to configure the infura API and to pay for its use, then IMO, they are welcome to.

To make sure i understand, it's not worth us mentioning Infura on this page?

closing in favor of #197


Create a new ReactRenderer for @tiptap ReactNodeView

Is your feature request related to a problem?
After upgrading to React 18, the @tiptap react-renderer encounters an async render problem https://github.com/ueberdosis/tiptap/pull/2985
Describe the solution you'd like
To make react renderer work well with Prosemirror, we need to build a new ReactNodeView renderer instead of @tiptap's one.
Describe alternatives you've considered
Fixed the problem by patching the @tiptap package but is not perfect. As a consequence of the implementation of the @tiptap react-renderer, react will through an annoying warning about flushSync usage.
Additional context
Making react 18's new render mode work with Prosemirror correctly may be a siginficant part of this refactor.

ref: https://github.com/remirror/remirror Remirror has upgraded to react 18 already and looks to work well.

[ ] Rename packages/editor to packages/legacy_editor, create new packages/editor. Prepare for editor migration: https://github.com/mashcard/mashcard/discussions/276
[ ] Create NodePortal NodePortalContainer in packages/editor for creating react portal in prosemirror node view.
[ ] Fork @tiptap/react into packages/legacy_editor
[ ] Replace @tiptap/react ReactRenderer and other stuffs with NodePortal.

cc @aligo @clszzyh @stackia @melchior-voidwolf
packages/editor is about to rename to packages/legacy_editor

After comparing remirror and tiptap, I think the problem is that tiptap still uses react legacy class component API.


Cannot get Laracast example to fire MAC OSX Mavricks

Hi,
I create the cron command file and start crontab.  But it never fires.  If I enter the command manually, it works.  When do crontab -l it shows the job????

php /Users/jimm/WebProjects/laravel42/artisan scheduled:run 1>> /dev/null 2>&1

@jimmck I has same problem with you. The command works when I run it in terminal. Please help me.

Which crontab did you add it to?  The root crontab or a specific user's crontab?

I create a cron.txt in my folder project and using crontab cron.txt command

Which user are you running that command as?  Can you also provide the contents of the cron.txt?

This is my cron.txt file:
* * * * * php /Users/mac/Code/myproject/artisan scheduled:run 1>> /dev/null 2>&1

Hi,
I had to put the path in my cron file.  Once done done, all works fine.
MAC OSX, Laravel 4, PHP 5.6.3
Here is my file:
SHELL=/bin/bash
PATH=/Applications/:/Applications/XAMPP/xamppfiles/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin

php /Users/jimm/WebProjects/laravel42/artisan scheduled:run 1>>
/dev/null 2>&1

Regards,
Jim
On Sat, Jan 10, 2015 at 2:34 PM, phuclh<EMAIL_ADDRESS>wrote:

This is my cron.txt file:

php /Users/mac/Code/myproject/artisan scheduled:run 1>>
/dev/null 2>&1

—
Reply to this email directly or view it on GitHub
https://github.com/Indatus/dispatcher/issues/67#issuecomment-69468675.

@phuclh Which crontab did you setup that crontab on?  If it's on the root crontab then that cron command is invalid and will not run.  It's on the root crontab if you added while either....

Being logged in as root
used sudo crontab or some other sudo to edit/install the cron

I edit my cron.txt file like:
`SHELL=/bin/bash
PATH=/Applications/:/Applications/XAMPP/xamppfiles/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin

php /Users/mac/Code/myproject/artisan scheduled:run 1>> /dev/null 2>&1`

And It fired :D.
Thank you very much @jimmck @bkuhl

Cool !!! Enjoy!
Jim
On Sat, Jan 10, 2015 at 2:59 PM, phuclh<EMAIL_ADDRESS>wrote:

I edit my cron.txt file like:
`SHELL=/bin/bash
PATH=/Applications/:/Applications/XAMPP/xamppfiles/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin

php /Users/mac/Code/myproject/artisan scheduled:run 1>>
/dev/null 2>&1`

And It fired :D.
Thank you very much @jimmck https://github.com/jimmck @bkuhl
https://github.com/bkuhl
—
Reply to this email directly or view it on GitHub
https://github.com/Indatus/dispatcher/issues/67#issuecomment-69469586.


Type checking has been failing in CI for the last 6 months

Type checking doesn't pass when running tests for PRs
https://github.com/pallets/quart/actions/workflows/tests.yaml
https://github.com/pallets/quart/actions/runs/9331655216

Hypercorn 0.17.0 added a "state" key (about 6 months ago) to 3 TypedDicts (*) used by Quart in testing type hints. This causes 11 of the 21 type check errors.
(*) HTTPScope, LifespanScope, WebsocketScope,
https://github.com/pgjones/hypercorn/blame/84d06b8cf47798d2df7722273341e720ec0ea102/src/hypercorn/typing.py#L58
Sticking plaster fix until someone figures out how to adjust the tests, pin hypercorn to <=0.16.0:

It is necessary either to remove the concrete types from the now non-generic AiofilesContextManager in Quart, or pin aiofiles to <=23.2.1
aiofiles 24.1.0 changed the type annotation of AiofilesContextManager , making it no longer a Coroutine, no longer a generic type.
quart/src/quart/wrappers/response.py
Line 151 in ef23e8b
self.file_manager: AiofilesContextManager[None, None, AsyncBufferedReader] = None
"Switch to more modern async idioms: Remove types.coroutine and make AiofilesContextManager an awaitable instead a coroutine."
https://github.com/Tinche/aiofiles/releases/tag/v24.1.0

AnyStr is deprecated in Python 3.13, and is being used incorrectly in any case (pretty much as per the example of incorrect usage in the docs https://docs.python.org/3/library/typing.html#typing.AnyStr).
https://github.com/pallets/quart/blob/ef23e8b063ee0b1ae95ba5828570552685ad89bb/src/quart/wrappers/response.py#L317

Will be completed by PR #374

Fixed now


Revert "test/Services: Quarantine 'Checks service on same node'"

This reverts commit ca4ed8dac7f7 ("test/Services: Quarantine 'Checks service on
same node'"). The original intention was to quarantine the test from #17919.
However, ca4ed8dac7f7 quarantined the /wrong/ test case which was not the failing
one since the code in mentioned commit only covers the IPv6 one and in the test
the IPv4 one was failing. Meanwhile, the IPv4 test for 'Checks service on same
node' was still left active on master, and looking at CI dashboard from last few
days on master there has been no failure related to any of the 'Checks service on
same node' tests.

/test


In-tree libraries vulnerable to RUSTSEC-2023-0052 due to jsonrpsee

Is there an existing issue?

[X] I have searched the existing issues

Experiencing problems? Have you tried our Stack Exchange first?

[X] This is not a support question.

Description of bug
jsonrpsee (currently 0.16) pulls in a variety of legacy networking crates, including ones vulnerable to RUSTSEC-2023-0052. AFAICT, updating to 0.20 updates everything (or almost everything) in the dependency tree from webpki to rustls-webpki, resolving the RUSTSEC (and also modernizing the tree in generally).
I did try to perform the work locally, yet the amount of changes to the subscription API made me realize I could not do a proper job within a reasonable amount of time due to my unfamiliarity with the codebase in question.
Apologies if this isn't optimally filed.
Steps to reproduce
No response

cc @niklasad1
There is a MR open here https://github.com/paritytech/substrate/pull/14771, but no integration work so far.

The bump bot actually claims that this closes the problem for us https://github.com/paritytech/substrate/pull/14812

certificate path building and verification is now capped at 100 signature validation operations to avoid the risk of CPU usage denial-of-service attack when validating crafted certificate chains producing quadratic runtime. This risk affected both clients, as well as servers that verified client certificates.

There is an open PR for jsonrpsee bump https://github.com/paritytech/substrate/pull/13992 but we have discovered a few regressions, it's on our radar.

Sorry for missing the PRs, and thanks for clarifying it's the client code alone.
0.16 uses webpki, which doesn't have a bump available, not rustls-webpki hence the issue (though again, I do hear that's just for the client code which I'm unsure if it impacts this repo directly).

It doesn't impact the substrate node (the RPC server) but there are some "tools" in the substrate repo that are using jsonrpsee client.

I'll fix this issue on jsonrpsee v0.16 as well

Much appreciated! Thank you for the patch release!


US Core v 4.0.0 tests performing differently based on Patient ID order

I have two patient IDs that I use to perform my single patient tests. I have found that the patient I use to get the provider token must be the first listed patient id, otherwise the tests perform differently.
Tests which see different outcomes: When the patient who was not used to authorize is listed first in the patient id field, 2.02, 2.061 and 10.02 skip. 10.08 fails.
Test run where first listed patient id was used to generate provider token:
https://inferno.healthit.gov/suites/test_sessions/06060b54-ae95-4c51-b742-203fb59cf716#us_core_v400
Test run where second listed patient id was used to generate provider token:
https://inferno.healthit.gov/suites/test_sessions/0740351d-451a-41c1-9dd3-6a8258d73532#us_core_v400

During the test, it is common that one patient may not have enough data covering all the test cases. So Inferno allows server to provide "additional" patients to cover those test cases not supported by the first patient. That is based on the precondition that server allows Inferno to read data for those "additional" patients.
To reduce the number of searches between Inferno and server, Inferno uses the resources returns from the the first search to form the search queries for the following searches.
For example, for Patient test group, the first search is 2.01 "Patient search by _id". The following test 2.02 "Patient search by identifier" uses one identifier found in the Patient resources returned in 2.01 to form the search query.
In your testing sessions, there are two patient 4755812, 4755813. In both sessions, 4755812 is the one used to generate token.
In your second test session, the patient 4755813 is listed first. What happened during the testing was that 2.01 searched ?_id=4755813 first because that is the first one in patient id list, followed by searching ?_id=4455812. When executing test 2.02, Inferno looked the Patient resources from 2.01 and used the identifier from the first patient it found (Patient.id=4758813), to form the query ?identifier=197 for the identifier search. From the result, we know that server returned an empty Bundle indicating that patient with identifier 197 did not exist or could not be accessed by Inferno. That created a logic conflict that server confirmed the existence of patient with identifier 197 in test 2.01 but could not return the same patient in 2.02.
You may wondering why 2.03 passed. It happened to be that both patient have the same last name.

@yunwwang  Thank you for the explanation - I really appreciate it!


Doesn't handle UTF-8 text well

The API seems to mangle UTF-8 encoded text in a way as if it had received ISO-8859-1 encoded bytes from IPFS and then transformed each byte to a multibyte UTF-8 sequence.
First we create a file with UTF-8 contents:
$ echo Frække frølår | ipfs add
added QmciJtwy4qy7nqzW4pDxhJbUhkpTjKiH5AAmkFLAWWfnmT QmciJtwy4qy7nqzW4pDxhJbUhkpTjKiH5AAmkFLAWWfnmT
$ ipfs cat QmciJtwy4qy7nqzW4pDxhJbUhkpTjKiH5AAmkFLAWWfnmT
Frække frølår
$

Next, let's try and fetch the same content from Python:
>>> import ipfsApi
>>> api = ipfsApi.Client('<IP_ADDRESS>', 5001)
>>> res = api.cat('QmciJtwy4qy7nqzW4pDxhJbUhkpTjKiH5AAmkFLAWWfnmT').rstrip('\n')
>>> print(res)
FrÃ¦kke frÃ¸lÃ¥r
>>> print(res.encode('iso-8859-1').decode('utf-8'))
Frække frølår
>>>

Reproduced under Python 3.5. The text comes out fine under Python2.7, so it's clear that the problem is a Python3 compatibility issue.
>>> res = api.cat('QmciJtwy4qy7nqzW4pDxhJbUhkpTjKiH5AAmkFLAWWfnmT').rstrip('\n')
>>> res
u'Fr\xe6kke fr\xf8l\xe5r'
>>> print res
Frække frølår
>>> res.encode('iso-8859-1').decode('utf-8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe6 in position 2: invalid continuation byte
>>> print res.encode('utf-8').decode('utf-8')
Frække frølår

I'm sorry that I didn't mention the Python version earlier.
There's nothing wrong with the locale settings in my terminal:
$ locale | grep -v '"'
LANG=en_DK.UTF-8
LC_ALL=en_DK.UTF-8
$

And usually python-requests handles UTF-8 (and other charsets) just fine:
>>> import sys
>>> import os
>>> print(sys.version)
3.5.1 (default, Dec  7 2015, 12:58:09)
[GCC 5.2.0]
>>> print(os.environ.get('LANG'))
en_DK.UTF-8
>>> print(os.environ.get('LC_ALL'))
en_DK.UTF-8
>>> import requests
>>> res = requests.get('https://enotty.dk/test/utf8.txt')
>>> print(res.text.rstrip('\n'))
Frække frølår
>>> print(res.headers['Content-Type'])
text/plain; charset=utf-8
>>>

However, go-ipfs doesn't indicate a charset in its response:
$ curl -Is 'http://<IP_ADDRESS>:5001/api/v0/cat?arg=QmciJtwy4qy7nqzW4pDxhJbUhkpTjKiH5AAmkFLAWWfnmT&stream-channels=true&encoding=json' | grep Content-Type
Content-Type: text/plain
$

So perhaps python-requests can be forgiven for not knowing the encoding of the data it receives from go-ipfs?

Btw, this is ipfs version 0.4.0-dev.

RFC 2616 says:

When no explicit charset parameter is provided by the sender, media subtypes of the "text" type are defined to have a default charset value of "ISO-8859-1" when received via HTTP. Data in character sets other than "ISO-8859-1" or its subsets MUST be labeled with an appropriate charset value.

So it seems requests' behaviour is correct. Perhaps I should submit an issue to the go-ipfs project.

Tests forthcoming. I found that the tests in test/functional/tests.py weren't being run by tox, and when I ran them I found that the behaviour of api.add doesn't pass tests on my version of ipfs, which was outdated. So it's all taking a while as I download, recompile, etc.

In my opinion requests, and by association py-ipfs-api, does the right thing. The go-ipfs API, according to RFC 2616, must set a charset specification, but doesn't.

It seems to be a wellknown, if still open, bug in the go-ipfs API: ipfs/go-ipfs#1817

It's a regression in 0.4. My tests for unicode passed in 0.3.8, fail in 0.4.
I'm going to make a pull request with the tests, call it a day.
Well, at least I got to install 0.4.0-dev...

Well done. Thanks!

Noworries, it's been fun!

Also other, existing tests fail now too under 0.4.0-dev. Don't know whether it's because the API has changed or because of other bugs.
I guess it will have to wait till tomorrow to fix it all and send it packing githubwards.

Speaking of failing tests, the tests in test/functional/tests.py fail as well. I was not able to dive into deep, but the assertEqual fails because the api call gives back an unicode string instead of lists/dictionaries. Getting of python objects (api.get_pyobj(...)) also fails with an error that hints an encoding issue. So I guess the issues are connected. (Macosx 10.10.5, python 3.4, ipfs 0.3.10, py-ipfs-api cloned yesterday).

@kseistrup @ingokeck @candeira Can anyone confirm whether this has been resolved in the current version of py-ipfs-api? We've fixed the functional tests and are now compatible with the latest release of go-ipfs, but I don't know if they ever resolved the encoding problem.

@whereswaldon I just installed py-ipfs-api on Python 3.5.2 and reran the python code shown in my original post. Same result as then.

@kseistrup Okay, thank you. I'll look into it.

This should be fixed by #65.

I just installed from git and I still see the same issue.
>>> res
'FrÃ¦kke frÃ¸lÃ¥r'

Did I miss something?

The result of cat should now be a binary string (since an IPFS object may hold any kind of data, not just UTF-8 text). Could you please document again the exact commands you've used?
Working example:
>>> import ipfsapi
>>> c = ipfsapi.Client()
>>> c.cat('QmciJtwy4qy7nqzW4pDxhJbUhkpTjKiH5AAmkFLAWWfnmT')
b'Fr\xc3\xa6kke fr\xc3\xb8l\xc3\xa5r\n'

Also note that the Python package has been renamed from ipfsApi to ipfsapi, so you might be using the stale PIP package instead.

@alexander255 I installed directly from github today (sudo pip install 'git+https://github.com/ipfs/py-ipfs-api.git') and used the exact same sequence as when I reported this issue. I was unaware that the API has changed — mea culpa. When I use your example I do indeed get a valid bytes buffer back from the API call. I'm sorry for the confusion.


Mosaic miners

Adds mosaic ML miners to master (text_prompting)
RE: https://www.mosaicml.com/blog/mpt-7b


ui: break up long community titles

Before

After

Ah, :thinking: so here is what this gives me by only using truncate-lines-2:

If I add both, I get the same as the original fix which indeed doesn't display the full title either even though it has enough space. So, hypothetically the truncate call: https://github.com/inveniosoftware/invenio-communities/blob/master/invenio_communities/assets/semantic-ui/js/invenio_communities/community/frontpage.js#L79C66-L79C66 could be removed, the word-break kept and the truncate-lines-2 added to get a header that is at most on 2 lines, uses up most of that 2 line and ends with an ellipsis. Is that what you were thinking of?
Unfortunately, I can't set up an environment to test this with v12 that just works given the high rate of change right now (I tried twice in the past and it's too time intensive for me and with our instance running v11 I don't have the bandwidth).
We could pause this PR and wait for things to settle or merge it and make a ticket for that nicer change to tackle later. I am fine with either since we were able to apply the css change on our instance.

Yes, it's what I meant. It's probably better to just keep this fix for now then!

I had to fix another occurrence of text breaking out of its boundaries  ("Additional Details" section of the record landing page"). This prompted a colleague of mine to ask:
"why isn't there a blanket overflow-wrap: break-word; thrown somewhere up the element hierarchy to cover all these cases?"
... Yeah, why isn't there? It would solve a lot of these that I am sure will continue to crop up.

I had to fix another occurrence of text breaking out of its boundaries ("Additional Details" section of the record landing page"). This prompted a colleague of mine to ask:
"why isn't there a blanket overflow-wrap: break-word; thrown somewhere up the element hierarchy to cover all these cases?"
... Yeah, why isn't there? It would solve a lot of these that I am sure will continue to crop up.

Agreed. I think it would be good to just set it overall since we keep finding new cases,  @kpsherva WDYT


make new-repo and CONTAINER_ENGINE

It may be worth adding to the documentation that the CONTAINER_ENGINE environment variable needs to be set when using podman.
I have an alias for docker redirecting to podman, running make new-repo led to a "permission denied" error. When running the make command the container is executed with the --user option. Adding --userns=keep-id avoids the "permission denied" issue with podman. Alternatively not having --user (the original intent in the Makefile when CONTAINER_ENGINE is set) works well too.
After the failure the user gets instructured to rerun:
/usr/bin/repo-init --release-repo /release --config="{..}"
which is not correct since this command is now run inside a container. This may be better to point to something like:
podman run --platform linux/amd64 --rm -it -v "/home/fgiloux/development/release:/release:z" registry.ci.openshift.org/ci/repo-init:latest --release-repo /release --config="{...}"

/close


WIP: ENH: add quality prediction to Carbon Flux

This adds a prediction from all global sites. This is still a work in progress: I need to make this a better predictor.

I need a baseline to start of with. To do this, I used a linear regressor on every site to predict the last year at the same site. I used the same model as a baseline with the global prediction, but predicted a different (unseen) site, not the latest year.
Apparently, the correlation coefficient is the metric cared about. I run these simulations 179 times (once for each station). Here are the distribution of correlation coefficients from these trials:

We can see that the linear model at one site significantly outperforms the global linear model.
The summary statistics are

Statistic
One site predicting last year at same site
All sites predicting one held out site

Median
0.573
0.441

Mean
0.497
0.398

Sounds promising.  Still WIP?

Still a WIP. Mostly, I know the baseline performance (what's considered "good"), and have a metric to improve upon.


Fallback to simple String serialization for custom data types

I found some room for improvement in RollbarSerializer (version 0.5.4). Consider the following Scala code as an example:
val customData = Map(
  "customParam" -> Some("hello world") //String -> scala.Option[String]
)

val customDataJava = customData.mapValues(_.asInstanceOf[AnyRef]).asJava //java.util.Map<String, Object>

rollbar.error(new RuntimeException("Some error"), customDataJava)

The JSON payload will have the following format in this case:
{
   ...
   "data":{
      ...
      "custom":{
         "customParam": //No value here
      },
      "notifier":{
         "name":"rollbar",
         "version":"0.5.4"
      }
   }
}

...which will result in a BadRequest status and a could not parse json message.
I think this could be improved by sending a toString() value when all other options are exhausted here.
I'll gladly open a PR for this, if this kind of behaviour would be acceptable for you.

@sbilinski yes, I also proposed it there: #42

@aistomin I just browsed the open issues, but didn't notice your PR - sounds awesome! I'll leave the  issue open for visibility in this case.

Merged