Update Travis requirements

Two comments:

Is it possible to run this using OpenJDK instead of OracleJDK?  Since Oracle started charging for OracleJDK, we should probably prefer OpenJDK if at all possible.
I noticed the addition of a lot of com.sun.* packages as dependencies.  Were these things that used to be in OracleJDK8 but were removed in OracleJDK11?  Do these dependencies work w/ OpenJDK?

Two comments:
1. Is it possible to run this using OpenJDK instead of OracleJDK?  Since Oracle started charging for OracleJDK, we should probably prefer OpenJDK if at all possible.

I''m using OpenJDK and OpenJ9 on my local development environment so yes, it's technically possible. I'll check to see if TravisCI offers a OpenJDK environment for builds.

2. I noticed the addition of a lot of `com.sun.*` packages as dependencies.  Were these things that used to be in OracleJDK8 but were removed in OracleJDK11?  Do these dependencies work w/ OpenJDK?

The jaxb components were deprecated in Java 9 as part of the overall JDK modularization efforts and removed in Java 11. Those dependencies do work with OpenJDK.


Pull Request for user interface.

Pull request for user interface.

Thanks @vivek80801 for the contribution 🏅


update vendor for containerd, cri-tools, and runc

Moving up to make it easier to add test cases to cri-tools and add seccomp support and fix the CI build breaks when multi-arch was pushed to the docker registry for nginx, busybox, etc..
Signed-off-by: Mike Brown<EMAIL_ADDRESS>

Will update this PR just as soon as the containerd team fixes their multi-arch issue https://github.com/containerd/containerd/issues/1498

The fix is a work in progress here: https://github.com/containerd/containerd/pull/1506 I've manually tested this and it resolves the CI test failures.

Updated commit to include the fix for multi-arch images.  Should be ready to review and merge now.


ci: unmanaged dependency check

Cont of https://github.com/googleapis/java-bigquery/pull/3108. Now we have the BOM.
b/320677879

Running Unmanaged Dependency Check against /home/runner/work/java-bigquery/java-bigquery/google-cloud-bigquery-bom/pom.xml
This pull request seems to add new third-party dependency, [org.apache.arrow:arrow-memory-core, org.threeten:threeten-extra, org.apache.arrow:arrow-memory-netty, org.apache.arrow:arrow-vector, org.apache.arrow:arrow-format, com.google.flatbuffers:flatbuffers-java], among the artifacts listed in ./google-cloud-bigquery-bom/pom.xml.

Now the shared dependencies BOM has the third-party dependencies (https://github.com/googleapis/java-bigquery/pull/3176, merged). The check should succeed.

"Unmanaged dependency check / unmanaged_dependency_check (pull_request) Successful"

Did I forget to merge this?

Did I forget to merge this?

I think so.


#117 translate sql double quotes to back quotes

Spark adds double quotes to column names when executing a sql query, which is not currently supported in the clickhouse jdbc.
Sql string double quotes get replaced with back quotes, in the clickhousifySql function, e.g.:
"a", "a b", "a 1 b-c_d"

translates to
`a`, `a b`, `a 1 b-c_d`

BTW, we have added support of SQL double quotes in today version of ClickHouse (1.1.54245).

As I can see there are cases where implementation proposed in this PR fails, for example
select 'there is "something" here'

Considering support of SQL double quotes had been implemented in server and that it is not that simple to add it in client, I propose to update the server and do nothing at client side.


Better flow between fiat and crypto

Switch prices on first page
Fiat and Crypto on different lines for each order

Not each order, I meant each option


It works on macOS 11

I just launched the FluffyDisplay on macOS 11.2 Big Sur. Thanks for creating such application. It helps me with 2 MacBooks Pro. I am creating a virtual display with FluffyDisplay on the first mac and connect from the second mac over VNC.

Thanks! Good that it is useful. I haven't really worked on it (or even used it) much lately, but once I get the inspiration again I will start using it more myself and do some further improvements.

(I guess there is no reason to keep this issue open as such then.)

Sure. Just wanted to say thanks.

just used the same method to connect an older macbook (2008). thanks!


Harden handling of CSMS calls, if the connectorId is not in valid range.

OCPP Version
OCPP1.6
Describe the bug
If from backend site, the connectorId is not in valid range, the relevant reject is answered to avoid undefined behavior.
Solution check the range:
See PR 882.
To Reproduce
No response
Anything else?
No response

Completed with https://github.com/EVerest/libocpp/pull/882


Manager summary stats should be per field

Because of this line summary stats can be overwritten if two fields use the same name. Summary stats should be a dict with keys being the unique registry keys of the field and values being the summary stats
https://github.com/scverse/scvi-tools/blob/00e03420c5b7b59bd9ae57238bbfe431d9ab8865/scvi/data/_manager.py#L315

what do you think @justjhong?

It's a logical change, but also results in verbose access to the field-specific summary stats. If we could create an API to easily fetch summary stats (e.g. get_summary_stat(field name, stat key)) then it could work well enough. Mainly was being conservative about the structure of summary stats as a single high-level object when I implemented this.

I think we should be raising an error here instead if someone has two fields with the same name.


Scale correction

If we use transforms etc. for all size related manipulation, we will need a feature that corrects the contents to still look like they should.

Framer does this by adding the layout attribute to child components that are also supplied by framer (eg. motion.div).

Closing as unfnished in anticipation of alpha release.


Fixes #54 and Fixes #55 - typos in comments of the shell script

Signed-off-by: Jim White<EMAIL_ADDRESS>

Sorry - Fixes #56 and Fixes #55 (not 54)


Empty folders

Hello,
v1.33 leaves empty folders. Simple test. Make sync. Change folder name in source. In destination new folder have all files as it should, but old folder still exists. Files are gone, but folders are still there.
Thanks

This is a known problem. See issue #100

what about we will have something in rclone like this:
rclone move ... will got option rclone move --delete ... possible?

@scriptzteam I'm not sure I understand what you mean - move will delete the files but won't delete the empty directories until #100 is fixed.

yes i mean the prefix --delete will be used as to remove empty dirs or --delete-empty


Allow blocks with partially resolved parentage information

Impact of the new feature
There could be many groups affected by such a change.
Is your feature request related to a problem?
This is a request for a feature change which is related to a recently found custom use case involving miscommunication between WMCore and DBS. Here is the related WMCore issue and description of the situation: https://github.com/dmwm/WMCore/issues/11260#issuecomment-1460044696
Long story short, we need to allow uploading blocks with partially resolved parentage information. This means few of the files present in a block may have no relation to files from the parent dataset (missing or partially resolved parentage information).
The main question here is eventually to Core SW and Physics groups as well - How mandatory is this constraint and How badly would such a change affect other steps?
Describe the solution you'd like

Solution #1: We simply remove the check for fully resolved parentage information for all files in a given block, here: dbs/fileparents.go#L344-L358

Solution #2: We allow files from the child block to refer to a NO parent file by setting the parent file id to a predefined default value (in accordance with the DBS data structures), which should be an indication that these files are missing the upper level files which they have been produced from.

Describe alternatives you've considered
The alternative approach wold be for WMCore to start developing a mechanism for automatically invalidating all those files from DBS and Rucio, which will go silently in the background. So far we have no numbers to cite about how frequent this is and that's why we cannot tell if and to what extent of data reduction it could lead. But in any ways, we are not in favor of this alternative path.
Additional context
None

@todor-ivanov , thanks for creating the issue. But I think neither solution is acceptable. For the solution 1 I do not know how it will impact other use-cases. For the solution 2 I do not know how to correctly do that in ORACLE DB, what is a default and why we should have it in file parent table. I think you are looking for solution 3:

client provides list of chiild, parent ids, e.g. [(1,null), (2,3)]
the first pair indicate that there is no parent for child id 1
we adjust DBS code to skip this pair during injection to DBS file parents table
since there is no parent record in file parent table for child id 1 then anyone who will look-up this lfn will no there is no parent for it.

How does it sound to you? @amaltaro please provide your input too.

@todor-ivanov , please provide additional information which DBS API is affected by this request and provide an example of HTTP POST input payload. For DBS API please inspect the corresponding python client to identify which DBS API it is using, here is DBS API documentation which lists all DBS APIs.

I can work on this if @klannon approve my time allocation for this task, or we can delegate this task to @d-ylee . Meanwhile here I provide list of action items which should be done within DBS codebase:

the affected DBS API is fileparents and it comes from fileparents.go module
we need to loose constrain on FileParents data structure, see https://github.com/dmwm/dbs2go/blob/master/dbs/fileparents.go#L63 and remove validate parameters for PARENT_FILE_ID such that it can accept null
the validate function should be adjusted to skip validation of PARENT_FILE_ID if it is not provided, but if it is provided it should still perform the validation.
the insert function should check PARENT_FILE_ID and if it is not provide just exit immediately
and, finally we need to provide new unit test in int_fileparents.go test file to test insertion of pair where file parent id is null.

@d-ylee Hi Dennis, did you manage to check the feasibility of the developments listed above? Please let us know if further clarification is required here.

@amaltaro I think the discussion above is clear to me. I can work on this after I get the updating physics group name completed. Thanks!

@vkuznet After looking through the code, I noticed that for the database schema:
https://github.com/dmwm/dbs2go/blob/928dc255e5a695546767c363c5d0fb9b37cd6fd8/static/schema/DDL/create-oracle-schema.sql#L866-L870
FILE_PARENTS has FILE_PARENT_ID as not null, and also has FILE_PARENT_ID as part of the PRIMARY KEY constraint. Would it be an issue if the CONSTRAINT contains a NULL value?

From the DBS perspective, would there be any impact and/or performance degradation if we allow NN_FP_PARENT_FILE_ID to be Null?
When using the fileparents API, this is an example of call:
https://cmsweb.cern.ch/dbs/prod/global/DBSReader/fileparents?logical_file_name=/store/mc/Phys14DR/WprimeToENu_M_4800_Tune4C_13TeV_pythia8/AODSIM/TEST_PU20bx25_PHYS14_25_V1_TEST_Alan-v9/00000/3E27F2B3-D975-E411-8793-02163E010EE6.root
which returns the following data
[
{"logical_file_name":"/store/mc/Phys14DR/WprimeToENu_M_4800_Tune4C_13TeV_pythia8/AODSIM/TEST_PU20bx25_PHYS14_25_V1_TEST_Alan-v9/00000/3E27F2B3-D975-E411-8793-02163E010EE6.root","parent_file_id":57020800,"parent_logical_file_name":"/store/mc/Fall13/WprimeToENu_M_4800_Tune4C_13TeV_pythia8/GEN-SIM/POSTLS162_V1-v1/30000/AEF4E42B-1462-E311-8D2E-0025901D48B2.root"}
,{"logical_file_name":"/store/mc/Phys14DR/WprimeToENu_M_4800_Tune4C_13TeV_pythia8/AODSIM/TEST_PU20bx25_PHYS14_25_V1_TEST_Alan-v9/00000/3E27F2B3-D975-E411-8793-02163E010EE6.root","parent_file_id":57020803,"parent_logical_file_name":"/store/mc/Fall13/WprimeToENu_M_4800_Tune4C_13TeV_pythia8/GEN-SIM/POSTLS162_V1-v1/30000/A4668E29-1462-E311-AEB5-1CC1DE18CFEA.root"}
]

do I understand it right that, in case 1) above, we would get an output data as (just an example):
[
{"logical_file_name":"/store/mc/Phys14DR/WprimeToENu_M_4800_Tune4C_13TeV_pythia8/AODSIM/TEST_PU20bx25_PHYS14_25_V1_TEST_Alan-v9/00000/3E27F2B3-D975-E411-8793-02163E010EE6.root","parent_file_id":none,"parent_logical_file_name":""}
]

while in case 2) the output response would be an empty list. Can you someone please confirm this behaviour?

Having or not parent file id in FILE_PARENTS table will not impact DBS performance as DBS APIs use streaming. But allowing null parent will increase table size. By how much it depends on how may null parents we will have for our files. Said that, your understanding of output of fileparents API is correct, for case 1 we'll have [{'logical_file_name': 'bla', "parent_logcial_file_name': null}], while in case 2 you'll get empty list since no record for file bla will be present in FILE_PARENTS table.

Hi @vkuznet @d-ylee @amaltaro,
To me option  2) is a NoGo. It is quite confusing and introduces more complexity and confusion rathar than simplicity. This is exactly the type of solutions relying on hidden logic and unwritten policies and agreement which are to be  well forrgotten in really near future and the knowledge left to be forwarded through the generations only by the means of spoken wisdom. That will make  the code from both sides quite unreadable only in a year from now. I am much more for the solution 1),  where the parrent is explicitly pointed as NULL, meaning that  DBS simply has no information for the origin of the child file, but does have all the rest of the information related to this file. Plus, IIUC, in many many cases it is much more important for physics to find the file itself.

@vkuznet @amaltaro @todor-ivanov
If we go with 1), would we need to update the database schema for the FILE_PARENTS table to

Allow NULL value for PARENT_FILE_ID
Redefine the PRIMARY KEY to not use PARENT_FILE_ID

For the PRIMARY KEY, I am still not clear how we should define this
From a past message:

1. if we loose constrain in FILE_PARENTS table then we can have entry in it for file ids which does not have parents

Would we need to have a entry in the FILES table that is for no parent files, but when getting from /fileparents, return none in the parent_file_id field if this entry ID is referenced?
Also, would there be cases in which allowing null parentage for FILES also affect DATASET_PARENTS and BLOCK_PARENTS?

hi @vkuznet @d-ylee
The negative value for missing parent file id is a valid workaround, which would prevent the need of table redefinition. But we need to be sure what is about to be returned to the client as well. Because, once recorded  as -1 I believe it will be also returned as -1 in subsequent client queries, rather than null, as originally recorded by the client. This is slightly confusing but may work, provided we check if any further change on how we digest the data from those APIs in WMCore would be needed.
Denis, can you list the APIs in DBS which would develop such a silent data type metamorphosis if we opt for this workaround, so that we can check where and when WMCore uses them?
FYI @amaltaro

@todor-ivanov
Not sure if this is what you are asking for, but the APIs that are related to parentage are these:

/fileparents
/blockparents
/datasetparents

Just an update of this issue as an outcome of the Zoom discussion we just had (Valentin, Dennis and myself), this is the list of action items that we came up with:
1. Dennis: Check if null json gets converted to 0 in Go language.
2. Todor/Alan - IF NEEDED: Update WMCore to provide parent id = 0 instead of None
3. Dennis: Find what are the constraints/validation performed on the DBS Server side.
4. Dennis - IF NEEDED: update DBS Server to deal with partial block information
5. Dennis: create a CERN oracle account
6. Dennis: Get a dump of testbed or production
 * re-create the ownership of the table (or preserve the ownership)
7. Dennis: deploy DBSServer in a test k8s cluster and share the instance url
8. Todor: deploy ReqMgr2 in a test cluster pointing to Dennis DBS url
9. Todor: Inject one of these blocks in this test instance
  * note that ReqMgr2 should not mark a workflow as ParentageResolved = True
10. Todor: Validate the read APIs

@d-ylee Hi Dennis, I wonder if you managed to look into the null marshaling process? Such that we can decide whether we will have to make further changes in WMCore or not.

@d-ylee thank you for these details! I think it is good enough and I understand that WMCore can keep providing a tuple of (int_child_file_id, None) whenever the parent file is gone (and/or invalidated). So no further changes should be expected in WMCore.
Please let us know how your communication with the CERN DBA goes and where you need support on anything on this front.

@d-ylee thanks for confirmation. PLEASE keep in mind that None is pure Python data-type and python client MUST provide proper JSON, e.g.
>>> import json
>>> json.dumps({"bla":None})
'{"bla": null}'

So, on a wire, i.e. when HTTP request is send, the python code MUST provide null and not None, those are two different things. The null is null value in JSON standard, while None is Python none data-type.

I did a request from Python in the same way as what @vkuznet detailed using json.dumps on a Python dict with key that has the value of None. The result is the same: Go decodes/unmarshal this as 0 for int64.

hi @d-ylee would you want us to organize another meeting if you need some more information from our side so we can proceed on this? If you think there is some step that we first need to do but in the meantime it is blocking the progress on the task, please let us know?

I spoke to @yuyiguo about this issue, and she explained to me some of the policies from the past and about this partial parentage issue.
Before DBS3, partial blocks were allowed, and these were able to be inserted into DBS when there were missing blocks. This was because we may have not had all files or we did not have the parent-child relationships yet.
This was an issue because we might forget about these partial blocks and end up not knowing who is the parent.
Yuyi mentioned that the policy in DBS3 was to only allow full blocks. Workflow management was to have its own database to keep unfinished blocks there, then insert into the production database.
If we want to change this policy, we need to have it written down and record who is responsible to complete the block.
Another concern is when a child has multiple parents. The current schema in the BLOCK_PARENTS database is the PARENT_BLOCK_ID and THIS_BLOCK_ID (the child). These two columns have a unique constraint. This means that there can't be more than one -1 allowed for a single child. If there are multiple parents with the same children and we use the -1 logic, the UNIQUE constraint would be broken.
Another question I have is, do you have the location of the written DBS3 design policies?
What are your thoughts?

@amaltaro @d-ylee
Sure, I can have a Zoom chat this afternoon. I have a meeting from 1:15-2:30ish. Do you wan to do it at 3 pm?

@amaltaro @yuyiguo I can also meet at 3PM

@amaltaro @yuyiguo 3pm FNAL time works for me as well. I am emailing you now with further coordinates. Thanks!

Great!

@d-ylee @yuyiguo thank you for taking the time for this chat.
From our discussion, it looks like we are going to explore the first solution proposed in this issue, this:
""" Solution #1: We simply remove the check for fully resolved parentage information for all files in a given block, here: dbs/fileparents.go#L344-L358 """
With that, a few concerns were raised and we need to check the following:

whether it is doable to relax the number of files constraint. In other words, change the server such that it allows a list of child files provided by the user to be <= (less or equal than) the actual list of files in the child block (which would be triggered when a file does not have any parent file).
check if DBS server is performing any data lookup based on the information provided by the client; or if DBS is actually using everything that is provided by the client. For instance, does DBS server use the parent dataset information; or does it find the parent dataset id based on the list of parent file ids?

Dennis, as we discussed as well, here is an example of a block JSON dump (which might be broken, as we are not able to insert it into DBS):
https://amaltaro.web.cern.ch/amaltaro/forWMCore/dbs2go_98/c44686c3-86fa-4fc3-9902-ae328914b317.json
and this is an example of listChildParent data structure, with first item being the child id and second the parent id:
https://amaltaro.web.cern.ch/amaltaro/forYuyi/parentage.json
as mentioned in this comment: https://github.com/dmwm/WMCore/issues/11260#issuecomment-1452455362
Please let me know your findings and/or if any extra information is required.

@amaltaro , the proper way to solve this issue is to either introduce a new DBS API which will do item #1 in your list or add additional parameter to existing API which end-user must specify for such behavior. This way, we preserve existing API backward compatibility and functionality, and allow user to bypass such constrain in existing API. If we agree on this, then number 2 in your list can do both since new API or existing API using external parameter can perform both actions. To avoid confusion I would favor of creating a new DBS API and therefore re-factor and share code from existing codebase in both APIs. But extra parameter will work too but it will be more confusing in my view.

@vkuznet that would be another way to address this issue, even though I would not call it "the proper" way.
The only clients of the dbs-client api insertFileParents is WMCore, which needs to have a mechanism to update children/parent relationship post block injection. If the constraint of number of files is relaxed to <= than the expected files in the block, this WILL still be backwards compatible.
In my opinion, I see no reason to over-complicate this development, which has already being dragged out for many months. As I said before, there is only 1 client for this API and the fix I propose should be pretty straight forward.

@amaltaro , good design practices suggests to separate APIs for different use-cases rather to introduce complicated if/else logic. The later was used in Python server code which (if you had a chance to look at it) is very hard to read and understand. Therefore, I expressed my opinion based on this argument. Plus, you never know that one client will only the case. I can easily foresee when we may have other clients, plain script (instead of python code), or CRAB, or any new development we may have in a future. And, even using single client as it is right now you really address different use-cases. For maintenance reason it is better to address them via separate API rather complicate server code (even if we'll need make additional  code-refactoring and wait a little bit longer).

@amaltaro , good design practices suggests to separate APIs for different use-cases rather to introduce complicated if/else logic.

to me this is exactly the opposite of what you are suggesting. Creating another API just to deal with a slightly different use case IS an over-complication, it is confusing and it causes mistakes on the client side.
The extra query string would be the best approach in my opinion, but that will require changes in:

DBS server side
DBS client side (new release has to be made as well and properly integrated into the WMCore stack)
WMCore

The later was used in Python server code which (if you had a chance to look at it) is very hard to read and understand.

The approach I am suggesting adds NO overcomplication and does not make the code more complicated, does not increase maintenance and is transparent to the end user. It also does not require the release of a new dbs3-client!
It literally is a change in the source code from something like:
files_in_block == files_passed_by_client

to something like
files_in_block <= files_passed_by_client

Anyways, I think we will spend the rest of our days disagreeing on this!
I'd rather let @d-ylee Dennis consider what is the best approach here, noting that at the moment we have 100s of workflows stuck in the system waiting for this development to be deployed in production.

Alan, I would like to take last change to convince you and the team to not add changes to the existing API. My last argument is the following:

by design (both Python and Go implementation) of DBS code provides comprehensive checks and input validation from the client
by using the same API and rely on different input we break the above requirements and open the door to handling input without cross check
if we'll implement the changes the way you propose DBS server will not have such ability to check if given input is right or wrong (i.e. it does or does not have parents) and it will just follow if/else logic without knowing and validating that client provide correct input

In my suggestion this is not possible by design since client will be forced to use either new API or provide external parameter to the API in question. Doing this way client a-priory will do a right job and DBS server will not rely on if/else logic of the input data. You may provide as many arguments as you want that such server implementation is more easy to do but you open a possibility that one day client will (by whatever mistake) provide incorrect input and DBS server will follow if/else logic to insert the data to the database. For instance, someone by mistake will not provide parents, or we may have a bug elsewhere in WMCore which may skip some parent, etc. In either case DBS server will follow the input if we will use the same API signature. To avoid this possibility we must enforce client to be explicit with the call (in other words client will know what it is doing by calling proper API or API with proper parameter rather providing an input only to DBS server).
I hope that you can take this arguments more seriously and properly evaluate the implication of implantation on  DBS, but I'll not argue any more about how to implement this logic and leave it up to your and the rest of WMCore team to decide (here I have dual role, I belong to WMCore team and in my view the implementation should have separate API or API+external parameter and I implemented DBS logic). I think Dennis should implement whatever WMCore will agree.
Said that, I tag @todor-ivanov to evaluate my concern and you both should come to conclusion how the implementation should be done. Once you'll reach such agreement I think for Dennis it will be more clear how to implement it. At the end, it is just implementation but my concern that the choice may impact data consistency in DBS database in a long run.

hi @amaltaro @vkuznet @d-ylee,
It is difficult to say here what is the better solution. The arguments and concerns for both cases are far in the future e.g. "are we going to stay the only user of this API", "Will we affect other's code behavior" etc... But as a general rule of thumb I'd not rely on the current situation and conclude for the far future.
One really important thing to take into account when making such a decision is the knowledge if our current policy, for not inserting files into DBS which lack parentage information, is acknowledged and well understood by every other group that  may be affected. And here I mean both sides:

Eventual users of WMCore modules or DBS apis directly (including CRAB), and also
Those who need to consume the output from DBS, such as physics

We should not talk only about WMCore and people responsible for DBS server. And this, I believe, is information we do not have at the moment. Obviously there is no policy document for DBS Server behavior..., ence our blindness on the applicability of this policy. There could easily be some proliferation of the reverse opinion among other groups who rely on the data returned by DBS - meaning they may rely on the parent-child relationship in DBS to be satisfied in any case. So we are actually in the state of lack of consensus on the policy itself.
Now about the possible solutions: The fix only on the server side is something that is less work and is an encapsulated change, but because of the above unknowns, I'd bet for the harder one - the one that forces the client to explicitly and consciously consciously stress which policy he would chose for uploading the files. It is not only backwards compatibility that  we are talking about here, but also about bearing the responsibility for doing the one or the other. At the end we will not change the API, regardless of which solution we  chose here - !!! we only change Server behavior !!!

Creating a new API is out of question - simply not worth it.
Setting an option to the already existing API for choosing the correct policy which would be defaulting to the current one, in my opinion is the way to go.

Yes this is going to affect/require changes from everybody, as @amaltaro  said:

DBS server side
DBS client side (new release has to be made as well and properly integrated into the WMCore stack)
WMCore

In the case of WMCore this should be fairly small change - adding the new option here and here The tests and validation procedure, on the other hand, would stay as usually long and need to be a full one.... well it has its price.
For the rest, I cannot tell .
Something more. Since this situation (or what concerns WMCore) is an expected one, and our use cases are well identified in advance - as early as building the lists for the parentage resolution itself, I'd not further complicate the code by implementing extra calls to DBS for checking dbs errors and retries. This would only affect code readability and scalability both together without any benefit. So I'd go directly for just changing the policy on the WMCore side by using the newly provided option to the same API.

I just back from my vacation and saw this discussion.  Right before @d-ylee and I left for vacation,  @amaltaro, Dennis, and I had a Zoom meeting on this.  We discussed why the "-1 " would not work with the DBS DB schema. Alan pointed out that the missing parent files would be lost forever, in other words, the file parentage would not be updated in the future.  So we concluded that it is better to just not put anything in DBS for the ones that lost their parents' files.
I think @vkuznet made a valued argument to protect data integrity.  To consider both DBS and WMCore, I would propose below API  and server upgrade:

Add missingfiles=0 to the existing API. This would be the default. So no effect on others except for WMcore.
When WMCore calls the API, it tells DBS how many missing files it has with missingfiles=n.
DBS server continues to verify the child-parent relationship. If the missing files does not match n, it will reject the input.
This way both client and server know exactly what should happen. This is my two cents.

@amaltaro @todor-ivanov If this is what we would like to do, I think we would need to incorporate the number of missing files to the check here: https://github.com/dmwm/dbs2go/blob/928dc255e5a695546767c363c5d0fb9b37cd6fd8/dbs/fileparents.go#L352
I do see that the utils.Equal function also makes sure the file id and the block file ids match. This should probably also be validated during the insert request and may need to be another parameter.
Another question I have is that would DBS need to keep track of these missing files? Or would this be on WMCore's side? Both this question and the above validation seem to affect each other.
Thoughts?

@d-ylee Hi Dennis, Yuyi's idea looks good to me and having the client to provide how many files are missing in the json data is a good approach (defaulting it to 0, if not provided).
On what concerns keeping track of the missing files. I would say that the DBS server should at least log that a block with incomplete parentage action was performed (sort of an access log). On the WM side, we do log that a given file failed to find its parentage file.
Please let us know if there are any remaining open questions.

@d-ylee Dennis, in addition to what we have discussed today, I just wanted to update this thread with a way to migrate dataset from prod/global do your dev cluster (such that we can test things out):
from dbs.apis.dbsClient import DbsApi
dbsApi = DbsApi(url = 'https://cmsweb-test3.cern.ch/dbs/dev/global/DBSMigrate/')
dbsUrl = 'https://cmsweb.cern.ch/dbs/prod/global/DBSReader'
dataset = "my dataset name"
migrateArgs = {'migration_url': dbsUrl, 'migration_input': dataset}
dbsApi.submitMigration(migrateArgs)

I looked into the dbs3-client code and came up with the following curl standalone call:
curl -k --cert $X509_USER_CERT --key $X509_USER_KEY -vv -X POST "https://cmsweb-test3.cern.ch:8443/dbs/dev/global/DBSMigrate" -d '{"migration_url": "https://cmsweb.cern.ch/dbs/prod/global/DBSReader", "migration_input": "/DM_MonoZPrime_A_Mx50_Mv5000_gDM1gSM0p25_Zprime2p0_TuneCP5_13TeV_madgraph-pythia8/RunIISummer20UL17MiniAODv2-106X_mc2017_realistic_v9-v2/MINIAODSIM"}' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'UserID<EMAIL_ADDRESS>-H 'User-Agent: Alan-cURL'

but all my attempts failed with:
< Response-Status: 405 Method Not Allowed

and I cannot figure why. Perhaps @vkuznet has a working example with curl other than this:
https://github.com/dmwm/dbs2go/blob/master/docs/MigrationServer.md#examples
which relies in a different service/API, localhost and curl command which is not available in the dbs2go images(?)

Moreover, each DBS server has its own set of APIs which you can easily find via /apis end-point, e.g. here is list of APIs for DBSMigrate which I took from testbed:
scurl https://cmsweb-testbed.cern.ch/dbs/int/global/DBSMigrate/apis
[{"api":"/dbs/int/global/DBSMigrate/blocks","methods":["GET"]},{"api":"/dbs/int/global/DBSMigrate/submit","methods":["POST"]},{"api":"/dbs/int/global/DBSMigrate/remove","methods":["POST"]},{"api":"/dbs/int/global/DBSMigrate/blockparents","methods":["GET"]},{"api":"/dbs/int/global/DBSMigrate/serverinfo","methods":["GET"]},{"api":"/dbs/int/global/DBSMigrate/","methods":["GET"]},{"api":"/dbs/int/global/DBSMigrate/total","methods":["GET"]},{"api":"/dbs/int/global/DBSMigrate/bulkblocks","methods":["POST"]},{"api":"/dbs/int/global/DBSMigrate/dummy","methods":["GET","POST"]},{"api":"/dbs/int/global/DBSMigrate/errors","methods":["GET"]},{"api":"/dbs/int/global/DBSMigrate/help","methods":["GET"]},{"api":"/dbs/int/global/DBSMigrate/status","methods":["GET"]},{"api":"/dbs/int/global/DBSMigrate/datasetparents","methods":["GET"]},{"api":"/dbs/int/global/DBSMigrate/healthz","methods":["GET"]},{"api":"/dbs/int/global/DBSMigrate/metrics","methods":["GET"]},{"api":"/dbs/int/global/DBSMigrate/apis","methods":["GET"]},{"api":"/dbs/int/global/DBSMigrate/process","methods":["POST"]},{"api":"/dbs/int/global/DBSMigrate/cancel","methods":["POST"]}]

Similar /apis end-point exist for reader, writer servers and you can easily see which method each API is using.

@vkuznet Valentin, you probably missed a few important points:

we are trying to test dbs2go-global-m, which is the service that receives dataset migration requests
the server expected API endpoint is dbs/dev/global/DBSMigrate/submit, instead of dbs2go-migrate/submit. Unless you need to run a different service which provides a different set of APIs. If so, then you start deviating from what you actually run in production (given that you are testing a different setup)
none of the dbs2go images provide curl, so we need to deploy another service in the kubernetes cluster just so we can use curl to reach a the service that we need with dbs2go-global-m.dbs.svc. Which is inconvenient.

The apis endpoint will be useful though. Thanks

Alan, I understand what you are doing. The provided dbs2go documentation does not list our production/test/dev URLS and refers to actual service. Of course I would expect that you know your server endpoint, e.g. /dbs/dev/global/DBSMigrate and use it instead of listed in documentation dbs2go-migrate. The curl should not be inside of dbs2go images. Instead, as I pointed many times just deploy new curl image in your k8s and use it to access any service.  Here its manifest file: https://github.com/dmwm/CMSKubernetes/blob/master/kubernetes/cmsweb/services/curl.yaml And, you can deploy it as easy as
kubectl apply -f curl.yaml

Then login to curl pod is created and you can login to it and use curl.

Okay, it looks like Dennis has some documentation work to be done at some point, such that a developer can go as closer to the production environment as possible, to have more meaningful tests.
I didn't remember we had a manifest only for curl, that will definitely be handy. Thanks.
@d-ylee Dennis, as we discussed yesterday, I finally got many json dumps for a StepChain workflow that you could play with. Note that they only provide dataset parent information, no file and or block parentage. You can find them here:
https://amaltaro.web.cern.ch/amaltaro/forDennis/
Their parentage relationship is:
  "ChainParentageMap": {
    "Step1": {
      "ParentDset": null,
      "ChildDsets": [
        "/RelValWToLNu_14TeV/CMSSW_12_4_0_pre2-GenSimFull_SC_EL8_Sept2023_Val_Alanv4-v11/GEN-SIM"
      ]
    },
    "Step2": {
      "ParentDset": "/RelValWToLNu_14TeV/CMSSW_12_4_0_pre2-GenSimFull_SC_EL8_Sept2023_Val_Alanv4-v11/GEN-SIM",
      "ChildDsets": [
        "/RelValWToLNu_14TeV/CMSSW_12_4_0_pre2-Digi_2021_SC_EL8_Sept2023_Val_Alanv4-v11/GEN-SIM-DIGI-RAW"
      ]
    },
    "Step3": {
      "ParentDset": "/RelValWToLNu_14TeV/CMSSW_12_4_0_pre2-Digi_2021_SC_EL8_Sept2023_Val_Alanv4-v11/GEN-SIM-DIGI-RAW",
      "ChildDsets": [
        "/RelValWToLNu_14TeV/CMSSW_12_4_0_pre2-RecoNano_2021_SC_EL8_Sept2023_Val_Alanv4-v11/DQMIO",
        "/RelValWToLNu_14TeV/CMSSW_12_4_0_pre2-RecoNano_2021_SC_EL8_Sept2023_Val_Alanv4-v11/MINIAODSIM",
        "/RelValWToLNu_14TeV/CMSSW_12_4_0_pre2-RecoNano_2021_SC_EL8_Sept2023_Val_Alanv4-v11/GEN-SIM-RECO",
        "/RelValWToLNu_14TeV/CMSSW_12_4_0_pre2-RecoNano_2021_SC_EL8_Sept2023_Val_Alanv4-v11/NANOAODSIM"
      ]
    }
  },

Please let me know if this enables you to move forward with this development and testing.

@amaltaro Thanks for the json dump.
@vkuznet I was going through the fileparent insert API and noticed that we are validating the incoming json twice:
https://github.com/dmwm/dbs2go/blob/e71c8a4ef1b6d64058aabee95d6ede53ab163d1c/dbs/fileparents.go#L85
https://github.com/dmwm/dbs2go/blob/e71c8a4ef1b6d64058aabee95d6ede53ab163d1c/dbs/fileparents.go#L368
Was there a reason for this?
To implement the missingFiles, I need to modify this validation step to accept 0 for PARENT_FILE_ID or to handle the second error in the Validate function:
https://github.com/dmwm/dbs2go/blob/e71c8a4ef1b6d64058aabee95d6ede53ab163d1c/dbs/fileparents.go#L209-L222

Why do you think we are validating twice? Those are two different APIs, the former one is Insert in FileParents
// FileParents represents file parents DBS DB table
type FileParents struct {
	THIS_FILE_ID   int64 `json:"this_file_id" validate:"required,number,gt=0"`
	PARENT_FILE_ID int64 `json:"parent_file_id" validate:"required,number,gt=0"`
}

record. The corresponding json will looks like this:
{"this_file_id": 123, "parent_file_id" : 456}

While latter is from InsertFileParentsBlockTxt APIs which reads  FileParentBlockRecord
type FileParentBlockRecord struct {
	BlockName         string    `json:"block_name"`
	ChildParentIDList [][]int64 `json:"child_parent_id_list"`
}

record.  The corresponding JSON record will look like this:
{"block_name": "block#123", "child_parent_id_list": [123,456]}

Both records comes in different DBS requests (you may need to figure out which ones). Therefore both require their own validations.
Regarding validation procedure. The 0 is default value if it is not provided in JSON record for given data-type. Therefore, I'm not a big fan to change validation code for it. Instead, I suggested to use -1 to clearly indicate missing parent. I'll leave you to discuss this with Alan.

@amaltaro and I decided to use -1 in order to keep the required tag for PARENT_FILE_ID

@vkuznet I made a PR and would like to deploy it to the CERN image registry. My plan is to use a conditional on the tag (adding -dev to the tag name) to not have it be pushed to preprod. Is it possible for me to publish to the image registry from my fork?

@d-ylee Dennis, provided that you can authenticate, yes, you should be able because I have done that for some WMCore images.
Manual build and upload should work as well, in case you prefer that option.

@d-ylee I second Alan reply. Yes, you can upload new image, and yes it would be useful to have -dev suffix to distinguish it, and yes you should use your new image for testing, and yes you can build it locally and use it too. Bottom line, in your dv k8s cluster you can do whatever you want. You may build new dbs2go executable and copy it too and restart your service within existing pod  too. There are so many possibilities that it is up to you to pick up the one which gives you best way to do debugging.

@vkuznet I am having difficulty making the image based off the commands in the GitHub build action. I looked at the DockerFile in dmwm/CMSKubernetes, and it requires the tag to be in this repository: https://github.com/dmwm/CMSKubernetes/blob/5ca68d24585051156eb08b6d1d85ff8e2188f2e9/docker/dbs2go/Dockerfile#L27-L28
Instead of merging, should I pull it into this repository as a new branch, then tag it to v00.06.43-dev to trigger the GitHub action?

@d-ylee why do you need CI/CD to build your custom image. Instead do the following:

clone CMSKubernetes repo
cd to docker/dbs2go area
modify Dockerfile to include your specific tag, branch, etc.
build yoiur image using docker build -t ... command
upload manually your image to registry.cern.ch
iterate as many times as necessary until you satisfied with your branch and merge, then CI/CD will build official image.

@amaltaro has tested the changes on test3, and we have agreed to move it to preprod for further testing.

I have pushed the changes to dbs2go-global-w and dbs2go-phys03-w on testbed.

Awesome, thanks Dennis! I hope to be able to upgrade testbed ReqMgr2 later today, otherwise it will happen tomorrow afternoon.
BTW, should we close this ticket with https://github.com/dmwm/dbs2go/pull/101 ?

@d-ylee kind ping (please see above)

Closing due to changes deployed to production.


Bump<EMAIL_ADDRESS>

Solves https://github.com/beakerbrowser/beaker-core/issues/27

Really sorry for the delay on hitting this - I shouldve rolled it into the 0.8.9/10 releases, but missed it. Merging now, will try to include it in a 0.8.11 in a bit.


Added content

I added one poem and one image to the homepage file.  Added poem to lines 68-84 and image to 88-89, they will need to be checked for proper formatting and image added to any other image files.
Issue https://github.com/suwebdev/mingled-worlds/issues/5

Initial Style
I think your code is looking really great so far! I love that you have implemented a poem and an image to start getting the content and main portion of the website put together and adjusted.
Upon looking at a live view of the website, it seems the image overlaps into the other title cards on the main page. They would either need to move down to another area of the website underneath or anove the poem and image combination, or removed completely for stylistic purposes.

Code
https://github.com/suwebdev/mingled-worlds/blob/5a25f35b7832766e7536c2a951aa6479eb54d37d/index.html#L69-L84
Upon looking at your code, I see that you have a lot of page break "" tags used to break apart the lines of the poem. While this is effective and gets the design display working properly, I think this is not the most efficient when adding a lot more poems and may end up reusing a lot of redundant code and endless copy+pasting.
A solution I think would be to make the container that the poem is in set to a specific height, then ass a word-wrap CSS properly. That way, you can just add a class to each poem rather than a long string of line breaks. This is a great example provided by W3Schools
After adjusting the code to refactor the redundancy and adjusting the image overlap into a neighboring section, this code should be ready to merge and deploy.

Thanks Darya!  I took away the extra card on the homepage and added the word-wrap property in the main CSS file.


Input Focus for Screens with text boxes

All screens which include a text input box, such as enter new password or enter wallet password should set the text input focus to the first text box on the screen, so user doesn't have to click on it in order to enter text.
Some screens have this feature, some don't.  For example, set wallet password doesn't but login with password does.
Another example - send transaction screen - input focus should go to the To input field.

Closed by #800


fix #299 - Support Ord quotation

Fixes #299
Problem
Ord currently only can be quoted as a sortBy parameter.
Solution
Make Ord an AST.
Checklist

[x] Unit test all changes
[x] Update README.md if applicable
[x] Add [WIP] to the pull request title if it's work in progress
[x] Squash commits that aren't meaningful changes
[x] Run sbt scalariformFormat to make sure that the source files are formatted

@getquill/maintainers

:+1:

:+1:


feature-wip Support Q-Error to measure the accuracy of derived statistics

Proposed changes
Issue Number: close #xxx
Problem summary
Describe your changes.
Checklist(Required)

[ ] Does it affect the original behavior
[ ] Has unit tests been added
[ ] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.94 seconds
stream load tsv:          476 seconds loaded<PHONE_NUMBER>9 Bytes, about 149 MB/s
stream load json:         37 seconds loaded<PHONE_NUMBER> Bytes, about 60 MB/s
stream load orc:          67 seconds loaded<PHONE_NUMBER> Bytes, about 15 MB/s
stream load parquet:          28 seconds loaded 861443392 Bytes, about 29 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230227114753_clickbench_pr_105081.html

run buildall

run buildall

run buildall

run buildall

run buildall

run p0


Multi-stage build doesn't reuse previous filesystem correctly

Running a multi stage build like:
FROM centos:7 AS base
RUN touch /1

FROM base
RUN ls /1

should work.  Instead, base is interpreted as a local image.

Fixed by #72


[workloadmeta/store] Fix filters that match entities

What does this PR do?
Fixes a bug in the workloadmeta store. It was not working correctly for filters that match entities, like workloadmeta.IsNodeMetadata used by the metadata controller.
I've set the no-changelog label because this doesn't affect any released versions.
Describe how to test/QA your changes
Skip. It's a bit difficult to test this case manually, but I added a unit test that should cover it.

/merge


Add ability to toggle certain warnings off

First of all, thanks for making this extension and LSP! The coloring, hovers, and lint are awesome to have during schema development. 🎉
Onto the issue: My schema has something similar to this:
definition folder {
    relation editor: user
    permission create_child_folder = editor
    // ... etc
}

This produces this warning:
Permission "create_child_folder" references parent type "folder" in its name; it is recommended to drop the suffix
I understand what this warning is getting at, but it's not exactly relevant in this case – the "child_folder" just happens to be the same resource as the parent "folder", hence the suffix. Since other warnings are still valuable, it would be great to selectively turn off this warning.
Generally, I'd like to toggle on/off whatever warnings are best for me. Perhaps a // zed-disable-warning: parent-name-in-suffix at the top of the file, or something in the .vscode/settings file?

@mateenkasim Mind filing this against SpiceDB itself? All the diagnostics are generated by this package: https://github.com/authzed/spicedb/blob/main/pkg/development/warningdefs.go

Should be easy to add something that uses the comment metadata on the nodes and disables it

Happy to!


Bad regexp - usernameRegexp = ^([a-zA-Z0-9_.-@])+$

For test enter: test-test
Change to usernameRegexp = ^([a-zA-Z0-9_.\-@])+$

Are u sure those should be valid:

@test
-test
test-
.test

I think you need to have a regex that ensure such special characters just in the middle of the string.


Adds fade out / fade in duration control in Transition Service

Overview
Adds new methods to Transition Service that let you specify fade out / fade in duration.
FadeInTime and FadeOutTime are now treated as defaults.
Changes

Fixes: #6450

Verification

As a reviewer, it is possible to check out this change locally by using the following
commands (substituting {PR_ID} with the ID of this pull request):
git fetch origin pull/{PR_ID}/head:name_of_local_branch
git checkout name_of_local_branch

/azp run

/azp run


(fix): -- is showing in query console instead of value 0

Description
This PR fixes #8256

Codecov Report

Merging #8257 (879c5ee) into master (d999aec) will decrease coverage by 6.72%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #8257      +/-   ##
============================================
- Coverage     70.78%   64.05%   -6.73%
+ Complexity     4243     4242       -1
============================================
  Files          1631     1586      -45
  Lines         85281    83399    -1882
  Branches      12845    12641     -204
============================================
- Hits          60364    53423    -6941
- Misses        20746    26128    +5382
+ Partials       4171     3848     -323

Flag
Coverage Δ

integration1
?

integration2
?

unittests1
66.98% <ø> (+<0.01%)
:arrow_up:

unittests2
14.11% <ø> (-0.01%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d999aec...879c5ee. Read the comment docs.


Missing disease description or truncated

@elipapa commented on Fri Jan 12 2018
@gkos-bio commented on Fri Mar 17 2017
could be a processing issue in the data pipeline or something went wrong when EFO was released.

@gkos-bio commented on Fri Mar 17 2017
PTEN hamartoma tumor syndrome: description is truncated in ORDO

@gkos-bio commented on Fri Mar 17 2017
Cowden syndrome: the definition from Orphanet is truncated.
The definition from DO is better:
An autosomal dominant disease characterized by multiple noncancerous, tumor-like growths (hamartomas) and an increased risk of certain forms of cancer, especially breast, thyroid and endometrium. It is caused by mutations in the PTEN, SDHB, SDHD and KLLN genes. [ url:http://rarediseases.info.nih.gov/gard/6202/cowden-syndrome/resources/1 url:http://en.wikipedia.org/wiki/Cowden_syndrome url:http://ghr.nlm.nih.gov/condition/cowden-syndrome ]

@ElaineMcA commented on Mon Aug 13 2018
@gkos-bio to submit ticket to EFO.  https://github.com/EBISPOT/efo/issues/228
@gkos-bio to write query for inclusion in data pipeline QC to bridge any gaps within OT.

Another example of missing disease description: diffuse intrinsic pontine glioma
There is nothing on the Open Targets platform and this on the EFO website: 'A neuroglial tumor that arises from the middle portion of the brain stem. It usually affects children and has a poor prognosis'

We still do not have disease description. According to Adams QC output, none of the EFO terms has a description:
efo.count: 11435
efo.missing_description.count: 11435
Do we know yet why we do not have any disease descriptions?

Check post EFO 3

This seems to be because EFO changed to using http://purl.obolibrary.org/obo/IAO_0000115 to describe descriptions/definition and data_pipeline was not updated to match.
This has now been done as part of  af-87-missing-description.


Creates comparison table for VRF methods

Description
Creates comparison table for VRF methods on the VRF Intro page.
...
Changes

Breaks out bullet point recommendations for the Choosing the correct method section into a comparison table
Removes a redundant, outdated bullet point list of supported networks, per @aelmanaa's recommendation

While we're waiting on a Vercel preview:


build apk fail in flutter 3.27.*

Duplicate Check

[X] I have searched the opened issues and there are no duplicates

Describe the requested feature
(  ●   ) Building .apk for Android ⏳...
/C:/Users/user/AppData/Local/Pub/Cache/hosted/pub.dev/flet-0.25.2/lib/src/utils/theme.dart:356:28: Error: A value of type 'TabBarThemeData' can't be returned from a function with return type
'TabBarTheme?'.

'TabBarThemeData' is from 'package:flutter/src/material/tab_bar_theme.dart' ('/D:/flutter/flutter/packages/flutter/lib/src/material/tab_bar_theme.dart').
'TabBarTheme' is from 'package:flutter/src/material/tab_bar_theme.dart' ('/D:/flutter/flutter/packages/flutter/lib/src/material/tab_bar_theme.dart').
return theme.tabBarTheme.copyWith(
^
/C:/Users/user/AppData/Local/Pub/Cache/hosted/pub.dev/flet-0.25.2/lib/src/utils/theme.dart:438:28: Error: A value of type 'DialogThemeData' can't be returned from a function with return type
'DialogTheme?'.
'DialogThemeData' is from 'package:flutter/src/material/dialog_theme.dart' ('/D:/flutter/flutter/packages/flutter/lib/src/material/dialog_theme.dart').
'DialogTheme' is from 'package:flutter/src/material/dialog_theme.dart' ('/D:/flutter/flutter/packages/flutter/lib/src/material/dialog_theme.dart').
return theme.dialogTheme.copyWith(
^
/C:/Users/user/AppData/Local/Pub/Cache/hosted/pub.dev/flet-0.25.2/lib/src/utils/theme.dart:482:26: Error: A value of type 'CardThemeData' can't be returned from a function with return type 'CardTheme?'.
'CardThemeData' is from 'package:flutter/src/material/card_theme.dart' ('/D:/flutter/flutter/packages/flutter/lib/src/material/card_theme.dart').
'CardTheme' is from 'package:flutter/src/material/card_theme.dart' ('/D:/flutter/flutter/packages/flutter/lib/src/material/card_theme.dart').
return theme.cardTheme.copyWith(
^
Target kernel_snapshot_program failed: Exception

FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:compileFlutterBuildRelease'.

Process 'command 'D:\flutter\flutter\bin\flutter.bat'' finished with non-zero exit value 1

Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.
Get more help at https://help.gradle.org.

BUILD FAILED in 37s
(    ● ) Building .apk for Andro   38,3s
Gradle task assembleRelease failed with exit code 1
[04:29:39] Error building Flet app - see the log of failed command above.
Run subprocess: ['D:\flutter\flutter\bin\flutter.BAT', '--version']
[04:29:40] Run subprocess: ['D:\flutter\flutter\bin\flutter.BAT', 'doctor']
[04:29:47] Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel stable, 3.27.1, on Microsoft Windows [Versi¢n 10.0.19045.5247], locale es-AR)
[√] Windows Version (Installed version of Windows is version 10 or higher)
[√] Android toolchain - develop for Android devices (Android SDK version 35.0.0)
[X] Chrome - develop for the web (Cannot find Chrome executable at .\Google\Chrome\Application\chrome.exe)
! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.
[√] Visual Studio - develop Windows apps (Visual Studio Community 2022 17.10.2)
[√] Android Studio (version 2024.2)
[√] Connected device (1 available)
[√] Network resources
       ! Doctor found issues in 1 category.

Suggest a solution
No response
Screenshots
No response
Additional details
In flutter 3.24 work build apk, but if i upgrade flutter to 3.27.1 break.
it isn't the filepicker problem.
Android SDK version 35.0.0
flutter 3.27.1
java : openjdk version "1.8.0_432"
OpenJDK Runtime Environment (Temurin)(build 1.8.0_432-b06)
OpenJDK 64-Bit Server VM (Temurin)(build 25.432-b06, mixed mode)

Read this: https://github.com/flet-dev/flet/issues/4548#issuecomment-2558093530


fix publish go ahead even if quorum is not met

Proposed changes
pick: #18696
If publish txn is not quorum met, writes will be stucked. Let publish go ahead if each tablet contains at least one succ replica and time exceeds 10 min.
This PR also change some hints.
Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46 seconds
stream load tsv:          531 seconds loaded<PHONE_NUMBER>9 Bytes, about 134 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          64 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size:<PHONE_NUMBER>8 Bytes

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 50.48 seconds
stream load tsv:          532 seconds loaded<PHONE_NUMBER>9 Bytes, about 134 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          64 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size:<PHONE_NUMBER>4 Bytes


sprint-3

Codecov Report

Merging #3 into master will increase coverage by 24.62%.
The diff coverage is 34.23%.

@@             Coverage Diff             @@
##           master       #3       +/-   ##
===========================================
+ Coverage    5.26%   29.88%   +24.62%
===========================================
  Files           5       22       +17
  Lines          38      261      +223
  Branches       15       27       +12
===========================================
+ Hits            2       78       +76
- Misses         36      183      +147

Impacted Files
Coverage Δ

src/components/Logo.tsx
0.00% <0.00%> (ø)

src/components/Theme.tsx
0.00% <0.00%> (ø)

src/pages/404.tsx
100.00% <ø> (ø)

src/pages/_app.tsx
0.00% <ø> (ø)

src/pages/_document.tsx
0.00% <0.00%> (ø)

src/pages/article/[year]/[month]/[slug].tsx
0.00% <0.00%> (ø)

src/pages/index.tsx
0.00% <0.00%> (ø)

src/pages/page/[slug].tsx
0.00% <0.00%> (ø)

src/pages/section/[section].tsx
0.00% <0.00%> (ø)

src/components/Grid/context.tsx
2.85% <2.85%> (ø)

... and 28 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b898bd2...579e055. Read the comment docs.


[WIP] Add a table of contents to the blog posts

Added a prototype for a post's table of contents
Still missing:

[x] Make it optional

[x] Move changes in css to styles

[ ]  Maybe make the table of contents available in the menu on mobile? or collapsable?

[x] Improve design (I am not the right person for this)

I just realized I edited the css directly instead of through stylus (never heard of it before 😅). Do you want me to change that? I see that this is a question you ask in Issue https://github.com/jakewies/hugo-theme-codex/issues/2 but I don't know how to answer, web development is not my thing and I haven't been initated to any of the shiny stuff with their bells and whistles :)

Hmmm.
I would definitely prefer not touching the css directly. We should only be touching the source code, which lives in src/styles. The styles are written with stylus.
However, the asset pipeline for this project has not been implemented fully since I transitioned my site to a theme. Tomorrow I will spend some time thinking about how to get that up and running. I'll document how to edit the styles in a CONTRIBUTORS.md file. This should give you a good landscape to work with.
That would be a blocker for this PR.

OK, thanks!

Hey @xvallspl,
I updated the asset pipeline in a PR earlier this morning. I also added a CONTRIBUTING.md file to help onboard contributors such as yourself! 😄
Here's the rundown for css:

Each page has a corresponding css file made up of unique styles for that specific page and partial css for other common use cases.

The source for these styles is in src/styles. They are written in stylus.

To make changes to css, make sure you are watching the assets so gulp can rebuild the static directory on any changes.

A good suggestion for developing this theme is to:

Run yarn develop in one terminal window. This will serve the example site.
Run yarn watch:assets in a second terminal window. This will trigger a rebuild of the static directory when any changes have been made.

**NOTE: The static directory is not playing nice with git. For reasons I'm not yet sure of, git is constantly marking all css files in the static directory as "modified". It might have something to do with the sourcemaps.

Thanks!

Hey @xvallspl,
I updated the asset pipeline in a PR earlier this morning. I also added a CONTRIBUTING.md file to help onboard contributors such as yourself! 😄
Here's the rundown for css:

Each page has a corresponding css file made up of unique styles for that specific page and partial css for other common use cases.
The source for these styles is in src/styles. They are written in stylus.
To make changes to css, make sure you are watching the assets so gulp can rebuild the static directory on any changes.

A good suggestion for developing this theme is to:

Run yarn develop in one terminal window. This will serve the example site.
Run yarn watch:assets in a second terminal window. This will trigger a rebuild of the static directory when any changes have been made.

NOTE: The static directory is not playing nice with git. For reasons I'm not yet sure of, git is constantly marking all css files in the static directory as "modified". It might have something to do with the sourcemaps.

From what I can see, the reason why git is marking them as modified is because they are. The files are missing linebreaks now. Why does that happen, I don't know yet 🙃

My first thought is that it has something to do with prettier. I'll do some more digging.

Hugo has a ToC document that could be helpful. Not sure if you are using it or not 👍

Yes, that's what generates the ToC automatically

@xvallspl just peeped your site and I love what I'm seeing with the ToC <3

Thanks! I just reworked the commits so the changes are applied in stylus and not in the css and I added an opt-in parameter to it

@jakewies I'm not sure having the ToC on mobile is one of my priorities right now. Maybe we can add it as an issue for later or for someone to take. As for making everything prettier, can I ask for your help? In that case we could merge as soon as you are happy with the changes and I applied your feedback

Agreed. We can hide on mobile for now. I’ll def help out where needed. Let me know when you’ve wrapped up your work on this branch and I’ll hop in.

You can hop in now if you want

Somewhat painfully rebased to the new layout structure.

Somewhat painfully rebased to the new layout structure.

Sorry about that! 😅
@kentnek is making some changes to the asset pipeline. We'll be switching away from stylus. It'll be good for this project in the long run, but I know it puts some strain on your work for the moment. We'll get this feature in the project soon though. I think it will be a great addition!

I actually appreciate his changes as I had something like that on the wishlist 😁

@xvallspl Sorry I've neglected this PR a bit. What's the status? Is it time for me to hop in and take a look at the styles?

I'll adapt it to the scss first!

Rebased to the latest master.
@jakewies Whenever you want :)

I'll take a look at the styles later today.

Just realized that this moves the blog post from the center of the page 🤔

Just realized that this moves the blog post from the center of the page 🤔

hmm this might be problematic if the user doesn't want the TOC but keep the blog's contents centered as normal.

We should discuss screen sizes and breakpoints. I would really like to keep the core layout of this blog-post template in tact. That is:
.post {
  max-width: 34rem;
  margin: 0 auto;

A post will be full width up to 34rem and centered. The Table Of Contents should be constructed in such a way that it does not affect the overall flow of the post layout, but instead gracefully works along side it.
I think this can be achieved, but we need to hide the ToC at some point. And since this is predominantly a desktop feature, I think we should hide it sooner than it's being hidden now (at 800px). It would be nice to support tablets, and I think we could with proper styling.
What are your thoughts @kentnek @xvallspl ?

I'm not the CSS expert here so @jakewies please chime in 😅
Also, please have a look at my comments on the
 heading and the js code above.

There are a number of discussions throughout this PR and I want to reign in what we're talking about and what needs to be addressed in this post:
Todo

[ ] General Styles
[ ] Decide when to hide the ToC
[ ] Only check for toc: true in post frontmatter
[ ] Update README to explain how to hide/show ToC for a post
[ ] Decide whether or not to put the post title in the ToC
[ ] Clean up JS

Bugs

I'm noticing some buggy behavior with the JS as it relates to the active classname on ToC text. Sometimes when scrolling I'll notice that a heading in the ToC gets skipped completely. This may have something to do with the way the JS is written.

I'm examining the "Markdown Syntax Guide" example post with toc: true, and some of the headings aren't being put into the ToC. These include the h1, h4 and h6 headings. Is this by design of Hugo? Doesn't make sense to me why they aren't being included.

@jakewies on the bugs

Yes, it's tricky. I was working on it right now 😄
2 .  You can specify until which level hugo parses the h tags.

Just realized that this moves the blog post from the center of the page 🤔

hmm this might be problematic if the user doesn't want the TOC but still keeping the blog's contents centered as normal.

Oh, IIRC it only happens with the toc visible.

You can specify until which level hugo parses the h tags.

This is a Hugo level config?

This is a Hugo level config?

yep check my previous comment.

when scrolling I'll notice that a heading in the ToC gets skipped completely.

@jakewies I think I got the bug when I was using the original JS code too, but changing to intersectionRatio > 0 makes the bug go away. However the side effect is multiple items in the TOC get highlighted if the corresponding sections are on the page all at once.
I think @xvallspl's intention is to highlight only one item in the TOC, but that makes the highlighting a bit finnicky.

Oh then I think we just need to float the TOC on the right and the post should be centered in the remaining space.

I'm working on styles now 👍

^ oh so I kinda guessed @xvallspl's intention correctly.
I was giving some thought to this problem too and I realized it might not be possible to solve this satisfactorily.
Let's imagine in the perfect scenario, when the user starts scrolling down, and we're only highlighting one item. This basically means we're marking the last (vertically) visible item (intersectionRatio > 0) as active. This works fine for later sections, but at the beginning, if we have, say, three
s at the top, then the first two
 would never get highlighted. Or if you highlight the first
 from the beginning, then the second heading will be skipped.
I think for simplicity's sake, highlighting multiple headings is fine and it only happens if those sections are too short vertically.

Yeah, the solution it's not perfect indeed. In this case I am following a simple rule: always highlight the last header that has intersected with the page.
If you prefer to go for the simplified version I can revert the commit.

always highlight the last header that has intersected with the page.

yeah this is similar to the scenario I describe above, which will result in headers being skipped at the top of the page. I know that you've been spending some time with this problem and I really appreciate that, but I think let's keep it simple and let users override the script for more complex behaviors if they really want to.

Here's another thought that I just want to toss in: How crucial is it to have highlighting at all?

@jakewies it's nice to have, especially on a super long blog post, you'd love to know where you are. So i think the simple solution works fine (@xvallspl has already reverted to that)

I spent a good amount of time with they post layout now that I've been working on the style of the ToC. I believe I've created a solution that not only looks good enough for posts with a ToC but remains as visually similar as possible to posts without a ToC.
While doing this I came across a few pain points surrounding the entire layout of the site. That's just a feeling I have towards something I built a long time ago. I would like to revisit it in the future but for now I think we can move forward with these styles.

Now that we've opted for the simple solution of highlighting any heading that is appearing on the screen, I think it would be ok to add the post title back in. What do you all think?
I am playing around with it and I actually like how multiple headings are being highlighted. It's acting as a "north star" and gives me context of where I'm at. Plus clicking any of the items brings you right where you want to go.

I knew that I had to split that commit in two 😆
I don't have a preference, really

@jakewies ooh I like what you are doing there with the styles! However I think the left-right margins become non-existent on phones now:

For the title, I think instead of the JS hack, we can do this way:
      {{ if .Params.toc }}

          {{ if .Site.Params.showPageTitleInTOC }}
{{.Title}}
 {{ end }}
          {{ .TableOfContents }}

      {{ end }}

here the class .toc-container would have the sticky CSS styles of the current #TableOfContents. below is a POC:

@jakewies ooh I like what you are doing there with the styles! However I think the left-right margins become non-existent on phones now:

I might be speaking too soon but I don't see this happening on every page. It could be something about the Markdown example post that's exposing a weakness in the styles. Will debug more later. Need to work lol
If any of you would like to take a crack at it though, by all means 😄

I don't see this happening on every page

Oh I think it's just my Chrome Inspect's Responsive mode behaving strangely. I tried resizing my browser instead and it worked as expected hmm
Other than that I think this is great already.
On a side note, I noticed that the burger bar is hidden on scroll now (which is quite cool and will probably fix #53). Just curious, how did you achieve that? I didn't see any changes to the burger container's styles.

the burger bar is hidden on scroll now

Never mind. I think that Markdown example page has some weird bugs on mobile. It turned out that the page was initially larger than the screen's size (so it was zoomed in). After I zoomed out to 100% the burger bar became sticky again. Here's a video of that.

I've seen it too. It's buggy for sure. I'm not 100% sold that it's the markdown page but it could be.  The ToC is getting visibility: hidden at 1024px and below, and normally this could be accomplished with display: none but because we're acting upon the ToC with JS, we can't use display: none. When manipulating the screen size I think there is some bugginess that might be attributed to the css style.

I know why. It's because of the table in the middle of the page.
I've seen websites making tables horizontally scrollable when it's too long, but I'm not sure how to do that (other than specifying overflow-x: auto, which seems not to work in this case). Could you help with that when you're free?

@kentnek how does the site param showPageTitleInTOC work? I tried adding it to config.toml but it has no effect.
Should we just wipe this option and default to showing the title in the ToC at this point? I feel like the use case is so small that we don't need to support it unless there is enough demand for it.

The last thing that needs doing, as @kentnek said in his previous post, is to document the usage of the ToC. This is a good time to discuss post frontmatter and general configuration of this theme.
I think it's best to be as explicit as possible to avoid confusion, so my proposal is that we update the blog archetype with all possible frontmatter that exists for a blog post. The archetype will include the default values for each blog post, and users can change them on a post-by-post basis.
In my opinion this is a better solution than letting users update post configuration in the site params via config.toml. Yes it requires a few more keystrokes but it's very explicit.
The example posts can all be set to toc: false except for one of them to show off how the ToC looks. I will add a commit to include these changes and some docs in the README and then if everything looks good to you guys we can merge!

Ok documentation is updated and all that's is to have you guys do a once over on everything before I rebase with master and merge this puppy.

Should we just wipe this option and default to showing the title

Oh I actually want to not showing the title by default 😆 So I figured just make it an option so that I don't need to override the layout later on.
Other than that and the wide table bug, I think the PR is good now.

@jakewies @xvallspl shall we merge this PR today?

@kentnek Yup I have some time this morning to take care of it. I'll handle the conflicts as  well.

Hey @xvallspl, thanks a ton for your work on this 🙌

Maybe make the table of contents available in the menu on mobile? or collapsable?

We can discuss this in a future PR. For now I'm going to merge this and cut a new release of the project. This was a fun feature to work on with you and @kentnek !

@all-contributors please add @xvallspl for code, design

@jakewies @kentnek Thanks for everything guys! I'm sorry I haven't been able to follow this for the last days.
Now I only have a couple of issues with the result: the fact that the post is not centered in the screen (I guess this is by design, as it wasn't before either), and that I see the table of contents conceptually belonging to the post. I find it out of place so far away from the post in a big screen. A  couple of screenshots:

Although this might be only me, I would like to hear your thoughts on it.

Thoughts? @jakewies @kentnek
Sorry I didn't look at this for weeks

^ yeah I actually had the idea of moving it a bit closer to the actual contents, so the whole (content - TOC) block is centered instead. Also for mobile the TOC can be right after the main title (or the  hook, although I'm not sure how to achieve that yet).
However I recently found a layout bug that affects contents that are too wide on mobile (and fixed that on my own fork), so I will deal with the TOC again once I finish fixing that.

Thanks @kentnek !

Hey, I noticed that the site param showPageTitleInTOC (introduced in this commit) hasn't been documented yet. Should I include it in the example site config and the README and create a PR?

Hey @SanchithHegde that would be awesome if you.


support for CSRF

Request
Context: I tried to integrate the ohif-viewer (thank you very much for your work, by the way) as a npm package to an application based on a framework using protection against csrf attack by default (type Django, Symphony...)
The problem here, this automatic protection need to add hidden field to post form (basically), to use a csrf token. With react package, it need to be set at the configuration step of the application to be sure to have the csrf token in the post forms: I didnt find any examples to set the viewer appropriately so maybe it is not possible as it ?
Thank you in advance
ps: the problem is visible only with POST request. I noticed it while implementing STOW-RS requests.
ps:  example with Django and react here
...
Why should we prioritize this feature?
I would improve the integration as a package and improve the security of the ohif viewer usage.
...

This seems like a server, concern, am I missing something?


[Feature Request] Support for Glob Pattern for module mapper configuration

Ideally it would be very simple to configure Jest to use the same glob pattern that typescript uses to resolve paths.
Initially I assumed it had that type of support, but discovered it's regex only per this so question.
If this is supported users can use the same glob pattern in jest that they would use in typescript.

Feel free to suggest it here: https://github.com/facebook/jest/issues/7185 we're working on simplifying the configuration and likely going all-globs + functions


scFoundation  how to call API for perturb-seq task?

Again, thanks for the great work - for the Perturb-seq task (your scFoundation model embedding + GEARS), you state in the README that "You can change the code in gears/model.py Line 130 to call the scFoundation API for model training".
Since line 30 (https://github.com/biomap-research/scFoundation/blob/1571ef085006aac63fa04fb592236f3198bd99d1/GEARS/gears/model.py#L130) currently returns a random gene embedding return torch.rand(x.shape[0], x.shape[1]-1,hidden_size).to(x.device), what exactly should we replace this with to call the API?
==============================
(Also, a small conceptual question - you say in the README that "you will get the gene context embeddings for each training batch with shape batch * 19264 * hidden" - but why is it 19264 (which is the size of the number of genes in your pretraining pipeline) and not the number of genes in the Perturb-seq dataset?)
Thanks again!
Patrick

Hi,
you can send the batch gene expression data into API, get the gene embeddings with gene batch mode from our API, and then just return them in Line 130. The output embeddings of the API have the same shape as this random embedding.
For the gene number, we uniformed the gene symbol of Perturb-seq data into 19264 size before training the perturbation model. So it has 19264 in shape 1.


Investigate pint instead of quantities

Investigate whether pint would be a good substitute for quantities.

There is already activity on this, see
https://github.com/NeuralEnsemble/python-neo/pull/297


Error when using SAS-policy and connection string with Service Bus Queue

When I use RootManageSharedAccessKey for the Service Bus Namespace I can see everything, see below

When creating a SAS policy (Manage, Send, Listen) on the queue only and connecting using the connection string:

<08:55:02> Failed to retrieve Service Bus topics. Exception: Microsoft.ServiceBus.Messaging.MessagingException: Cannot get entity 'my-queue' because it is not of type TopicDescription. Check that you are using method(s) with the correct entity type. ---> System.Runtime.Serialization.SerializationException: Error in line 1 position 2599. Expecting element 'TopicDescription' from namespace 'http://schemas.microsoft.com/netservices/2010/10/servicebus/connect'.. Encountered 'None'  with name '', namespace ''.

<Error>401<Detail>User is not authorized. TrackingId:...-...</Detail></Error>

When creating a SAS policy (Manage, Send, Listen) on the topic and working with subscriptions it seems to work:

Anything that I might have missed when connecting to queues?
Thanks for assisting!

For me it does not work with queues nor topics. There are at least two PRs that added/enhanced this funtionality, #375 and #562.
@rajrao and @avdv, does it work for you?

Latest version 5.0.7 also has the same issue "User is not authorized" while using SAS-Policy in Queue, whereas only Topic Subscription works fine.

Any news on this issue. We're facing the same problem.

Based on my local debugging, it seems the issue is caused by the usage of ServiceBusHelper2 in HandleQueueControl and HandleTopicControl
Basically when you select a queue / topic with a entity level policy it attempts to use the ServiceBusAdministrationClient (via ServiceBusHelper2) (and it fails, as expected).
On my local machine, to simply work around this, I've commented the usage of ServiceBusHelper2 in HandleQueueControl, however I would defer to you for a long term solution, as I'm not familiar enough with the original intentions behind this design.

When testing this on the latest commit in the develop branch both connecting to a queue and a topic works for me but I do get tons of error messages in the log window.
Is it only the error messages that is the problem?

It connects successfully, however if you select the topic/queue the view to the right of the tree view doesn't load, and an exception is logged.

@ErikMogensen - not sure if you got the chance to look into this.

@AntonBarwald, @chandraatgithub, @cosminstirbu
Please test https://github.com/paolosalvatori/ServiceBusExplorer/releases/tag/5.0.15-preview and report back whether this issue is resolved or not. Please keep an eye out for issues regarding connecting.

Tested the preview version and it fixed the issue. So far so good, didn't find any other issues, although my testing was rather limited.
Thank you!

Happy to read that!


Replace .border-radius mixin with border-radius css in brackets_patterns_override.less

Description
In src/styles/brackets_patterns_override.less there are some mentions of .border-radius mixin.
These can safely replaced with the corresponding css border-radius.
So .border-radius(0 0 3px 3px); will be border-radius: 0 0 3px 3px;
There are some other mentions in the tree, but they are in bootstrap files or in files where it is necessary, so there should be left untouched.

Hi, I would like to take this issue

Made a pull request to fix this issue. Checkout #13235


Using Keyboard shortcuts to adjust the selection box

When working with small note values it may be beneficial to be able to use keyboard shortcuts to adjust the selection box
For example the keyboard shortcuts could be as follows:
Shift + Left Arrow Key: Moves the left bound one snap unit left
Ctrl + Shift + Right Arrow Key: Moves the left bound one snap unit right
Shift + Right Arrow Key: Moves the right bound one snap unit right
Ctrl + Shift + Left Arrow Key: Moves the right bound one snap unit left

Done, LMK if there are any issues.


Enhancements for OpenStack security group auditor

Handle null cidrs for Rackspace Internet accessible security groups with unit tests
Officially add #1039 fix to OpenStack security group auditor with unit test
Fix singular/plural names in a few OpenStack watchers

Coverage increased (+0.4%) to 72.752% when pulling 805772e2f70b9e749a59466b632c1923779bcd2d on mstair:rackspace into 7d22cc153d31728c23b1ab5ac6b9e2339e63e46a on Netflix:develop.


Heatmap: Flip horizontally

**ScottPlot Version:**4.1.61
Would it be possible to include a flag for the heatmap to flip it horizontally? There is one for flipping vertically and it would be great for completeness.

@Neopentane1 I don't think this is possible, but it would be a great addition to ScottPlot!
Do you want to make the code change and submit it as a pull request? If so, I would be happy to review it and merge quickly so it can be in the next release!
Looking at how FlipVertically is implemented, I'm sure you can squeeze in a similar FlipHorizontally option:
https://github.com/ScottPlot/ScottPlot/blob/181be953b0544f520a57fd29ae881ad1df02a4d6/src/ScottPlot4/ScottPlot/Plottable/Heatmap.cs#L171-L174
https://github.com/ScottPlot/ScottPlot/blob/181be953b0544f520a57fd29ae881ad1df02a4d6/src/ScottPlot4/ScottPlot/Plottable/Heatmap.cs#L461-L464
Related:

#1864

PS: I tried flipping it like this but it did not work:
var plt = new ScottPlot.Plot(600, 400);
double[,] imageData = ScottPlot.DataGen.SampleImageData();
var hm = plt.AddHeatmap(imageData);
hm.XMax = 0;
hm.XMin = imageData.Length;
plt.SaveFig("flip.png");


Gdsfactory-gen Improvments

TODOs for improving gdsfactory-gen code. Feel free to add more items here:

[ ] ReadME enhancements + add documentation content to the OpenFASOC docs
[ ] rename pygen
[ ] organize gds files in tapeout_and_RL directory
[ ] import klayout drc code from pdk (.drc script)
[ ] publish package to pypi
[ ] comments regarding port trees
[ ] python notebook demo
[ ] expand the existing docs directory (with the txt files)
[ ] follow up on requirements.txt: GDSFactory, sky130, and GF180 pypi packages cannot be installed at the same time without broken deps
[ ]  reformat python docustrings into markdown format for better printing in VScode

@saicharan0112 @msaligane @alibillalhammoud

Added comments from @harshkhandeparkar

follow up on requirements.txt: GDSFactory, sky130, and GF180 pypi packages cannot be installed at the same time without broken deps

Elaborating on this: gf180 and sky130 have different version requirements of gdsfactory. sky130 requires v7.3.0 and gf180 requires v7.1.4. When either package is installed, it replaces the previously installed version of gdsfactory, leading to broken dependencies. The following error is displayed on installing sky130 after gf180.
Successfully uninstalled gdsfactory-7.1.4
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
gf180 0.0.3 requires gdsfactory[cad]<7.2.0,>=7.1.1, but you have gdsfactory 7.3.0 which is incompatible.
Successfully installed gdsfactory-7.3.0

Even though this error is displayed, the generators still seem to work.

Opened an issue in the relevant gdsfactory repo: https://github.com/gdsfactory/skywater130/issues/73.

I am closing this issue and creating a new one as these tasks have been completed or deprecated


Add onSelectedRangeChange

Description
To what I understood, which can be wrong, the UITextView/NSTextView is hidden therefore it's impossible to know where the cursor is within the document. So I exposed both bindable for selection range value.
Example
It's working for my iOS application and SwiftDownDemo.
Feel free to close or request changes. I'm happy to contribute.

I'm sorry I managed to run SwiftDownDemo and found out it's not working on macOS.

Fixed

Hey @ingun37,
Thanks for your contributions, I'm currently moving from one location to another so I might not be able to take a closer look right now. But I will as soon as possible!

@qeude This one I'll finish soon

I'm having trouble setting up my new laptop. I know I can just edit from GitHub since it's a very simple fix but I just want to run it before merge just in case. So if it's not in hurry please wait.

Hi @ingun37,
No worries, there is no rush on this 🙏 Take your time and do this when you are ready to do so 😄

@qeude Removed HStack. Please review again.


Subscriptions Overhaul

Fixed diagram - it is now clearer and much more user friendly
Image mapping for JSON file
Added information on adjustment requests
other minor fixes

lgtm


Cluster config parsing

What are we solving and why

We lack an entity that represents a full cluster configuration.

We have cluster.Spec but it doesn't have provider specific api structs (like machine configs or datacenter).
This has created duplicated logic, pushing providers into implementing the same thing again and again.
It complicates providers: since they don't receive their specific api structs as method arguments, they need to store them as members. This introduces complexity for both data management and construction. Moreover, it tides providers to a specific cluster config as a side effect.

The api package has grown a lot, assuming too many responsibilities. It unmarshals, sets defaults and validates. And all of this tied to an inflexible public api that only takes paths to files in local disk as input.

This makes it difficult to construct, validate and set defaults for these structs in alternative ways.
This has been a patent problem in our E2E tests, which would benefit significantly from more flexibility on this area.
In order to move to a controller based workflow, we need to be able to run some of this logic in the validation and mutation webhooks.

Proposal
I propose to introduce a new entity:
type Config struct {
	Cluster               *anywherev1.Cluster
	VSphereDatacenter     *anywherev1.VSphereDatacenterConfig
	DockerDatacenter      *anywherev1.DockerDatacenterConfig
	VSphereMachineConfigs map[string]*anywherev1.VSphereMachineConfig
	OIDCConfigs           map[string]*anywherev1.OIDCConfig
	AWSIAMConfigs         map[string]*anywherev1.AWSIamConfig
	GitOpsConfig          *anywherev1.GitOpsConfig
}

cluster.Config represents the whole cluster API definition, including all necessary objects specified by the user and linked to the Cluster object.
cluster.Spec will wrap around cluster.Config, adding other external data objects that complement the cluster config to form the full cluster specification
Why exported fields? This facilitates the mutation of the cluster config: eg. defaults being set by other packages like providers. It also facilitates testing, since any test package can set the data appropriately. This is supposed to be a data container and doesn’t need to enforce anything else. As a drawback, any piece of code receiving cluster.Spec can modify the cluster configuration. It is up to us to define when it’s ok and when it’s not to modify it. The only decision this proposal make in that regard is that validations should never modify the cluster config.
The implementation should allow to construct a cluster.Config from a:

Yaml file

When parsing from a yaml file, and due to the fact that most flows in the CLI happen without going through the kube api server, we need to “manually” take care of validation and defaults. This should be independent from parsing and the logic should be reusable even when not building cluster.Config from a file.

Kube api server (using a Kubernetes client)

When using the api server, defaults and validation should already be taken care of (by the validation and mutation webhooks), so this implementation should assume so.

Validations
There are two types of validations:

Context agnostic validations: there are no external dependencies, having the API object is enough to validate it: mutually exclusive fields, string format, empty fields…

These should live in the pkg/api/version package
These should be the ones called in the validation webhook

Context aware validations: these are dependent on other entities like clients or even other API objects

These could leave in different places, I’m not making a decision right now: providers, validations package… This is probably worth a separate discussion.
Try to keep these modular and reusable (eg. Don’t make them part of the providers struct’s api)
If they only need other API objects from the same package, they can live in the api package
Controller: these shouldn’t be run in a webhook and must be part of the main reconciliation loop

Defaults
These follow the same categorization and rules as the validations:

Context agnostic

In api package
Run un mutation webhook

Context aware

Keep them modular
In controller reconcile loop

Implementation plan

Add new cluster.Config struct (this PR)
Add utils to build Config from yaml (this PR)
Make sure all validations in the api/v1alpha1 package comply with the proposed definition of context unaware validations
Make sure all defaults in the api/v1alpha1 package comply with the proposed definition of context unaware defaults
Add some validate tooling to cluster.Config to be able to run api level validations
Embed cluster.Config in cluster.Spec

Testing (if applicable):
The PR includes unit tests
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/lgtm


Error: Calling Formula.sha1 is disabled!

Bug reports:
When I try to run, brew install https://raw.github.com/quantiverge/homebrew-binary/pdftk/pdftk.rb
it gives me an error like below
######################################################################## 100.0%
Error: Calling Formula.sha1 is disabled!
Use Formula.sha256 instead.
/Users/naggigoishi/Library/Caches/Homebrew/Formula/pdftk.rb:16:in `<class:Pdftk>'
Please report this bug:
  https://git.io/brew-troubleshooting
/usr/local/Homebrew/Library/Homebrew/utils.rb:85:in `odeprecated'
/usr/local/Homebrew/Library/Homebrew/utils.rb:93:in `odisabled'
/usr/local/Homebrew/Library/Homebrew/compat/sha1.rb:3:in `sha1'
/Users/naggigoishi/Library/Caches/Homebrew/Formula/pdftk.rb:16:in `<class:Pdftk>'
/Users/naggigoishi/Library/Caches/Homebrew/Formula/pdftk.rb:12:in `load_formula'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:25:in `module_eval'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:25:in `load_formula'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:42:in `load_formula_from_path'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:97:in `load_file'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:153:in `load_file'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:88:in `klass'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:84:in `get_formula'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:241:in `factory'
/usr/local/Homebrew/Library/Homebrew/extend/ARGV.rb:43:in `block in formulae'
/usr/local/Homebrew/Library/Homebrew/extend/ARGV.rb:41:in `map'
/usr/local/Homebrew/Library/Homebrew/extend/ARGV.rb:41:in `formulae'
/usr/local/Homebrew/Library/Homebrew/cmd/install.rb:107:in `install'
/usr/local/Homebrew/Library/Homebrew/brew.rb:94:in `<main>'

I have no idea what I should do.  And it says

Please report this bug:

So I am reporting this.
Thank you!
brew config
ORIGIN: https://github.com/Homebrew/brew
HEAD: 0f529dae1043639058130092cf35e68d18907cb8
Last commit: 12 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 1a7bfb82fde7ed52ee035e863fa8c7e85a852a71
Core tap last commit: 18 minutes ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_REPOSITORY: /usr/local/Homebrew
HOMEBREW_CELLAR: /usr/local/Cellar
HOMEBREW_BOTTLE_DOMAIN: https://homebrew.bintray.com
CPU: quad-core 64-bit broadwell
Homebrew Ruby: 2.0.0-p648
Clang: 7.3 build 703
Git: 2.7.4 => /Library/Developer/CommandLineTools/usr/bin/git
Perl: /usr/bin/perl
Python: /Users/naggigoishi/.pyenv/shims/python => /Users/naggigoishi/.pyenv/versions/3.5.2/bin/python3.5
Ruby: /Users/naggigoishi/.rbenv/shims/ruby => /Users/naggigoishi/.rbenv/versions/2.2.3/bin/ruby
Java: 1.8.0_101
macOS: 10.11.6-x86_64
Xcode: N/A
CLT: <IP_ADDRESS>.1.1461711523
X11: N/A```

The tap you're trying to install from (https://github.com/quantiverge/homebrew-binary) needs to update its formula to use SHA256 checksums instead of SHA1.  But that repository looks abandoned.


add 404 page

fixes #167
@blukai feel free to push to this branch for the pr if you want to make it more elaborate.

Do we want to merge now or wait for the @blukai treatment? :)

You can merge it if you want, we can always open more PR's.


[Connectedk8s] Update distro/infra enum list

This checklist is used to make sure that common guidelines for a pull request are followed.

For new extensions:

[x] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your PR is merged into master branch, a new PR will be created to update src/index.json automatically.
The precondition is to put your code inside this repo and upgrade the version in the PR but do not modify src/index.json.

Connectedk8s

Connectedk8s