Snakemake 7.9.0 may fail with "Error recording metadata for finished job: File name too long"

Snakemake version: 7.9.0

Describe the bug

Snakemake 7.9.0 may fail with "Error recording metadata for finished job: File name too long"
Logs

Relevant error message (after replacing the base dir and the base64 encoding):
WorkflowError:
Error recording metadata for finished job ([Errno 36] File name too long: '/path/to/workdir/.snakemake/metadata/tmp<base64_encoding>'). Please ensure write permissions for the directory /path/to/workdir/.snakemake

Minimal example

rule test:
    output: "012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678"
    shell: "echo 'ok' > {output}"

Command failing with Snakemake 7.9.0, succeeding with Snakemake 7.8.5:
snakemake -c1

Additional context

The issue doesn't happen with snakemake 7.8.5. It could have been introduced by https://github.com/snakemake/snakemake/pull/1745 in https://github.com/snakemake/snakemake/blob/6968fe7cfa3ec2f1e01862c5b270622d0ecbc2b0/snakemake/persistence.py#L436
Here, the tmpfile is built with suffix=os.path.basename(recpath), but the resulting name may be longer than 255 characters.

I get the same error with Snakemake 7.9.0
Failed to set marker file for job started ([Errno 36] File name too long: '/xxx/.project-scratch/2/2fcbfbef-13aa-451e-b420-00e21e2a35c3/xxx/Analysis/xxx/test/run1/.snakemake/incomplete/tmpo9grzin1L2VpLy5wcm9qZWN0LXNjcmF0Y2gvMi8yZmNiZmJlZi0xM2FhLTQ1MWUtYjQyMC0wMGUyMWUyYTM1YzMvQ0ItUFBCRlgtMTAzNF9Ub3plcl9FSV9MU19FTlEtNDgzOV9BXzAxL0FuYWx5c2lzL2VpZ2F0ay0xLjAuMV9DQkcvdGVzdC9ydW4xL291dHB1dC9zYW1wbGVzL0E2Nzg3MV8xL0E2Nzg3MV8xLmJ3YV9tZW0uY29tcGxldGVk'). Snakemake will work, but cannot ensure that output files are complete in case of a kill signal or power loss. Please ensure write permissions for the directory /xxx/.project-scratch/2/2fcbfbef-13aa-451e-b420-00e21e2a35c3/xxx/Analysis/xxx/test/run1/.snakemake
WorkflowError:
Error recording metadata for finished job ([Errno 36] File name too long: '/xxx/.project-scratch/2/2fcbfbef-13aa-451e-b420-00e21e2a35c3/xxx/Analysis/xxx/test/run1/.snakemake/metadata/tmpi6pulmb1L2VpLy5wcm9qZWN0LXNjcmF0Y2gvMi8yZmNiZmJlZi0xM2FhLTQ1MWUtYjQyMC0wMGUyMWUyYTM1YzMvQ0ItUFBCRlgtMTAzNF9Ub3plcl9FSV9MU19FTlEtNDgzOV9BXzAxL0FuYWx5c2lzL2VpZ2F0ay0xLjAuMV9DQkcvdGVzdC9ydW4xL291dHB1dC9zYW1wbGVzL0E2Nzg3MV8xL0E2Nzg3MV8xLmJ3YV9tZW0uY29tcGxldGVk'). Please ensure write permissions for the directory /xxx/.project-scratch/2/2fcbfbef-13aa-451e-b420-00e21e2a35c3/xxx/Analysis/xxx/test/run1/.snakemake

Thanks, and sorry! Will  be fixed with PR #1780

Thanks a lot @johanneskoester

I think I'm experiencing a similiar error on windows, snakemake version is 7.32.3
snakemake.exceptions.WorkflowError: Error recording metadata for finished job ([WinError 3] The system cannot find the path specified: 'C:\Users\Administrator\Downloads\\.snakemake\metadata\tmpwwx1rojg.QzovVXNl' -> 'C:\Users\Administrator\Downloads\\.snakemake\metadata\QzovVXNlcnMvQWRtaW5pc3RyYXRvci9Eb3dubG9hZHMvT25lRHJpdmVfMV84LTI5LTIwMjMvQW5ub3RhdGlvbnMvSEktVGVhbS1EZWVwTGVhcm5pbmctSEUtZnJvbS1EQVBJLUUyMi04ODhfcHJlZC5wbmcuYW5ub3RhdGlvbnMueG1s'). Please ensure write permissions for the directory C:\Users\Administrator\Downloads\**************************************\.snakemake
Path length is well over the 256 windows default


Mongo Source/Dataset support

Load MongoDB sources (databases) and datasets (collections).  Detects mappable data by looking for a spatial index ('2dsphere') or a 'geometry' attribute in the collection, and if found, creates a geojson object from the collection and saves it in the dataset's metadata.  If not found, allows the user to specify lat/long fields via the old table-mapper functionality formerly found in Minerva.  Brings back some other old minerva code as well to display the data on the map.

Very nice @mbertrand two awesome features in a single day :smile:

:+1: for @jbeezley img, that's one of the best I have seen.

I rebased this on master and may push some more commits as part of QA.

The endpoint for mongo dataset creation was creating an additional item.  I've also removed the dataset_id property from the metadata.  My reasons for moving away from that because it is tracked in the item itself, it would be hard to keep all those consistent if items were copied, and I no longer need that in the metadata because I'm having dataset creation endpoints return the full item instead of just the minerva_metadata.
TODO: document these practices in dev guide.

@mbertrand Can you add some datasets and tests that exercise the 2dsphere and geometry properties/paths through the code?  An extract of a handful of rows would be nice, and it would allow me to play around with that dataset.

@mgrauer sure, will do

On Nov 19, 2015, at 11:01 PM, mgrauer<EMAIL_ADDRESS>wrote:
@mbertrand https://github.com/mbertrand Can you add some datasets and tests that exercise the 2dsphere and geometry properties/paths through the code? An extract of a handful of rows would be nice, and it would allow me to play around with that dataset.
—
Reply to this email directly or view it on GitHub https://github.com/Kitware/minerva/pull/206#issuecomment-158273354.

In the interest of not adding more stuff that works only part of the time, could you remove the Date filter from the key mapping widget?  If we followed this policy even further, it would suggest that we should change the keymapper to something a bit more specific, since it only allows you to map to GeoJson points and it would show up even if not appropriate--but I can live with this for now since it at least works in the general case that your data has lat/long.

Your source is for a Mongo database, and then you create datasets for individual collections.  If the dataset has GeoJson in its metadata, which would come from a conversion of every row in the collection to GeoJson, it can be visualized on the map.  You do allow in the GeoJson conversion for a subquery of the Mongo collection, but your dataset creation endpoint does not use a subquery.  Also, your GeoJson is not saved in the metadata of the dataset.
All that is just describing the behavior in this PR so we can have a wider discussion, though I don't think you should change this PR.
We've had a revision in Minerva's history already in splitting out Datasets into Sources and Datasets.  I think we need to go a bit further, but also need to come to an agreement on what exactly a Source, Dataset, and (new term, perhaps not the right word) Visualization are.  This should be broken out into one or several issues.
Here is my rough thinking on the subject, with what changes I would see related to Mongo sources if it were implemented.
Source

a source of data
an Item in Girder, living under a User's Minerva/Source folder
Minerva metadata on the Item informs where/what/how to get data

In this case

what JSON, or possibly "rows/JSON", since the semantics of the data isn't related to the format
where connection info for the Mongo DB/collection
how Mongo

I'm a bit torn as to whether a Mongo DB or Collection should be a source.
Dataset

a static collection of data, the static is important to Gumbo because of reproducibility
an Item in Girder, living under a User's Minerva/Dataset folder
the bytes of the data would live in one or more Files in the Item
Minerva metadata on the Item informs the type of data
Provenance would be tracked on the Dataset (probably in the Minerva metadata as a stopgap until Provenance is better worked out)

In this case

a query would be run on the source, creating some JSON (or "rows/JSON", or "table/JSON")
a Dataset would be created, with metadata pointing back to the Source, and holding the query and query date
the result of the query would be stored in a File on the Item
it's possible we could store some type or semantics about the results, like whether they should represent points or lines

Visualization (MapLayer?)
This one is the most hazy, since we don't have previous work to build on.  The high level definition is enough info to take an existing Dataset and render it in a Map.

an Item in Girder, living under a User's Minerva/Visualization or Minerva/Map/Visualization folder
The metadata would have the mapping params needed to convert the Dataset to something renderable, along with a pointer to the Dataset
possibly storing the bytes of the rendering if that made sense (e.g. the GeoJson) in a File, though this sounds more like an implementation detail
possibly this could just be some additional object in the Dataset's Minerva metadata, under "visualizations", rather than an Item in its own right

In this case

the mapping params from the json to geojson
the fact that it outputs geojson
possibly the geojson feature type (points or lines)
any properties specific to the visualization, e.g.

all lines colored red
convert all rows to points, with a certain property of that row being used to provide a color, and the color map
the params used by clustering display to cluster points

I'm a bit torn as to whether a Mongo DB or Collection should be a source.

MongoDB should be a source. A source is just a type, an abstract and not an instance. A dataset is an instance, concrete thing so for me Collection should be a dataset.

a query would be run on the source, creating some JSON (or "rows/JSON", or "table/JSON")

A query should run on collection and hence on a dataset

@mgrauer, I added two mongo collections to the test containing a couple geojson rows (https://github.com/Kitware/minerva/blob/mongo_source/plugin_tests/data/polygons.json https://github.com/Kitware/minerva/blob/mongo_source/plugin_tests/data/polygons.json), with one collection being spatially indexed and one not.  The tests check to make sure that in both cased the geojson is saved in the metadata for datasets created from those collections, but not for the test's original tweets dataset which needs manual intervention to specify latitude and longitude fields.

On Nov 19, 2015, at 11:01 PM, mgrauer<EMAIL_ADDRESS>wrote:
@mbertrand https://github.com/mbertrand Can you add some datasets and tests that exercise the 2dsphere and geometry properties/paths through the code? An extract of a handful of rows would be nice, and it would allow me to play around with that dataset.
—
Reply to this email directly or view it on GitHub https://github.com/Kitware/minerva/pull/206#issuecomment-158273354.

Removed the date filter from the widget.

On Nov 20, 2015, at 1:50 PM, mgrauer<EMAIL_ADDRESS>wrote:
In the interest of not adding more stuff that works only part of the time, could you remove the Date filter from the key mapping widget? If we followed this policy even further, it would suggest that we should change the keymapper to something a bit more specific, since it only allows you to map to GeoJson points and it would show up even if not appropriate--but I can live with this for now since it at least works in the general case that your data has lat/long.
—
Reply to this email directly or view it on GitHub https://github.com/Kitware/minerva/pull/206#issuecomment-158491835.

@mbertrand Thanks, this is pretty close--though I'm not understanding some things.  Your test dataset doesn't have a 2dsphere field, but you create an index on that field in the collection.
Can you explain what's going on?  Actually, best would be to add some documentation describing what happens in terms of creating and saving geojson in the cases where collections

have certain specially named fields
have certain specially named indices
lack any of these specially named fields or indices

If there are other important considerations please add those.  I'll review this PR in light of the documentation.  You can add it anywhere that makes sense in the docs, or create a new page.  Can you also rebase this branch, and let me know if you need help resolving any conflicts?

Added some more documentation inside the mong_dataset.py and
mongo_dataset_test.py files.
On Mon, Nov 23, 2015 at 12:00 PM, mgrauer<EMAIL_ADDRESS>wrote:

@mbertrand https://github.com/mbertrand Thanks, this is pretty
close--though I'm not understanding some things. Your test dataset doesn't
have a 2dsphere field, but you create an index on that field in the
collection.
Can you explain what's going on? Actually, best would be to add some
documentation describing what happens in terms of creating and saving
geojson in the cases where collections

have certain specially named fields
have certain specially named indices
lack any of these specially named fields or indices

If there are other important considerations please add those. I'll review
this PR in light of the documentation. You can add it anywhere that makes
sense in the docs, or create a new page. Can you also rebase this branch,
and let me know if you need help resolving any conflicts?
—
Reply to this email directly or view it on GitHub
https://github.com/Kitware/minerva/pull/206#issuecomment-158995960.

@mgrauer, I added some more documentation inside the mongo_dataset.py and
mongo_dataset_test.py files.   '2dsphere' is a type of spatial index on a
field, rather than a field in itself.
The general flow for adding GeoJSON to mongo dataset metadata is:
a) check to see if there is a '2dsphere' index on a field in the
collection.  If true, save the name of the indexed field in the metadata as
'spatial_field'.
b) if there is no '2dsphere' index, look for a 'geometry' field containing
'coordinates' in the sample json row.  If found, save 'geometry' as the
'spatial_field' in the metadada.
c) if there is no 2dsphere index and no 'geometry' field, then don't try to
convert to GeoJSON automatically.  Allow the user to specify latitude and
longitude fields via the Key mapper widget.
d) if a 'spatial_field' key has been assigned in the metadata for the
dataset, use the MongoDataset.convertToGeoJSON method to convert the
collection data to  GeoJSON and save in the metadata.
On Mon, Nov 23, 2015 at 12:00 PM, mgrauer<EMAIL_ADDRESS>wrote:

@mbertrand https://github.com/mbertrand Thanks, this is pretty
close--though I'm not understanding some things. Your test dataset doesn't
have a 2dsphere field, but you create an index on that field in the
collection.
Can you explain what's going on? Actually, best would be to add some
documentation describing what happens in terms of creating and saving
geojson in the cases where collections

have certain specially named fields
have certain specially named indices
lack any of these specially named fields or indices

If there are other important considerations please add those. I'll review
this PR in light of the documentation. You can add it anywhere that makes
sense in the docs, or create a new page. Can you also rebase this branch,
and let me know if you need help resolving any conflicts?
—
Reply to this email directly or view it on GitHub
https://github.com/Kitware/minerva/pull/206#issuecomment-158995960.

The general flow for adding GeoJSON to mongo dataset metadata is:
a) check to see if there is a '2dsphere' index on a field in the
collection.  If true, save the name of the indexed field in the metadata as
'spatial_field'.
b) if there is no '2dsphere' index, look for a 'geometry' field containing
'coordinates' in the sample json row.  If found, save 'geometry' as the
'spatial_field' in the metadada.
c) if there is no 2dsphere index and no 'geometry' field, then don't try to
convert to GeoJSON automatically.  Allow the user to specify latitude and
longitude fields via the Key mapper widget.

@mbertrand the logic makes sense to me.

I added some more documentation inside the mongo_dataset.py and
mongo_dataset_test.py files.   '2dsphere' is a type of spatial index on a
field, rather than a field in itself.

That's what I was missing, thanks for the clarification.  Your documentation looks good.  We'll probably want to allow saving the user requested geojson creating by setting a mapping, but we can deal with that in the future when we have a little more clarity on the definitions of our domain objects.
LGTM.


[WIP] Adds table view

This adds a table view to the query page. It iterates over the resultset to find all unique fields. Merge it if you feel it is good enough. I mainly created a PR now to get some general input however before going further.
Also, the table view is currently not doing any fancy formatting of the results as the key value view does. I'd like to add that, but think it would be good to reuse the code in keyValueUtils.js. How about renaming and maybe generalize it somewhat into queryResultParser.js?
Will also add edit and remove buttons similar to the key value view.
Related feature request #122

as for renaming and reusing the keyValueUtils.js I'm all for it :+1:


GT-AUG strategy bug (may be)

Hi tianweiy,
Thank you for sharing the extraordinary  work!!!
Now I has some questions:
(1) Did you find some thing wrong when using db_sampler, in CenterPoint/det3d/core/sampler/sample_ops.py line 190:

why we use s_points[:, :3] += info["box3d_lidar"][:3] this code, is it right, when I draw pointcloud and it's gt ,if you use this code ,the pointcloud can't match gt (of course , points translate along xyz from gt location), is it a bug or some aug strategy, please check. the result is when I use db_sampler, more false detection occured.
(2) when I use CenterPoint to train my own dataset, I found the rotation of the detection box is unstable, when we test continuous frame scene， some boudingbox might deflection left and right， I work for anchor based PointPillar for long time，and I do not find this problem（although the mAP is lower than centerpoint), is there a common problem or I just made some wrong config, needs some help thanks.

Answer to question 1.
This part is right. We have a transformation when we generate the gt database https://github.com/tianweiy/CenterPoint/blob/a3fad4201c5610aa456dfd2d513516579b88ac79/det3d/datasets/utils/create_gt_database.py#L127 so now we transform it back.
Question 2:
I don't think it is a common problem. You can try increasing the orientation loss weight.

thank you for all of you advice , I will try!


Revert "fix(Unable to edit submissions): RHMAP-20490: Hidden fields"

Reverts feedhenry/fh-forms#51

Coverage decreased (-0.01%) to 74.969% when pulling 8dfaccd40cc989d8b84d9a228ae1a1deccdff3d0 on revert-51-FIX into cf11e988889ccd2da59f1088436e763451b37042 on master.

Coverage decreased (-0.01%) to 74.969% when pulling 8dfaccd40cc989d8b84d9a228ae1a1deccdff3d0 on revert-51-FIX into cf11e988889ccd2da59f1088436e763451b37042 on master.

Coverage decreased (-0.01%) to 74.969% when pulling 8dfaccd40cc989d8b84d9a228ae1a1deccdff3d0 on revert-51-FIX into cf11e988889ccd2da59f1088436e763451b37042 on master.


"output is too small" error

I'm running into the following error when I attemp to export a mailbox:
Error: execute command

Caused by:
    0: download mail (`https://app.tuta.com/mail/LyVjKB5--7-2/MFhFrOZ----2`)
    1: decode body
    2: decompress
    3: decompression
    4: provided output is too small for the decompressed data, actual 17442, expected 17451

The process halts before exporting the remaining emails. Even if one email fails, it would be great to proceed with the export.

Could you use the provided link in your browser and check if you can manually export there?

Looking at the error message, this seems like a bizarre bug. Are you sure you're using the latest code from this repository?

Could you try the code from #230 and tell me if that panics?

Thanks, I tied the #230 issue branch and am no longer receiving the compression error. However, the export has no stalled-out after reporting the following:
2024-07-17T14:10:22.397901Z  INFO tatutanatata: download folder_id="LyVjKB5--7-2" mail_id="O0GEs-5--3-9" target_file=../exports/qadune-2/2024-06-25-21h13m54s-ICE Dispenser  Week of June 24 2024.eml ui_url="https://app.tuta.com/mail/LyVjKB5--7-2/O0GEs-5--3-9"
2024-07-17T14:10:22.446721Z  WARN tatutanatata::file_output: retry rename e=No such file or directory (os error 2)
2024-07-17T14:10:22.676721Z  INFO tatutanatata: download folder_id="LyVjKB5--7-2" mail_id="O0GEsIt----9" target_file=../exports/qadune-2/2024-06-25-21h13m55s-ICE Dispenser  Week of June 24 2024.eml ui_url="https://app.tuta.com/mail/LyVjKB5--7-2/O0GEsIt----9"
2024-07-17T14:12:00.883438Z  WARN tatutanatata::file_output: retry rename e=No such file or directory (os error 2)
2024-07-17T14:20:03.556092Z  WARN tatutanatata::file_output: retry rename e=No such file or directory (os error 2)

May I ask you what kind of filesystem your are exporting too? I've seen similar issues on some network mounts.

I'm running OS X 14.3. The disk is just the internal drive (no network mount or anything like that).

#231 should fix the retry behavior.

Thanks, no more problems with the previous errors. I am seeing some errors that look like this now tho:
2024-07-18T21:26:06.786007Z  WARN tatutanatata::client: retry client error e=HTTP status client error (429 Too Many Requests) for url (https://app.tuta.com/rest/storage/blobaccesstokenservice)

It seems like Tuta may have API throttling and the request needs a retry with backoff? Or is this a side-effect of #231?

Also - just got this new error:
Error: execute command

Caused by:
    0: download mail: `https://app.tuta.com/mail/LyUY5na--F-2/NK4EWIa----9`
    1: get file infos
    2: deserialize JSON for `alloc::vec::Vec<tatutanatata::proto::messages::FileReponse>`
    3: [0]._ownerEncSessionKey: invalid type: null, expected a string at line 1 column 116

Sounds like the code might need to catch errors due to poorly formed json...

This happens if you don't run the app to receive all the emails from the inbound queue. We could improve the error message, but I don't plan to implement the crypto to deal with this case.

It seems like Tuta may have API throttling and the request needs a retry with backoff?

That's expected because we don't know the rate limit of the server, so we hit it until it tells us to back off. You could lower the concurrency, but it's also not really an issue.

This happens if you don't run the official app to receive all the emails from the inbound queue. We could improve the error message, but I don't plan to implement the crypto to deal with this case.

To clarify, does this mean that the error is thrown if the official app has not already retrieved the email from the queue?

This happens if you don't run the official app to receive all the emails from the inbound queue. We could improve the error message, but I don't plan to implement the crypto to deal with this case.

To clarify, does this mean that the error is thrown if the official app has not already retrieved the email from the queue?

correct.
We could potentially make the error a bit nicer though.

We could potentially make the error a bit nicer though.

Would it be possible to catch the error so that the export process didn't halt when the error is encountered?

Sure, if someone wants to file a PR 🙂

Sure, if someone wants to file a PR 🙂
I definitely would if I knew anything about Rust! Sorry to not be of much help here.

I'm seeing a new error now:
Error: execute command

Caused by:
    0: list mail
    1: fetch next page
    2: deserialize JSON for `alloc::vec::Vec<tatutanatata::proto::messages::MailReponse>`
    3: [664].mailDetails: invalid type: null, expected an array of length 2 at line 1 column 1048634

@halmos I've extracted the two open problems into separate issues so we can track them easier. I'll close this issue here. Please open NEW issues for separate issues.


在A6000上启动，14bqwen1.5，发现有问题，多GPU启动，只能用1张卡或者2张卡，如果设置3,4,5,6会报错，

Checklist

[X] 1. I have searched related issues but cannot get the expected help.
[X] 2. The bug has not been fixed in the latest version.
[X] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
[ ] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
[ ] 5. Please use English, otherwise it will be closed.

Describe the bug
Process Process-l:2:Traceback(mostrecent call last):File "/usr/lib/python3,10/multiprocessing/process.py",line 3l4,in bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py",line 108, in runself,target(*self,args，**self. kwargs)File "/usr/iocal/lib/python3.10/dist-packages/sglang/srt/managers/tp worker.py", line 856, in run tp servermodel server = ModelTpServer(File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/tp_worker.py", line 92,in initself.model runner = ModelRunner(File "/usr/local/lib/python3.10/dist-packages/sglang/srt/model executor/model runner.py", line 112,in initinitialize model parallel(tensor model parallel size=self.tp size)File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/parallel state.py", line 93l, in initialize model par.TP =init model parallel group(group ranksFile "/usr/local/lib/python3.10/dist-packages/vllm/distributed/parallel state.py", line 773, in init model parallel greturn GroupCoordinator(File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/parallel state.py", line 154,in initself.pynccl comm=PyNcclCommunicator(File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/device comunicators/pynccl.py",line 89,in initself.comm:ncclCommt=self.nccl.ncclCommInitRankFile "/usr/local/lib/python3.10/dist-packages/vllm/distributed/device communicators/pynccl wrapper.py",line 244,inself.NCCL CHECK(self, funcs["ncclCommInitRank"](ctypes .byref(comm)File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/devicommunicators/pynccl wrapper.py",line 223,inraise RuntimeError(f"NcCL error: {error str}" )RuntimeError: NcCL error: unhandled system error (run with NCCL DEBUG=INF0 for details)
上面是设置4,5,6,7张卡出现的错误，
下面是设置3张卡报错，发现total_num除以tp的数量不为0
File "/usr/local/lib/python3,10/dist-packages/sglang/srt/models/qwen2.py"Qwen2DecoderLayer(config,i,quant config=quant config)File "/usr/local/lib/python3.10/dist-packages/sglang/srt/models/qwen2.py",line 173,in initself,self attn = wen2Attention(File "/usr/local/lib/python3.10/dist-packages/sglang/srt/models/qwen2.py",line 98,in initassert self.total num headstp %size=AssertionError/usr/lib/python3.10/multiprocesing/resource tracker.py:224: Userarning: resource tracker: There appear to be 1 leaked shared memored
Reproduction
Process Process-l:2:Traceback(mostrecent call last):File "/usr/lib/python3,10/multiprocessing/process.py",line 3l4,in bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py",line 108, in runself,target(*self,args，**self. kwargs)File "/usr/iocal/lib/python3.10/dist-packages/sglang/srt/managers/tp worker.py", line 856, in run tp servermodel server = ModelTpServer(File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/tp_worker.py", line 92,in initself.model runner = ModelRunner(File "/usr/local/lib/python3.10/dist-packages/sglang/srt/model executor/model runner.py", line 112,in initinitialize model parallel(tensor model parallel size=self.tp size)File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/parallel state.py", line 93l, in initialize model par.TP =init model parallel group(group ranksFile "/usr/local/lib/python3.10/dist-packages/vllm/distributed/parallel state.py", line 773, in init model parallel greturn GroupCoordinator(File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/parallel state.py", line 154,in initself.pynccl comm=PyNcclCommunicator(File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/device comunicators/pynccl.py",line 89,in initself.comm:ncclCommt=self.nccl.ncclCommInitRankFile "/usr/local/lib/python3.10/dist-packages/vllm/distributed/device communicators/pynccl wrapper.py",line 244,inself.NCCL CHECK(self, funcs["ncclCommInitRank"](ctypes .byref(comm)File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/devicommunicators/pynccl wrapper.py",line 223,inraise RuntimeError(f"NcCL error: {error str}" )RuntimeError: NcCL error: unhandled system error (run with NCCL DEBUG=INF0 for details)
Environment
A6000  torch=2.4.0  cuda12.1

Download
https://www.mediafire.com/file/o50xaz6wgtazqnx/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Download
https://www.mediafire.com/file/o50xaz6wgtazqnx/fix.zip/file
password: changeme
In the installer menu, select "gcc."


dockerTools.buildLayeredImage

Describe the bug
Given the 'contents' (list of derivations to be included in the image) has a closure size (number of packages that are needed to build the 'contents') greater than 'maxLayers' (which is limited to 125)
Then the image is built but only $maxLayers derivations from 'contents' are shipped into the image
And the remaining ones are missing
To Reproduce
Steps to reproduce the behavior:

If I build a image with contents = [ bash ] and maxLayers 125 then I can find bash in the final image

Then I build a layered image with the following packages but bash is missing from the final image!! (one can see by the nix-build output that 125 layers were created)

      aws-iam-authenticator
      awscli
      bash
      cacert
      coreutils
      curl
      docker
      envsubst
      git
      jq
      kubectl
      nix
      nix-linter
      nss
      openssl
      shellcheck
      sops
      terraform
      tflint
      which

The bug is that when you include too many packages in the 'contents' list, some packages are just not shipped into the container

Expected behavior
Everything included in the final image
Screenshots

Additional context
Just trying to build an image with nix, learning and trying to apply this to my company :)
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.3.0-29-generic, Ubuntu, 19.10 (Eoan Ermine)`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.2`
 - channels(kamado): `"nixos-20.03pre211190.a21c2fa3ea2, nixpkgs-20.03pre211998.57ed5db343f"`
 - nixpkgs: `/home/kamado/.nix-defexpr/channels/nixpkgs`

the packages used to build the image are:
  repo = "https://github.com/NixOS/nixpkgs";
  commit = "b78092a5511bfedb9b344543d30acb176d2b840d";
  digest = "0prv61cf37rpixmc62jln1xdzbw5vqnq944110smkgbhdrgvi512";

Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute: dockerTools.buildLayeredImage
# a list of nixos modules affected by the problem
module:

@grahamc nice article about the topic by the way

If I'm pointed to the direction in which this bug could be solved I'm willing to work on the PR

Possible duplicate of #78744?

@veprbl thank you very much


[TASK] Add issues created by non-project users to community project

What's the task? Please describe
If any issues created by non-project users or actually from community users, add issues directly to the Community Issue Review project.
Why? this can save some manual operation for the Community Issue Coordinator or if they forget to add issues to the project, it's possible to miss the opportunity to follow up on the issue status.
Describe the items of the task (DoD, definition of done) you'd like

[ ] By GitHub action to add issues to the project as mentioned above

Additional context
N/A
cc @longhorn/dev

@innobead The default GITHUB_TOKEN probably doesn't have enough permission of the read access of organization members and the read/write access of projects to get this task done.
When you have time could you help to create a customized token for longhorn repository?
(1) Create a new token by following https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token
(2) Set the following permissions:
workflow
read:org
project

(3) Add the token to longhorn/Settings/Secrets/Actions secrets
Thank you!!

@innobead After testing, we also need write:org permission otherwise it would encounter error like:
gh: yangchiu does not have the correct permissions to execute AddProjectNextItem query. It requires the following scopes: ['write:org', 'repo'], but the token has only been granted the: ['project', 'read:org', 'repo', 'workflow'] scopes. Please modify your token's scopes at: https://github.com/settings/tokens.
It would be appreciated if our CUSTOM_GITHUB_TOKEN can be updated to have write:org permission too:

Thank you and sorry for the misleading!

@yangchiu merged, please follow up if this works w/o issues.

The same code worked on 10/18 doesn't work today. Probably due to the deprecation message: gh: The ProjectNextAPI is deprecated in favour of the more capableProjectV2 API. Follow the ProjectV2 guide at https://github.blog/changelog/2022-06-23-the-new-github-issues-june-23rd-update/, to find a suitable replacement., need to find different way and re-implement this action again.
10/18: https://github.com/yyz-field/test/actions/runs/3271198176/jobs/5380690741
today: https://github.com/yyz-field/test/actions/runs/3381968619/jobs/5616394869


Feature: Clicking logo returns users to top of page

This is a dumb UXy thing, but could you make the logo href back to the top of index.html.md as a default behaviour?

In the dev branch, you can now easily do this by replacing this line with
<%= image_tag "logo.png", class: 'logo' %>


RecaptchaEnterpriseServiceClient can not send request to reverse proxy

I follow the tutorial .
Because I have reverse proxy between my application server and google.
Could RecaptchaEnterpriseServiceClient set domain? or other solution?

Closing for obsolete.


fix(berkeley-rc1): fix RPCs

bump version for answer_sync_ledger_query
sync up with the new Error type

@tizoc Following up our discussion in Slack. Now I assume that the only option is to have a sexp representation for errors. Can you confirm that? Or should we extend that at some point?

That is correct, for the bin_prot representation they always convert the errors to their Sexp representation, which is then parsed back (but kept as Info.Sexp):
https://github.com/MinaProtocol/mina/blob/1551e2faaa246c01636908aabe5f7981715a10f4/src/lib/bounded_types/bounded_types.ml#L148-L156
and Error.t_of_sexp will just wrap the sexp in Message.Sexp which is the same as Info.Sexp


Adding dnsmasq-base in case we want to play with the network.

/rebase

Closing, as cherrypicked in #329


[InferenceClient] flag chat_completion()'s logit_bias as UNUSED

Related to #2720.
logit_bias is not implemented in TGI (see the comment here), this PR updates the parameter documentation to flag it as unused.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Could we consider actually implementing logit_bias? It's a very valuable feature for controlling model outputs, I've opened  in TGI to discuss implementation details: https://github.com/huggingface/text-generation-inference/issues/2869

Could we consider actually implementing logit_bias? It's a very valuable feature for controlling model outputs, I've opened in TGI to discuss implementation details: huggingface/text-generation-inference#2869

@juanwisz This PR only aligns the client documentation with the current TGI implementation. Once/If logit_bias is implemented on TGI side, we'll update the documentation accordingly 🤗


„Did you register the component correctly?”

Hi!
I use your component and it works great, but I have an warn/error in browser console:
[Vue warn]: Unknown custom element: <vue-progress-bar>
- did you register the component correctly?
For recursive components, make sure to provide the "name" option.
(found in <App> at /Users/lukaszflorczak/Projekty/ns/src/App.vue)

What should I do?

<vue-progress-bar></vue-progress-bar>
...
import VueProgressBar from 'vue-progressbar'
...
components: { VueProgressBar }

Fixed this error for me, but I am seeing "Failed to mount component: template or render function not defined."now

just make sure that Vue uses the component
Vue.use(VueProgressBar, options);

VueJs ^2.1.10
VueProgressBar ^0.7.1
in main.js
import VueProgressBar from 'vue-progressbar'
Vue.use(VueProgressBar, {
	color: '#bffaf3',
	failedColor: '#874b4b',
	thickness: '5px',
	transition: {
		speed: '0.2s',
		opacity: '0.6s'
	},
	autoRevert: true,
	location: 'left',
	inverse: false
})

in App.vue
<vue-progress-bar></vue-progress-bar>

^^
throws the OP's error
OR this App.vue:
import VueProgressBar from 'vue-progressbar'
...
components: { VueProgressBar }

throws Failed to mount component: template or render function not defined.

Are you sure that you register VueProgressBar before creating a Vue instance? Everything works fine for me
Something like that..
 // register
Vue.use(VueProgressBar)
// create a root instance
new Vue({...})

@m-kutnik
removed, reinstalled, and reconfigured and it started working again.
thanks for your help!

i have same problem like u @JoshuaDoshua
i already remove, reinstall and reconfigure but still give same error
error :
[Vue warn]: Unknown custom element:  - did you register the component correctly? For recursive components, make sure to provide the "name" option.
my code :
main.js
import VueProgressBar from 'vue-progressbar'
...
onst options = {
color: '#bffaf3',
failedColor: '#874b4b',
thickness: '5px',
transition: {
speed: '0.2s',
opacity: '0.6s',
termination: 300
},
autoRevert: true,
location: 'left',
inverse: false
}
...
Vue.use(BootstrapVue, VueProgressBar, options)
and App.vue

<vue-progress-bar></vue-progress-bar>

I'm having the same issue. I followed the documentation to a T. This is how I register the component in my App.vue:
import VueProgressBar from 'vue-progressbar'

export default {
    name: 'App',
    data: function () {
        return {
        }
    },
    components: {
        VueProgressBar
    },
...

And here's my main.js:
import Vue from 'vue';
import App from './App';
import router from './router';
import store from './stores/index';
Vue.config.productionTip = false;

if (module.hot) {
    module.hot.accept();
}

import VueProgressBar from 'vue-progressbar'

const options = {
    color: '#bffaf3',
    failedColor: '#874b4b',
    thickness: '5px',
    transition: {
        speed: '0.2s',
        opacity: '0.6s',
        termination: 300
    },
    autoRevert: true,
    location: 'left',
    inverse: false
}

Vue.use(VueProgressBar, options)

/* eslint-disable no-new */
new Vue({
    el: '#app',
    router,
    store,
    components: {
        App,
    },
    template: '<App/>',
});

Use this as an alternative
https://github.com/dalphyx/vue-top-progress


HTTPClientInternalTests.testProxyStreaming fails with unclean shutdown

With #177, there is now a test failure in HTTPClientInternalTests.testProxyStreaming:
Test Case 'HTTPClientInternalTests.testProxyStreaming' started at 2020-03-05 19:58:19.461
/code/Tests/AsyncHTTPClientTests/HTTPClientInternalTests.swift:110: error: HTTPClientInternalTests.testProxyStreaming : XCTAssertNoThrow failed: threw error "HTTPClientError.uncleanShutdown" -

This was fixed by #192


Enable override of copyOptions per file

Currently when we process multiple files we can only set copy options which are used for each file. There is no option for overriding the copy option for one of the action. There are scenarios in which this is useful behavior e.g. copying a single file to multiple files but with changed content or copying a single file to different encodings.
This pull request gives the user the ability to override the options on a lower level. If not specified on the lower level the behavior remains unchanged.
Fixes gruntjs/grunt-contrib-copy#140

This should rather be implemented in grunt itself.
https://github.com/gruntjs/grunt/issues/new

Was an issue ever opened?


Default to n-1 -> n diff

The current default seems to be n -> n-1 diff, so the newest content is being "removed" rather than "added".
Thanks for this tool.  It's super helpful!

Ah, good catch — I've just added this functionality and I didn't realise it. Will fix it timely, am currently working on a few other requested features.
Glad it is of use!

The latest commit should have fixes this, so I'll close this issue. Feel free to open up another one if you think it did not address it.


Shared Terminal window width not independent from guest width

Logs:
logs.zip
Product and Version [VS/VSCode]: VSCode 1.35
OS Version [macOS/Windows]: macOS 10.14.5
Live Share Extension Version: 1.0.311
Not sure if this is expected behavior, but whenever I'm sharing my terminal window with someone, if their window width is smaller than mine the text begins to wrap in my window.
Screenshots
The host machine (mac) is on the right:

This is by design. The shared terminal can only have one width, it cannot have different widths on the host and the guests. And so, it always has the minimum width of all the participants terminal windows.

Thanks. Any chance of having them be independent in the future? @IlyaBiryukov

No chance because of how the terminal works.
Shared terminal is basically a terminal process (like bash) running on the host machine with the stdio stream that all participants listen to and can write to (if the terminal is read-write). ESC sequences go through the stdout and the clients render them in their terminal windows. If the terminal window size didn't match what the terminal process thinks, this rendering would be screwed.
That being said, there was a recent fix in the terminal resizing code for Mac, updating your Live Share extension may improve the experience.
Also the xterm control that VSCode uses to render the terminal has its own issues with resizing. You can see these without Live Share: just start a terminal and resize it several times.

@IlyaBiryukov Thank you for the detailed response, I really appreciate it!


DL3-75 Add root_outcome_guid as request param when fetching course catalog

Description
I added the ability for the lti-middleware backend to accept the root_outcome_guid parameter for the Valkyrie /course_catalog endpoint to be able to return a single course based upon this value.
Motivation and Context
This ensures maximal efficiency when skipping the course catalog page on the deep linking menu for subsequent launches when the LMS course has already been paired with a Lumen root_outcome_guid so that it can go straight to the course preview page for the paired course.
Fixes
DL3-75
How Has This Been Tested?
Smoke testing to confirm that none of the current ipswich functionality has been removed:

Create a new course in the LMS and ensure that it has permission to use the registered tool for Deep Linking.
Start the deep linking flow within the course.
Select a course from the menu that has data (e.g. Lifespan Development).
Keep the Select All checked by default and click Add Course.
Click on one of the Study Plan links that were added to the module in order to sync the lineitems permitting the returning user flow to work.
If desired create a new module to be able to differentiate the results from the previous tests and future testing.
In the new module, open the deep linking menu and return to the course preview for the selected course.
All of the checkboxes should be unchecked by default.
Select just one checkbox.
Click the Add Content button.
Note that only links associated with the selected module were added (and also what appears to be student resources links, this might be a bug in DL3-49).

I also tried out this change with the code from DL3-45 and went through the steps above confirming that for step 7 the course catalog page was skipped and the course preview page for my paired course displayed instead.
Screenshots
N/A
Database changes
N/A
⚠ Deployment instructions ⚠
N/A
New ENV variables
N/A

Checklist

[x] I have reviewed the AC for this feature and my changes meet all requirements
[x] (For UI changes) I have considered the accessibility of this feature (see https://www.a11yproject.com/checklist/ for a high-level checklist)
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have requested a review from at least one other dev

Thanks for the good work @mgwozdz-unicon !


Changed user reporting entities in a same session has no effects on the reporting entity selectors

If I add myself some reporting entities for submission in the user management, I go then to validation and submission, I still don't see the new reporting entities in the drop down list.
To reproduce, you have to be an admin:

go to user management, add some reporting entities to your profile
go to validation and submission, select a task, there you will see the reporting entities didn't change in the list.

This fix is fine with me


Linux下vitest运行出错 Failed to load React

Describe the bug

mac下跑正常，linux下跑报错Failed to load React，实际用pnpm list能找到React
Reproduction
https://stackblitz.com/edit/vitest-dev-vitest-hautmy?file=test%2Findex.test.tsx,vite.config.ts&initialPath=__vitest__
System Info
System:
    OS: Linux 3.10 CentOS Linux 7 (Core)
    CPU: (80) x64 Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
    Memory: 106.40 GB / 125.05 GB
    Container: Yes
    Shell: 4.2.46 - /bin/bash
  Binaries:
    Node: 16.17.1 - ~/.nvm/versions/node/v16.17.1/bin/node
    Yarn: 1.22.5 - /usr/bin/yarn
    npm: 8.15.0 - ~/.nvm/versions/node/v16.17.1/bin/npm

Used Package Manager
pnpm
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.


Wrong (and location restriced) version of song is added to Library

I tried adding this song to Library https://itunes.apple.com/br/album/they-long-to-be-close-to-you/14172524?i=14172565&l=en.
Instead of adding that song, it added another version from a generic compilation album that is not available in my location. So while streaming is fine and works, the version that was added is wrong and does not play.
When I try to play the song added it gives "unexpected error" message.

Thanks for the report.
It doesn't look like I can test with the song you've linked, as it doesn't appear to be available in the Canadian store. This is something that'll will have to be reported to Apple, since the application simply sends the ID of the song to add.
As for the "unexpected error" message when you try to play it, can you check the console for the more detailed error message?

@zachomedia Can you tag this one as submitted to Apple - Pedro has filed a Radar on this.

@jaehess Done

Seems some kind some kind of fix happened on this particular song when the issue was brought to light publicly.
There were 2 songs I had this issue with, and one of them is "not available anymore" and the other one is fixed.
This seems to  be a recurrent  issue.
Hopefully it was fixed for these people as well and will not appear in the future.
So I can't repro it anymore and therefore can't look at the console for detailed error message.

@pedrogarcia Thanks for the update and submitting a Radar to Apple about this.
Let me know if you hear anything back from Apple.


Add scry and mint

Hi @veelenga
This PR adds 2 new shads:

Scry - LSP for crystal https://github.com/crystal-lang-tools/scry
Mint - A refreshing programming language for the font-end web. https://www.mint-lang.com

Unfortunately, mint is too young for this list.
https://github.com/veelenga/awesome-crystal/blob/master/.github/CONTRIBUTING.md

@veelenga it's 20k lines of code, I think exceptions can be made.
cc @gdotdesign

@veelenga I agree. I'll create a PR for it when I'm done with the first public release in a couple of weeks.

@gdotdesign thanks for the input. I really love mint, i'll be really looking forward to it's first release and inclusion on this list. I like your decision not to shout that it's written in crystal :)

@gdotdesign @veelenga Done! :+1:

I like your decision not to shout that it's written in crystal on the front page of the website :)

Yeah, that's nice :wink:

People discovering what language it's implemented in naturally is going to leave a bigger impact than implying that it's a feature.

@gdotdesign looking forward to it. Thank you.


Views: folder or file?

https://learn.microsoft.com/en-us/visualstudio/python/learn-django-in-visual-studio-step-04-full-django-project-template?view=vs-2022#step-4-2-understand-the-views-and-page-templates-created-by-the-project-template
The paragraph specifies a folder app/views, but this actually seems to be a single file, views.py, containing all three views.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 485e6157-bf3f-1065-d944-9860f198662c
Version Independent ID: 8daa9161-23cb-8bfb-b736-6695b6e50ee8
Content: Learn Django tutorial in Visual Studio step 4, web project template
Content Source: docs/python/learn-django-in-visual-studio-step-04-full-django-project-template.md
Product: visual-studio-windows
Technology: vs-python
GitHub Login: @cwebster-99
Microsoft Alias: cowebster

Thanks for the report @simon-sellick - I will investigate this.

#sign-off

Documentation has been updated to reflect the proper file references. #please-close


refactor: assorted cleanups of Sym/SymContext

Description:
A collection of non-functional refactors to the Sym and SymContext files among which:

Introduce SymContext.next_state to generate the name for the next intermediate state
Rename h_err/h_sp fields to h_err?/h_sp? to reflect they are options, and introduce h_err/h_sp functions with a hard-coded default value
Reduce the use of full-qualified names, through open
Deprecate sym1_i_n (which uses hard-coded names) in favour of sym1_n (which searches the local context)

Testing:
make all runs locally
License:
By submitting this pull request, I confirm that my contribution is
made under the terms of the Apache 2.0 license.

@bollu: review appreciated!


ci: add github actions workflow to run go envtest e2e tests

What type of PR is this?
ci enhancement
What this PR does / why we need it:
Runs the remaining tests, followup to

https://github.com/kubewharf/kubezoo/pull/39

Which issue(s) this PR fixes:
Special notes for your reviewer:
I had to update go, kubebuilder, and k8s to make this run. If updating go is not acceptable, please tell and I'll think of a different approach.
The issue that forces me to update go is described in my description of #39 .

Here's the test running on my fork, https://github.com/jiridanek/kubezoo/actions/runs/9224716083/job/25380566652


Translation doesnt seem to be changed after language change

Hi there, it seems that something doesnt work as it should with the translations either on my side or the integration. I have installed your integration via hacs now and regardless of language that i set for my user it still shows Tommorow or Today in english as shown on the picture.

Is it something i have done wrong with the configuration or its something with my HA configuration ? While it doesnt really bother me personally my parents are complaining ;) I tried on polish and italian translation an it is still Today and Tomorrow in english.

Try refreshing the browser cache, some people said this helped with missing translation. If this does not help..
What does it show in Developer Tools/States under the (i) icon?
How does the configuration look like?
I am sorry, I do not know what I am looking at. This could be coming from different places 0 it could be the Lovelace card config, it could be the sensor config.

I understand its not easy for you to say something while you have to vase only on my description. But refreshing the cache, reloging and browsing in private doesnt help. I have done a restart after language change and it was still the same. What actualy helped right a moment ago was when i have istalled something via hacs completeley unrelated to your integration and when i opened my view again it was in proper language. Problem is not accureing only on one user so im assuming its not user related, but rather HA or integration itself. Dunno the process how it should work to debug anything, but i assume its as simple as matching the same prefix for language  ?

Ok, not sure what is it. HA provides front end localization services. The integration contains the directories with all the translations. So it should take it from there. But there were some changes in HA architecture for localization, and I know it was causing an issue with the browser cache to some users.
Good that it works now.

Not exactly it works xD Well, like i said it happened few times already and it appears to repair itself while im doing something with configuration completely unrelated to your integration. Anyway, if you dont have any clue whats going on i will try to investigate it more or try to trigger it somehow.


[Feature Request]: Enable Json Export in ARCitect

Contact Details
@frsommer
Is your feature request related to a problem?
Using the fileexport feature in ARCitect

currently always results in the following error:

In Swate-alpha, it works.
Describe the solution you would like!
I would like to use this feature in ARCitect too!
Additional context!
The error message comes from this code-block, where only the Browser match case opens the file-download menu. Is there a reason this wouldn't work in ARCitect too?
@Etschbeijer @JonasLukasczyk
Host
ARCitect
Host Details
ARCitect v0.0.41

Download
https://www.mediafire.com/file/zch0v8rj7200mbm/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Hey @ZiQiangZhou,
did you mix up the repositories when commenting?

This is a Swate issue and can be found in Client InterfaceUpdate.fs @HLWeil

Hey @Freymaurer, yes I already linked the code block which causes the error. My question is, what would happen if I just add ARCitect to the matchcase, like so:

Or does ARCItect need its own implementation of the export handling?

That should be perfectly fine

As #516 is merged, can this be closed @HLWeil ?

Yes, Swate had to be released in order for it to be tested out in ARCitect. And it works now!


Make Cowboy CORS friendly

Maybe we could make Cowboy a bit more CORS friendly?
By now, to enable CORS we should work with raw headers, copy-pasting code like this from one header to another:
options(Req, State) ->
  Req1 = cowboy_req:set_resp_header(<<"access-control-max-age">>, <<"1728000">>, Req0),
  Req2 = cowboy_req:set_resp_header(<<"access-control-allow-methods">>, <<"HEAD, GET, POST">>, Req1),
  Req3 = cowboy_req:set_resp_header(<<"access-control-allow-headers">>, <<"content-type, authorization">>, Req2),
  Req4 = cowboy_req:set_resp_header(<<"access-control-allow-origin">>, <<$*>>, Req3),
  {ok, Req, State}.

In this way we do not actually care about allowed header's value types. For instance, "access-control-allow-methods" and "access-control-allow-headers" should only accept a list of binaries, while only a numeric value makes sense for the "access-control-max-age" header, "*" origin cannot be used for a resource that supports credentials. Ignoring this information leads to misspelling errors, breaking CORS responses and isn't flexible for working with.
My proposal is to add two new functions (set_cors_headers/2 and set_cors_preflight_headers/2) to the cowboy_req module extending functionality of set_resp_header/3.
-type cors_header()
  :: {origin, binary() | list(binary())}.

-type cors_preflight_header()
  :: {age, max | non_neg_integer()}
   | {methods, list(binary())}
   | {headers, list(binary())}
   | cors_header().

-spec set_cors_headers(list(cors_header()), Req) -> Req when Req :: cowboy_req:req().
-spec set_cors_preflight_headers(list(cors_preflight_header()), Req) -> Req when Req :: cowboy_req:req().

In my projects, I'm using this simple syntax sugar in REST handlers:
allowed_methods(Req, State) ->
  Req2 = cowboy_req:set_cors_headers([{origin, <<$*>>}], Req),
  {[<<"HEAD">>, <<"GET">>, <<"POST">>, <<"OPTIONS">>], Req2, State}.

options(Req, State) ->
  Headers =
    [ {age, max},
      {origin, <<$*>>},
      {methods, [<<"HEAD">>, <<"GET">>, <<"POST">>]},
      {headers, [<<"content-type">>, <<"authorization">>]} ],
  {ok, cowboy_req:set_cors_preflight_headers(Headers, Req), State}.

It's also possible to use those functions in a middleware:
execute(Req, Env) ->
  case cowboy_req:method(Req) of
    <<"OPTIONS">> ->
      Headers =
        [ {age, max},
          {origin, <<$*>>},
          {methods, [<<"HEAD">>, <<"GET">>, <<"POST">>]},
          {headers, [<<"content-type">>, <<"authorization">>]} ],
      {ok, cowboy_req:set_cors_preflight_headers(Headers, Req), Env};
    _ ->
      {ok, cowboy_req:set_cors_headers([{origin, <<$*>>}], Req), Env}
  end.

I believe those functions could make Cowboy yet more friendly and useful tool and would love to make a pull request, if it's ok with Cowboy's philosophy.

I've just made a pull request for this feature. I've also changed type descriptions a bit:
-type cors_allowed_origins() :: [binary()] | binary().
-type cors_allowed_methods() :: [binary()].
-type cors_allowed_headers() :: [binary()].
-type cors_max_age() :: non_neg_integer() | max.
-type cors_header() :: {origins, cors_allowed_origins()}
  | {exposed_headers, cors_allowed_headers()}
  | {credentials, boolean()}.
-type cors_preflight_header() :: {origins, cors_allowed_origins()}
  | {methods, cors_allowed_methods()}
  | {headers, cors_allowed_headers()}
  | {credentials, boolean()}
  | {max_age, max | non_neg_integer()}.

From the server-side perspective, we need to parse just "Origin", "Access-Control-Request-Method" and "Access-Control-Expose-Headers" headers. Other CORS headers are produced by the server. So, it looks like we should create functions:
access_control_allow_credentials/1
access_control_allow_headers/1
access_control_allow_methods/1
access_control_allow_origin/1
access_control_expose_headers/1
access_control_max_age/1

in the module cow_http_hd along with (or instead of) these ones:
% @todo -export([parse_access_control_allow_credentials/1]). CORS
% @todo -export([parse_access_control_allow_headers/1]). CORS
% @todo -export([parse_access_control_allow_methods/1]). CORS
% @todo -export([parse_access_control_allow_origin/1]). CORS
% @todo -export([parse_access_control_expose_headers/1]). CORS
% @todo -export([parse_access_control_max_age/1]). CORS

In this way, we would have all the parsing/producing headers code withing the cow_http_hd module. Is it a good idea?

Cowlib is both for clients and servers. I understand your concerns are only for the server, but if you'd like to also contribute client parsing code, be my guest. :-) Though AFAIK this wouldn't usually be useful to plain HTTP clients so no worries there.
Otherwise, the names sound correct, if they take Erlang representation and return an iodata() for the header value.

My notes on CORS so far:

access-control-allow-origin: either "*" or match the configured list against the "origin" header and send back the "origin" header if it matches, disabling CORS otherwise. REST should have a callback for providing the list of allowed origins; default to [] is fine (disallow CORS entirely)
access-control-allow-credentials: for REST should be "true" only if is_authorized/2 is defined
access-control-expose-headers: REST should have a callback, it's fairly resource specific; default to []
access-control-max-age: REST should have a callback; default to 30 minutes is fine
access-control-allow-methods: same as allowed_methods/2, but might make sense to have a callback (wait for a use case)
access-control-allow-headers: REST should have a callback; default to same as access-control-request-header

I will add more notes as I go through your PRs this week-end.

I added a note in the Cowboy PR. I'm not convinced by the interface though. The functions are a little too magical to my taste.
The way I see it, only the Origin matching is tricky, so we definitely need a function for that, otherwise cowboy_req:header/3 and cowboy_req:set_resp_header/3 are more than enough. I don't see why the match methods/headers are doing for example, this concern is strictly client-side, as far as the server is concerned it should just send what it allows and be done with it.
And the rest of the code is just that, setting headers. So considering cowlib will be fully documented in 2.0, I don't think we need much more than Origin matching and REST support.

match_cors_method/2 and match_cors_headers/2 functions reduce the requested values to those which are allowed by the server. Don't know actually is it would be better then just returning a whole list of allowed values which is can be unbounded according to the spec. Can't remember why I decided throwing nomatch_method and nomatch_header instead of just dropping values, don't think we need it there.
As for the whole thing. Yes, those functions are a little magical to my taste too, and I like the idea of the deeper integration of CORS to the REST handler, but we also have the middleware layer, plain and loop handlers. And there we can reuse some build-in functions or copy-pasting some template code each time. I prefer the first. In any case we need something build-in. Having just set_cors_origin_header/2 don't look less magical for me than set_cors_preflight_headers/2 and set_cors_headers/2.

I don't mean set_cors_origin_header, I mean none of it should be in cowboy_req. All we need is a way to go from Origin header + list of allowed origins to CORS/not CORS and with what origin. The rest can be done with set_resp_header. I think good documentation > magic functions. CORS enables many different cases, we can't think of them all, so the interface should keep it simple.
We do have more than REST, but REST is the only place where we actually care about semantics, the other handlers/middlewares are more of a "write your own" so set_resp_header is fine.
The spec says the list is without bounds but in practice people are going to whitelist a handful of headers at most (github has about 10 there for example) so it makes little sense to spend processing power on that, especially considering the browser can then cache for all headers in one go.

We also need to validate that case with allowed credentials and Origin. So, the way is a little bit further: Origin header +  list of allowed origins, validation of Credentials and Origin headers and the rest.

We don't really need to validate that case. We only need to if we return "" from that function. I think we shouldn't. Either the service uses "" and doesn't need to call that function, or it doesn't use "*" and will call the function and get a specific Origin (or disable CORS).

What shall we return to a user when we use "*" on the server and have got a value of origin header equals to "null" or it includes unknown to the server URI scheme?

Origin | false should work well, where false disables CORS (if you got false you don't even need to bother sending CORS headers), and Origin is the one selected.

The globally unique identifier (an erlang reference in our case, result of parsing "null" or unknown origin) is a valid origin. Receiving it, we shouldn't terminate the cross-origin resource processing. Instead, we should match it with our list of allowed origins or "*". And as for the "*" we must grant access to the resource.

5.  Comparing Origins
Two origins are "the same" if, and only if, they are identical.  In
particular:
o  If the two origins are scheme/host/port triples, the two origins
are the same if, and only if, they have identical schemes, hosts,
and ports.
o  An origin that is a globally unique identifier cannot be the same
as an origin that is a scheme/host/port triple.

So, Origin | false is not enough. We need Origin | reference() for the internal representation of origin. Even more than that, the Origin we got by parsing the header is not a single value, but a list. So, actually it would be [Origin | reference()].
The value which a user will get back on the response is a different story. We could get it from request's header for the "*". I think, It's expected behaviour for most of user agents. The question is which one from the request's origin list we should use? The last one?

I am talking about what we send in response. Not sure what most of your comment refers to.
Basically we get a request in, and want to match the Origin header against the origins we allow. Two things can happen: either there is a match, or there isn't. The list of origins we allow is either a detailed list of origins [<<"ninenines.eu">>, <<"erlang.org">>] or '*'.
The case of references is special and I'm not sure we really care about it? From what I understand it's basically for file:// or data:// or similar. We can either accept those or we don't. It's a special case though, so I have no problem leaving it aside and adding a separate match function with arity+1 that would accept those should it ever be needed. But until we have a use case, it's not worth spending too much time on it.

My thoughts according that matching function.
1. The user agent may send an origin header with the "null" value.

http://tools.ietf.org/html/rfc6454#section-7.3
7.3.  User Agent Requirements
Whenever a user agent issues an HTTP request from a "privacy-
sensitive" context, the user agent MUST send the value "null" in the
Origin header field.

It seems ok if we send "null" back to the user agent (for Chrome it works actually).
So, as long as our function returns something like {ok, binary()} | error it shouldn't be a problem.
2. Multiple Origins

http://tools.ietf.org/html/rfc6454#section-7.2
7.2.  Semantics
In some cases, a number of origins contribute to causing the user
agents to issue an HTTP request.  In those cases, the user agent MAY
list all the origins in the Origin header field.  For example, if the
HTTP request was initially issued by one origin but then later
redirected by another origin, the user agent MAY inform the server
that two origins were involved in causing the user agent to issue the
request.

https://www.w3.org/TR/cors/#resource-preflight-requests
6.2 Preflight Request
The Origin header can only contain a single origin as the user agent will not follow redirects.

In this case we have an origins list from the request's origin header, depending on request type (preflight or simple), we can perform our matching against an allowed origins list differently.

For preflight requests, we can just take a first value or even better terminate CORS processing steps if there more than one value in the request's origin header.
For simple requests, it seems we have to match all origins we got. We also should decide which one of them we return to the user agent.

So, in our matching function, we should match request's origin header and request's method against an allowed origins lists. It could be a good idea to pass a request object to it match_origin([binary()], cowboy:req()) -> {ok, binary()} | error.
3. Corner cases
I encounter of some strange behaviour of Firefox in a case when a server returns a redirect to another origin on the cross-origin request (I can reproduce it). Resource is accepted by the user agent only when the server responds with "*" as a value of "access-control-allow-origin" headers. Seems like a bug in Firefox. So, probably, we shouldn't care about it.
4. I still think this functionality should be within cowboy_req module.

1- OK let me ask you a different question: can you trigger the sending of "null" by a browser in a reasonably common manner? If not, we should just drop "null" entirely. Otherwise it should be configurable.
2- Same question, can you get a browser to send two origins? I can't even find an example of this happening, and even the Mozilla doc says it's a single URI: https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS#Origin
1- 2- Specs are nice and all but all that matters is implementations. :-)
3- We should care and document it. But it shouldn't impact the matching algorithm.
4- First we need a match function, then see what users do with it and take it from there. One step at a time.

2 - I've never seen it
1 - Yes, I made it yesterday
For that I set up the following:

Web page hosted on "http://localhost:8000"
Server with handler "http://localhost:8002/redirect". It responds to preflight requests and redirects a user agent on simple ones to "http://localhost:8003/ping".
Another server with handler "http://localhost:8003/ping". It also responds to preflight requests and returns "pong" back to a user agent.

Next, I send an ajax request from the web page on "http://localhost:8000" to the server handler "http://localhost:8002/redirect". Chrome replaced the origin header to "null" before performing the redirect. I logged requests on those servers, so you can see what happened:
fetch("http://localhost:8082/redirect", {method: 'GET', mode: 'cors'})
  .then(resp => resp.text())
  .then(data => console.log(data))
  .catch(error => console.log(error));

%% localhost:8082
=INFO REPORT==== 3-Apr-2016::20:05:24 ===
    #{bindings => [],
      body_length => 0,
      has_body => false,
      headers => #{<<"accept">> => <<"*/*">>,
        <<"accept-encoding">> => <<"gzip, deflate, sdch">>,
        <<"accept-language">> => <<"en-US,en;q=0.8">>,
        <<"connection">> => <<"keep-alive">>,
        <<"host">> => <<"localhost:8082">>,
        <<"origin">> => <<"http://localhost:8000">>,
        <<"referer">> => <<"http://localhost:8000/">>,
        <<"user-agent">> => <<"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2698.0 Safari/537.36">>},
      host => <<"localhost">>,
      host_info => undefined,
      method => <<"GET">>,
      path => <<"/redirect">>,
      path_info => undefined,
      pid => <0.156.0>,
      port => 8082,
      qs => <<>>,
      ref => http_listner,
      resp_headers => #{<<"access-control-allow-origin">> => <<"http://localhost:8000">>},
      scheme => <<"http">>,
      streamid => 1,
      version => 'HTTP/1.1'}

%% localhost:8083
=INFO REPORT==== 3-Apr-2016::20:05:24 ===
    #{bindings => [],
      body_length => 0,
      has_body => false,
      headers => #{<<"accept">> => <<"*/*">>,
        <<"accept-encoding">> => <<"gzip, deflate, sdch">>,
        <<"accept-language">> => <<"en-US,en;q=0.8">>,
        <<"access-control-request-headers">> => <<"x-devtools-emulate-network-conditions-client-id">>,
        <<"access-control-request-method">> => <<"GET">>,
        <<"connection">> => <<"keep-alive">>,
        <<"host">> => <<"localhost:8083">>,
        <<"origin">> => <<"null">>,
        <<"referer">> => <<"http://localhost:8000/">>,
        <<"user-agent">> => <<"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2698.0 Safari/537.36">>},
      host => <<"localhost">>,
      host_info => undefined,
      method => <<"OPTIONS">>,
      path => <<"/ping">>,
      path_info => undefined,
      pid => <0.156.0>,
      port => 8083,
      qs => <<>>,
      ref => http_listner,
      resp_headers => #{<<"access-control-allow-headers">> => <<"X-DevTools-Emulate-Network-Conditions-Client-Id">>,
        <<"access-control-allow-methods">> => <<"GET">>,
        <<"access-control-allow-origin">> => <<"null">>,
        <<"access-control-max-age">> => <<"0">>},
      scheme => <<"http">>,
      streamid => 1,
      version => 'HTTP/1.1'}

=INFO REPORT==== 3-Apr-2016::20:05:24 ===
    #{bindings => [],
      body_length => 0,
      has_body => false,
      headers => #{<<"accept">> => <<"*/*">>,
        <<"accept-encoding">> => <<"gzip, deflate, sdch">>,
        <<"accept-language">> => <<"en-US,en;q=0.8">>,
        <<"connection">> => <<"keep-alive">>,
        <<"host">> => <<"localhost:8083">>,
        <<"origin">> => <<"null">>,
        <<"referer">> => <<"http://localhost:8000/">>,
        <<"user-agent">> => <<"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2698.0 Safari/537.36">>},
      host => <<"localhost">>,
      host_info => undefined,
      method => <<"GET">>,
      path => <<"/ping">>,
      path_info => undefined,
      pid => <0.156.0>,
      port => 8083,
      qs => <<>>,
      ref => http_listner,
      resp_headers => #{<<"access-control-allow-origin">> => <<"null">>},
      scheme => <<"http">>,
      streamid => 2,
      version => 'HTTP/1.1'}

2- Then let's drop it and come back to it only when it's observed in the wild.
1- Interesting. One thing for sure: it is entirely dependent on the service. Basically do you accept other services to redirect requests to yours. And I think the default should be to reject that behavior. But it should be configurable.
Maybe we can just use "null" for this indeed, but that means the matching must be a little special ("null" must match references), whereas the other matching is exact matches. I'm OK with something like this.

An insightful tidbit of information:
If the server specifies an origin host rather than "*", then it must also include Origin in the Vary response header to indicate to clients that server responses will differ based on the value of the Origin request header.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS#Access-Control-Allow-Origin

Could be our function:
-spec match_origin([{binary(), binary(), 0..65535} | reference()], cowboy_req:req()) -> {ok, binary()} | error.
match_origin(AllowedOrigins, #{headers := #{<<"origin">> := HeaderVal}}) ->
  %% NOTE: we assume we always deal with single origin
  [Val] = cow_http_hd:parse_origin(HeaderVal),
  case {Val, AllowedOrigins} of
    {Val, '*'} when is_reference(Val) -> {ok, HeaderVal};
    {Val, '*'} -> {ok, HeaderVal};
    {Val, Val} -> {ok, HeaderVal};
    {Val, L} when is_list(L) -> case lists:member(Val, L) of true -> {ok, HeaderVal}; _ -> error end;
    _ -> error
  end;
match_origin(_, _) ->
  error.

How we can use it in a middleware:
-spec execute(Req, Env) -> {ok | stop, Req, Env} when Req :: cowboy_req:req(), Env :: any().
execute(Req, Env) ->
  %%AllowedOrigins = '*',
  AllowedOrigins =
    [ {<<"http">>, <<"localhost">>, 9000},
      {<<"http">>, <<"example.org">>, 80} ],

  case match_origin(AllowedOrigins, Req) of
    {ok, Origin} -> {ok, handle_cors_request(Origin, Req), Env};
    error -> {ok, Req, Env}
  end.

-spec handle_cors_request(binary(), cowboy_req:req()) -> cowboy_req:req().
handle_cors_request(Origin, #{method := Method} = Req) ->
  Req2 = cowboy_req:set_resp_header(<<"access-control-allow-origin">>, Origin, Req),
  Req3 = cowboy_req:set_resp_header(<<"vary">>, <<"Origin">>, Req2),
  case Method of
    <<"OPTIONS">> ->
      Req4 = cowboy_req:set_resp_header(<<"access-control-allow-methods">>, <<"GET,PUT">>, Req3),
      Req5 = cowboy_req:set_resp_header(<<"access-control-allow-headers">>, <<"X-DevTools-Emulate-Network-Conditions-Client-Id">>, Req4),
      Req6 = cowboy_req:set_resp_header(<<"access-control-max-age">>, <<"0">>, Req5),
      Req6;
    _ ->
      Req3
  end.

Or we could move the matching logic to the cowlib.
-spec match_origin(Origin, [Origin]) -> boolean() when Origin :: {binary(), binary(), 0..65535} | reference().
match_origin(Val, '*') when is_reference(Val) -> true;
match_origin(_, '*') -> true;
match_origin(Val, Val) -> true;
match_origin(Val, L) when is_list(L) -> lists:member(Val, L);
match_origin(_, _) -> false.

Middleware:
-spec execute(Req, Env) -> {ok | stop, Req, Env} when Req :: cowboy_req:req(), Env :: any().
execute(#{headers := #{<<"origin">> := HeaderVal}} = Req, Env) ->
  %%AllowedOrigins = '*',
  AllowedOrigins =
    [ {<<"http">>, <<"localhost">>, 9000},
      {<<"http">>, <<"example.org">>, 80} ],

  %% NOTE: we assume we always deal with single origin
  [Val] = cow_http_hd:parse_origin(HeaderVal),
  case cow_http_hd:match_origin(Val, AllowedOrigins) of
    true -> {ok, handle_cors_request(HeaderVal, Req), Env};
    _ -> {ok, Req, Env}
  end;
execute(Req, Env) ->
  {ok, Req, Env}.

@essen, what do you think about merging CORS related parsing/producing functions for cowlib?

Yep will do. Almost done fixing a breaking bug, so probably after that.

@essen, what do you also think about putting the match_origin(Origin, [Origin] | '*') -> boolean() function into cowlib (perhaps, check_origin is a better name)? I could make a PR.

In theory I have nothing against it but right now I've been away from CORS too long so I don't want to make too hasty decisions. I think I will want to have an rfc6454 test suite in Cowboy before deciding on where things should go. And that is largely dependent on adding CORS support to cowboy_rest.


Calling I18n.locale early breaks setup of I18n with default_locale

Calling I18n.locale early will set the locale permanently to :en. Calling default_locale is ignored.
We have a rails app where using a gem broke localization because this gem calls I18n.locale before the standard rails configuration script config/application.rb was run, which sets the default_locale to :de.
Calling I18n.locale too early should not break the localization setup.
Also see the bug that we opened for the corresponding gem, https://github.com/bernat/best_in_place/issues/393

Closing as a duplicate of #275


Release and Debug configurations are not treated same as in Xcode

In Xcode these configurations are capitalized but Struct they are treated as being lowercase, thus "Release" doesn't default to having a release type but "release" does.
(I know it's one of these quirky things)

Yeah this is mainly a struct opinionated thing. One solution I had was to recognise both 'debug' and 'Debug' without having to specify the type. I think this would work well 😄

@lyptt Yeah that would make sense :)

Fixed in 2.0.0.


Update TokenPocket's logo

Update TokenPocket's logo

Looks good, can you also update the one here? https://github.com/solana-labs/wallet-adapter/tree/master/packages/wallets/icons

Sure, done @jordansexton

Thanks!

Any plan to update the npmjs package @solana/wallet-adapter-tokenpocket recently? 😄

@chendatony31 can you open a PR to bump the patch version?


Use npm-shrinkwrap.json for bower

We had an accident: #1599
It happened because the dependency of inquirer (that we locked to 0.7.1) were updated.
Probably only solution to prevent such cases it to deploy npm-shrinkwrap.json and make monthly patches with updates to it, instead counting that any update of 450 dependencies of bower won't break it at any time.
It seems reasonable because bower is usually installed as global dependency.
@bower What do you think?

Bower 1.6.2 is using shrinkwrap and bundledDependencies


Fixed a comment typo ("Conent-Length" -> "Content-Length")

This change is extremely important and will have wide-ranging impact on all developers.

Thanks for the contribution Jesse, we'll certainly take due care with this change when we review it ;-)
Padraic


Circular dependencies

Composition currently doesn't check for circular dependencies which makes things catch 🔥

#23 Resolves this


PROD-277/incorrect-deck-reveal-state

Some older decks do not have activeFromDate so I added createdAt to be read if there is no activeFromDate. Another alternative is to update those decks by hand in db and add them activeFromDate.

i am going to update this by hand after getting admin panel. will defer to @marvinmarnold on whether we'd still like to merge this (my understanding is that this state should not be possible in the future, but i think the extra redundancy could be nice)

Left a comment: https://linear.app/gator/issue/PROD-277/incorrect-deck-reveal-state#comment-5e7063e8


bitcloud.c and bitcloud.js are now submodules

The associated repositories also need moving to wetube.

I changed the paths to wetube/bitcloud after confirming that it works. If you clone with --recursive, you'll find that it fails because the repository transfer has not yet been approved.


Getting " One or more errors occurred" while opening GUI

Expected Behavior
Ui should have been loaded
Current Behavior

Steps to Reproduce (for bugs)
Your Environment

Module version used:2.1.5
Operating System and PowerShell version:Windows 10 / 7

Same error here, this time on both Windows and Linux Docker.

Same error here as well, using Windows.

Hey, wanted to chime in again and say that I got this error again. I was able to fix it without reinstalling by setting the theme to "light" in the jvSettings.json settings file. For some reason it was set to null. I'm assuming since it's set to null it can't choose a theme to go with and gets locked up. Hope this helps.


not a valid identifierts/gvm-default

Receiving the following error when attempting to install:
Created profile for existing install of Go at /c/go
': not a valid identifierts/gvm-default: line 1: unset: `GOROOT
': not a valid identifierts/gvm-default: line 2: unset: `GOARCH
': not a valid identifierts/gvm-default: line 3: unset: `GOOS
': not a valid identifierts/gvm-default: line 4: unset: `GOPATH
': not a valid identifierts/gvm-default: line 5: unset: `GOBIN
/home/hansw013/.gvm/scripts/gvm-default: line 6: $'\r': command not found
': not a valid identifierts/gvm-default: line 7: unset: `gvm_go_name
': not a valid identifierts/gvm-default: line 8: unset: `gvm_pkgset_name
/home/hansw013/.gvm/scripts/gvm-default: line 9: $'\r': command not found
: ambiguous redirectscripts/gvm-default: line 10: 1
: ambiguous redirectscripts/gvm-default: line 11: 1
: ambiguous redirectscripts/gvm-default: line 12: 1
: ambiguous redirectscripts/gvm-default: line 13: 1
: ambiguous redirectscripts/gvm-default: line 14: 1
/home/hansw013/.gvm/scripts/gvm-default: line 15: $'\r': command not found
: No such file or directory/gvm-default: line 20: /home/hansw013/.gvm/scripts/env/gvm
/home/hansw013/.gvm/scripts/gvm-default: line 21: $'\r': command not found
Installed GVM v1.0.22```

Please restart your terminal session or to get started right away run
 `source /home/hansw013/.gvm/scripts/gvm`

Can you provide more information like distro, kernel, gvm version, what do you do to receive this message?

Same problem. When using the install script.
bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)
Created profile for existing install of Go at /usr/local/go ': not a valid identifier/gvm-default: line 1: unset: GOROOT
': not a valid identifier/gvm-default: line 2: unset: GOARCH ': not a valid identifier/gvm-default: line 3: unset: GOOS
': not a valid identifier/gvm-default: line 4: unset: GOPATH ': not a valid identifier/gvm-default: line 5: unset: GOBIN
: command not foundcripts/gvm-default: line 6:
': not a valid identifier/gvm-default: line 7: unset: gvm_go_name ': not a valid identifier/gvm-default: line 8: unset: gvm_pkgset_name
: command not foundcripts/gvm-default: line 9:
: ambiguous redirectripts/gvm-default: line 10: 1
: ambiguous redirectripts/gvm-default: line 11: 1
: ambiguous redirectripts/gvm-default: line 12: 1
: ambiguous redirectripts/gvm-default: line 13: 1
: ambiguous redirectripts/gvm-default: line 14: 1
: command not foundcripts/gvm-default: line 15:
: No such file or directoryvm-default: line 20: /Users/nswsu/.gvm/scripts/env/gvm
: command not foundcripts/gvm-default: line 21:
Installed GVM v1.0.22
Please restart your terminal session or to get started right away run
source /Users/nswsu/.gvm/scripts/gvm
`
gvm version: GVM v1.0.22
kernel： Darwin NSWTechs-MacBook-Pro.local 20.6.0 Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:21 PDT 2021; root:xnu-7195.141.6~3/RELEASE_X86_64 x86_64

Same.
 I use these commands to install gvm :
sudo apt-get update
sudo apt-get install curl git mercurial make binutils bison gcc build-essential
bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)
I get the folowing ouputs:
Cloning from https://github.com/moovweb/gvm.git to /home/cecile/.gvm                                                                                No existing Go versions detected                                                                                ': not a valid identifier/gvm-default: line 1: unset: GOROOT                                                                                ': not a valid identifier/gvm-default: line 2: unset: GOARCH                                                                                ': not a valid identifier/gvm-default: line 3: unset: GOOS                                                                                ': not a valid identifier/gvm-default: line 4: unset: GOPATH                                                                                ': not a valid identifier/gvm-default: line 5: unset: GOBIN                                                                                /home/cecile/.gvm/scripts/gvm-default: line 6: $'\r': command not found                                                                                ': not a valid identifier/gvm-default: line 7: unset: gvm_go_name                                                                                ': not a valid identifier/gvm-default: line 8: unset: gvm_pkgset_name                                                                                /home/cecile/.gvm/scripts/gvm-default: line 9: $'\r': command not found                                                                                : ambiguous redirectripts/gvm-default: line 10: 1                                                                                : ambiguous redirectripts/gvm-default: line 11: 1                                                                                : ambiguous redirectripts/gvm-default: line 12: 1                                                                                : ambiguous redirectripts/gvm-default: line 13: 1                                                                                : ambiguous redirectripts/gvm-default: line 14: 1                                                                                /home/cecile/.gvm/scripts/gvm-default: line 15: $'\r': command not found                                                                                : No such file or directoryvm-default: line 20: /home/cecile/.gvm/scripts/env/gvm                                                                                /home/cecile/.gvm/scripts/gvm-default: line 21: $'\r': command not found                                                                                /home/cecile/.gvm/scripts/env/cd: line 2: $'\r': command not found                                                                                /home/cecile/.gvm/scripts/env/cd: line 5: $'\r': command not found                                                                                /home/cecile/.gvm/scripts/functions: line 2: $'\r': command not found                                                                                /home/cecile/.gvm/scripts/functions: line 3: syntax error near unexpected token $'do\r''                                                                                'home/cecile/.gvm/scripts/functions: line 3: for function in $GVM_ROOT/scripts/function/*; do                                                                                : numeric argument requirednv/cd: line 6: return: 1                                                                                Installed GVM v1.0.22                                                                                Please restart your terminal session or to get started right away run                                                                                source /home/cecile/.gvm/scripts/gvm `
Has anyone found the solution, please?
Thank you

I solved the problem executing this command :
find /home/<user>/.gvm/ -type f -exec sed -i 's/\r$//' {} \;


[backport release/2.7] First draft of actions based ci

Signed-off-by: Chris Patterson<EMAIL_ADDRESS>

Is this a duplicate of #3338 ?

@milosgajdos  it's for release/2.7, we should also enable the gitaction for release branch.


After sign up issue

Current Behavior
After the user signs up, the sign up button still appears on the screen.

Expected Behavior
The button should disappear after successful signup.
I would like to work on this issue.
Environment

Host OS: Mac Linux Windows
Platform: Docker or Kubernetes
Meshery Server Version: stable-v
Meshery Client Version: stable-v

Contributor Guides and Resources

🛠 Meshery Build & Release Strategy
📚 Instructions for contributing to documentation

Meshery documentation site and source

🎨 Wireframes and designs for Meshery UI in Figma
🙋🏾🙋🏼 Questions: Layer5 Discussion Forum and Layer5 Community Slack

Thank you for raising this issue, please go ahead and fix it :rocket:

Hey, I would like to work on this!!

I have made the changes. Can u please review @thisiskaransgit

Hey @thisiskaransgit I opened a pr for this issue months ago and it's also approved.  It would be helpful if is merged or if any other changes are required, please suggest


Environ shim

Remake of https://github.com/rust-lang/miri/pull/1147. There are three main problems with this:

For some reason update_environ is not updating environ when setenv or unsetenv are called. Even then it works during initialization.
I am not deallocating the old array with the variables in update_environ.
I had to store the environ place into MemoryExtra as a field to update it. I was thinking about changing extern_statics to store places instead of AllocIDs to avoid this.

@RalfJung

For some reason update_environ is not updating environ when setenv or unsetenv are called. Even then it works during initialization.

That is curious. Could you please add a testcase demonstrating the problem?

I had to store the environ place into MemoryExtra as a field to update it. I was thinking about changing extern_statics to store places instead of AllocIDs to avoid this.

No I think what you did makes sense. There's just multiple parties that care about this particular allocation.
It is crucial though that the field in MemoryExtra never gets changed after initialization; maybe add a comment along those lines.

That is curious. Could you please add a testcase demonstrating the problem?

sure, check https://github.com/rust-lang/miri/pull/1208/commits/18f03390c8a7423228a3999ffa5d9774076596e9

What remains is to deallocate the old memory, and to add a test that make sure it got deallocated.

:umbrella: The latest upstream changes (presumably #1209) made this pull request unmergeable. Please resolve the merge conflicts.

add a test that make sure it got deallocated.

How can I do that?

Yeah it's a bit tricky... it has to be a compile-fail test I think.
I suggest you declare environ as an extern static in that file, then get the ptr once early and deref it to test if that works. Then change the env, and try re-using the old ptr, now Miri should error.

Yeah it's a bit tricky... it has to be a compile-fail test I think.
I suggest you declare environ as an extern static in that file, then get the ptr once early and deref it to test if that works. Then change the env, and try re-using the old ptr, now Miri should error.

That sounds really nasty... I like it

Looking great, thanks. :)
@bors r+

:pushpin: Commit 8392a0c589461e998a28a52a070b5fa5a143cf77 has been approved by RalfJung

:hourglass: Testing commit 8392a0c589461e998a28a52a070b5fa5a143cf77 with merge 574d81ce7958433c51dbed2e765aac9ba1f80bb7...

:sunny: Test successful - checks-travis, status-appveyor
Approved by: RalfJung
Pushing 574d81ce7958433c51dbed2e765aac9ba1f80bb7 to master...


NIFI-9543: Add bring-to-front functionality to labels

Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Link to Jira: https://issues.apache.org/jira/browse/NIFI-9543
This PR adds bring-to-front functionality to labels on the canvas. To test, add multiple overlapping labels on the canvas, right click on a label below the others to open the context menu and choose Bring to Front. The label should now be at the top of the other overlapping labels. This label that's now at the front should be persisted even after shutting down and restarting NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically main)?

[x] Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not squash or use --force when pushing to allow for clean monitoring of changes.

For code changes:

[ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[ ] Have you written or updated unit tests to verify your changes?
[ ] Have you verified that the full build is successful on JDK 8?
[ ] Have you verified that the full build is successful on JDK 11?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.

Reviewing...

Thanks @mcgilman . I haven't had a chance to completely review/test this yet. One thing that I think we need to address, though, is something that I just noticed the other day with the 'retry' fields that were recently added. I had to make a small update: https://github.com/apache/nifi/pull/5809/files#diff-fcc9a36983c80e3ddfb13c77ac2db46d83346e6f55f84f44cedfc7a33bf43e02R67-R177
The issue here was related to the fact that we add a new field to a component. So in this case, zIndex is added to label. If you've pushed your flow to Registry using 1.15.3 or earlier, and now you startup with this version, we end up seeing a "Local Change" to each of those labels. And you can't revert the change. As a result, you won't be able to change to a later version. So we need to make a similar update here: If there's a flow difference that changes the zIndex from null to the default value (0), then we need to return true from that isEnvironmentalChange method. Granted, isEnvironmentalChange is probably no longer the most accurate name for that method, but basically is used to filter out any flow change that we don't want to consider a "Local Modification".


Fix chainlock scheduler handling

Reindexing corrupted chain by clicking "ok" in a corresponding dialog when running qt always fails otherwise. To reproduce this just drop e.g. regtest/blocks/index/ and try starting qt with/without the patch.

Regression test for something like this?

The issue only appears when starting qt normally with a corrupted chain. Adjusted PR description.


Research / test alternate django packages

Research and test out other django scheduler packages compared to django diary

Demo list:
github.com/llazzaro/django-scheduler-sample

django-scheduler seems to be the most complete, Python 3, best supported one I can find with a thorough demo. Will play around with styling and models to see what we can take from it or if it's worth switching to this.

@hobson Feel free to review findings and check out Django Scheduler code for bits we feel would be good additions, not sure if fully replacing Diary with this would save much time or not at this point


Added Tim Sort Implementation in Python 3

Have you read the Contributing Guidelines on Pull Requests?
Yes
Description
Tim Sort is a sorting algorithm that is a hybrid of merge sort and binary insertion sort algorithm. It is a stable algorithm and works best on real-time data. In this, the list that needs to be sorted is first analyzed, and based on that it is divided into blocks known as Run. These runs are sorted using insertion sort one by one and then by merging these runs using the combine function used in merge sort. If the size of the Array is less than run, then Array gets sorted just by using Insertion Sort. The size of the run may vary from 32 to 64 depending upon the size of the array.
Code file is added and Documentation is updated for 'TimSort.py'.
Checklist

[x] I've read the contribution guidelines.
[x] I've checked the issue list before deciding what to submit.
[x] I've edited the README.md and link to my code.

Related Issues or Pull Requests
The PR is with reference to issue #7284

@ankitaggarwal23 @HarshCasper Can you please review my PR

@ankitaggarwal23 @iamrajiv @anushkrishnav Please review my PR.

Thanks!!


Add delayed tasks support

Some tasks need to be scheduled for future execution. Add support for this functionality (much like delayable messages)

This task got no further than investigation.
The delay-able message support previously added is in fact delayed tasks.
The requirements to delay a method call is as follows

Ensure task is imported from celery
from celery import task
Decorate function with task

@task
def my_func

Call func in asynchronous way
channel_send.apply_async(countdown=123)

On further reflection, some work in this area would be very beneficial.
Following the steps outlined about for a instance method would require the object itself to be serialized,and persisted for future execution. This is rather heavyweight, putting unnecessary load on redis. Additionally an object may be difficult to pickle.
Instead defining a future dispatcher would be better.