pyAFQ within QSIprep does not read --recon_spec json file correctly?

Summary
When running reconstruction using qsiprep for pyafq_tractography it seems that the custom config (given by --recon-spec) is not read correctly.
Additional details

QSIPrep version:  QSIprep 0.20.1.dev0+geda84d5.d20240112
Singularity version: 3.11.0

What were you trying to do?
Modify the pyafq_tarctography.json to change random_seeds parameter to true and n_seeds to 1500000. I created a new pyafq_tarctography_NYU.json config (below) based on pyafq_tarctography.json and mounted it inside the singularity image so it can be read using --recon-spec
{
    "description": "Use pyAFQ to perform the full Tractometry pipeline",
    "space": "T1w",
    "name": "pyafq_tractometry_NYU",
    "atlases": [],
    "nodes": [
        {
            "name": "pyafq_tractometry_NYU",
            "software": "pyAFQ",
            "action": "pyafq_tractometry",
            "input": "qsiprep",
            "output_suffix": "PYAFQ_TRACTOMETRY",
            "parameters": {
                "use_external_tracking": false,
                "export": "all",
                "random_seeds": true,
                "n_seeds": 1500000,
                "odf_model": "CSD",
                "seg_algo": "AFQ",
                "parallel_segmentation": "{'n_jobs': -1, 'engine': 'joblib', 'backend': 'loky'}",
                "n_points": 100,
                "clean_rounds": 5,
                "distance_threshold": 3,
                "length_threshold": 4,
                "min_sl": 20,
                "stat": "mean",
                "scalars": "['dti_fa', 'dti_md']",
                "import_tract": ""
            }
        }
    ]
}

What did you expect to happen?
Tracking until 1500000 bundles are created (example below comes from running pyAFQ (1.2) natively on the same data)
INFO:AFQ:Loading Image...
INFO:AFQ:Generating Seeds...
INFO:AFQ:Getting Directions...
INFO:AFQ:Tracking...
1%|          | 16149/1500000 [01:31<2:17:09, 180.30it/s]
Code run:
BIDS_DIR = "/scratch/jk7127/qsiprep_run/BIDS"

tracking_params = {"random_seeds" : True,
                  "odf_model" : "CSD",
                  "n_seeds" : 1500000}

brain_mask_definition = ImageFile(
    suffix="mask",
    filters={'desc': 'brain',
             'space': 'T1w',
             'scope': 'qsiprep'})

myafq = GroupAFQ(bids_path=BIDS_DIR,
                 segmentation_params={"seg_algo": "afq"},
               	 scalars=["dti_fa", "dti_md"],
  		 clean_params={"n_points":100, "clean_rounds":5, "distance_threshold":3,"length_threshold":4,"min_sl":20, "stat":"mean"},
                 tracking_params=tracking_params,
		 preproc_pipeline='qsiprep',
		 participant_labels=subjid
)

bundle_html = myafq.export_all(viz=False,afqbrowser=False, xforms=False,indiv=True)>

What actually happened?
INFO:AFQ:Loading Image...
INFO:AFQ:Generating Seeds...
INFO:AFQ:Getting Directions...
INFO:AFQ:Tracking...
31%|███       | 62192/200885 [05:57<16:00, 144.46it/s]
Seems that the parameters are not read as max number of fibers is 200885
Code run:
sub=$1

# load the freesurfer module
module load freesurfer/6.0.0
source /share/apps/freesurfer/6.0.0/SetUpFreeSurfer.sh

singularity run --cleanenv --containall --writable-tmpfs \
    -B /scratch/jk7127/qsiprep_run/work:/work,/scratch/jk7127/qsiprep_run/BIDS/,${FREESURFER_HOME}/license.txt:/opt/freesurfer/license.txt,/scratch/jk7127/qsiprep_run/pyafq_tractometry_NYU.json:/pyafq.json \
    /scratch/jk7127/Singularity/qsiprep.sif \
    /scratch/jk7127/qsiprep_run/BIDS/ /scratch/jk7127/qsiprep_run/BIDS/derivatives/ participant \
    --fs-license-file /opt/freesurfer/license.txt \
    --participant_label $sub \
    --output-resolution 1.5 \
    -w /work \
    --recon_only \
    --recon_spec /pyafq.json \
    --recon_input /scratch/jk7127/qsiprep_run/BIDS/derivatives/qsiprep/
exit 0

below is report.srt created by qsiprep before running reconstruction (seems all parameters are read correctly). report.srt comes from:
/scratch/jk7127/qsiprep_run/work/qsirecon_wf/sub-wlsubj121_pyafq_tractometry_NYU/sub_wlsubj121_ses_nyu3t02_acq_138vols_space_T1w_desc_preproc_recon_wf/pyafq_tractometry_NYU/run_afq/_report/report.rst

Original Inputs
---------------

* bval_file : /scratch/jk7127/qsiprep_run/BIDS/derivatives/qsiprep/sub-wlsubj121/ses-nyu3t02/dwi/sub-wlsubj121_ses-nyu3t02_acq-138vols_space-T1w_desc-preproc_dwi.bval
* bvec_file : /scratch/jk7127/qsiprep_run/BIDS/derivatives/qsiprep/sub-wlsubj121/ses-nyu3t02/dwi/sub-wlsubj121_ses-nyu3t02_acq-138vols_space-T1w_desc-preproc_dwi.bvec
* dwi_file : /scratch/jk7127/qsiprep_run/BIDS/derivatives/qsiprep/sub-wlsubj121/ses-nyu3t02/dwi/sub-wlsubj121_ses-nyu3t02_acq-138vols_space-T1w_desc-preproc_dwi.nii.gz
* itk_file : /scratch/jk7127/qsiprep_run/BIDS/derivatives/qsiprep/sub-wlsubj121/anat/sub-wlsubj121_from-MNI152NLin2009cAsym_to-T1w_mode-image_xfm.h5
* kwargs : {'n_seeds': 1500000, 'random_seeds': True, 'odf_model': 'CSD', 'seg_algo': 'AFQ', 'parallel_segmentation': {'n_jobs': -1, 'engine': 'joblib', 'backend': 'loky'}, 'n_points': 100, 'clean_rounds': 5, 'distance_threshold': 3, 'length_threshold': 4, 'min_sl': 20, 'stat': 'mean', 'scalars': ['dti_fa', 'dti_md'], 'import_tract': None, 'omp_nthreads': 8}
* mask_file : /work/qsirecon_wf/sub-wlsubj121_pyafq_tractometry_NYU/sub_wlsubj121_ses_nyu3t02_acq_138vols_space_T1w_desc_preproc_dwi_specific_anat_wf/resample_mask/sub-wlsubj121_desc-brain_mask_trans.nii.gz
* tck_file : <undefined>

Seems n_seeds parameter is not read at all. Setting it to a string in the recon_spec file does not cause any error.
Original Inputs
---------------

* bval_file : /scratch/jk7127/qsiprep_run/BIDS/derivatives/qsiprep/sub-wlsubj121/ses-nyu3t02/dwi/sub-wlsubj121_ses-nyu3t02_acq-138vols_space-T1w_desc-preproc_dwi.bval
* bvec_file : /scratch/jk7127/qsiprep_run/BIDS/derivatives/qsiprep/sub-wlsubj121/ses-nyu3t02/dwi/sub-wlsubj121_ses-nyu3t02_acq-138vols_space-T1w_desc-preproc_dwi.bvec
* dwi_file : /scratch/jk7127/qsiprep_run/BIDS/derivatives/qsiprep/sub-wlsubj121/ses-nyu3t02/dwi/sub-wlsubj121_ses-nyu3t02_acq-138vols_space-T1w_desc-preproc_dwi.nii.gz
* itk_file : /scratch/jk7127/qsiprep_run/BIDS/derivatives/qsiprep/sub-wlsubj121/anat/sub-wlsubj121_from-MNI152NLin2009cAsym_to-T1w_mode-image_xfm.h5
* kwargs : {'n_seeds': 'wrong_string', 'random_seeds': True, 'odf_model': 'CSD', 'seg_algo': 'AFQ', 'parallel_segmentation': {'n_jobs': 8, 'engine': 'joblib', 'backend': 'loky'}, 'n_points': 100, 'clean_rounds': 5, 'distance_threshold': 3, 'length_threshold': 4, 'min_sl': 20, 'stat': 'mean', 'scalars': ['dti_fa', 'dti_md'], 'omp_nthreads': 8}
* mask_file : /work/qsirecon_wf/sub-wlsubj121_pyafq_tractometry_NYU/sub_wlsubj121_ses_nyu3t02_acq_138vols_space_T1w_desc_preproc_dwi_specific_anat_wf/resample_mask/sub-wlsubj121_desc-brain_mask_trans.nii.gz
* tck_file : <undefined>

Thanks for reporting this! Just to see that I understand what's going on, could you share your recon_spec file and how you called it?

Oh - sorry - I only saw the last message and not the entire thread before it! I see your recon_spec is up there (and sorry for the slowness picking up this issue!).

OK - I think that I understand what's going on. The issue is that all the arguments in the recon_spec are being passed to ParticipantAFQ as one dict kwargs here: https://github.com/PennLINC/qsiprep/blob/bd788392cfe6182a581473d0c5685f6d06bcc8c3/qsiprep/interfaces/pyafq.py#L106, but tracking parameters (for example n_seeds) need to be in a separate nested dict called tracking_params, otherwise they are ignored. My suggestion would be to add the tracking parameters defaults as a dict in the same way that parallel_params are added: https://github.com/PennLINC/qsiprep/blob/bd788392cfe6182a581473d0c5685f6d06bcc8c3/qsiprep/data/pipelines/pyafq_tractometry.json#L36, but not 100% sure whether that would work.
@jk619 : I hope what I am saying makes sense to you. Would you mind running your processing with a recon_spec that looks like below? I've changed one line marked with a comment:
{
    "description": "Use pyAFQ to perform the full Tractometry pipeline",
    "space": "T1w",
    "name": "pyafq_tractometry_NYU",
    "atlases": [],
    "nodes": [
        {
            "name": "pyafq_tractometry_NYU",
            "software": "pyAFQ",
            "action": "pyafq_tractometry",
            "input": "qsiprep",
            "output_suffix": "PYAFQ_TRACTOMETRY",
            "parameters": {
                "use_external_tracking": false,
                "export": "all",
                "tracking_params: {"random_seeds": true, "n_seeds": 1500000},    #  <= This is what I've changed
                "odf_model": "CSD",
                "seg_algo": "AFQ",
                "parallel_segmentation": "{'n_jobs': -1, 'engine': 'joblib', 'backend': 'loky'}",
                "n_points": 100,
                "clean_rounds": 5,
                "distance_threshold": 3,
                "length_threshold": 4,
                "min_sl": 20,
                "stat": "mean",
                "scalars": "['dti_fa', 'dti_md']",
                "import_tract": ""
            }
        }
    ]
}

Hi Arliel,
Thanks for taking a look!
I tried both but none worked
"tracking_params": {"random_seeds": true, "n_seeds": 1500000},
"tracking_params": "{'random_seeds': true, 'n_seeds': 1500000}" - same  syntax as for parallel_segmentation

Introducing the tracking_params dict to the json file removed random_seeds and n_seeds from the report.rst (somewhat expected I think) but there is no sign of tracking_params there.
#previous:
* kwargs : {'n_seeds': 'wrong_string', 'random_seeds': True, 'odf_model': 'CSD', 'seg_algo': 'AFQ', 'parallel_segmentation': {'n_jobs': 8, 'engine': 'joblib', 'backend': 'loky'}, 'n_points': 100, 'clean_rounds': 5, 'distance_threshold': 3, 'length_threshold': 4, 'min_sl': 20, 'stat': 'mean', 'scalars': ['dti_fa', 'dti_md'], 'omp_nthreads': 8}
current:
* kwargs : {'odf_model': 'CSD', 'seg_algo': 'AFQ', 'parallel_segmentation': {'n_jobs': -1, 'engine': 'joblib', 'backend': 'loky'}, 'n_points': 100, 'clean_rounds': 5, 'distance_threshold': 3, 'length_threshold': 4, 'min_sl': 20, 'stat': 'mean', 'scalars': ['dti_fa', 'dti_md'], 'import_tract': None, 'omp_nthreads': 8}
PS. I looked at printouts produced by the recon workflow and seems I never get this info:
https://github.com/PennLINC/qsiprep/blob/bd788392cfe6182a581473d0c5685f6d06bcc8c3/qsiprep/workflows/recon/pyafq.py#L97-L100
Would be easy to trace the kwargs error based on that

I have a feeling that the order of the loops here might be causing the problem.
https://github.com/PennLINC/qsiprep/blob/bd788392cfe6182a581473d0c5685f6d06bcc8c3/qsiprep/workflows/recon/pyafq.py#L22-L48
Line 35 is never reached as tracking_params is not a default parameter provided by arg_dict = afb.func_dict_to_arg_dict(). arg_dict has parameters like n_seeds but they are not under a tracking_params group which seems to be relevant.
This is an attempt to fix it, however, the config file will only have parameters that belong to one of three groups "CLEANING_PARAMS", "SEGMENTATION", "TRACTOGRAPHY_PARAMS"). Not sure if any other should be considered.
import AFQ
import AFQ.utils.bin as afb

def _parse_qsiprep_params_dict(params_dict):
    arg_dict = afb.func_dict_to_arg_dict()
    kwargs = {}
    special_args = {
        "CLEANING_PARAMS": "clean_params",
        "SEGMENTATION": "segmentation_params",
        "TRACTOGRAPHY_PARAMS": "tracking_params"}

    for key in arg_dict.keys():
        if key in special_args.keys():
            if special_args[key] in params_dict.keys():
                for actual_key in params_dict[special_args[key]].keys():
                    if actual_key in arg_dict[key].keys():
                        kwargs[special_args[key]] = afb.toml_to_val(params_dict[special_args[key]])
        else:
            kwargs[key] = afb.toml_to_val(params_dict[key])

    for ignore_param in afb.qsi_prep_ignore_params:
        kwargs.pop(ignore_param, None)

    return kwargs

params_dict = {}
params_dict['tracking_params'] = {}
params_dict['tracking_params']['n_seeds'] = 1500000
params_dict['tracking_params']['random_seeds'] = True
params_dict['clean_params'] = {}
params_dict['clean_params']['n_points'] = 200
kwargs = _parse_qsiprep_params_dict(params_dict)
print(kwargs)

OUTPUT:
{'tracking_params': {'n_seeds': 1500000, 'random_seeds': True}, 'clean_params': {'n_points': 200}}
PS. Seems that TRACTOGRAPHY_PARAMS has relevant fields for tracking_params not TRACTOGRAPHY (line 29)

I think the problem lies here:
https://github.com/PennLINC/qsiprep/blob/bd788392cfe6182a581473d0c5685f6d06bcc8c3/qsiprep/workflows/recon/pyafq.py#L22-L48
All parameters are assigned directly to kwargs  (i.e kwargs[n_seeds] instead of kwargs[tracking_params][n_seeds]. I think the easiest is to find to which category n_seeds belongs (in this case [tracking_params]) and assign it there. This would avoid playing with the json file and adding tracking_params there.
Alternatively, when adding tracking_params to the json file (as you proposed)
"tracking_params": {"random_seeds": true, "n_seeds": 1500000},
line 35 is never reached as tracking_params is not a default parameter loaded in line 23.
This is my solution (hope it helps and makes sense) but parameters that are not a part of tracking_params, segmentation_params or cleaning_params are not currently assigned so probably it needs another fix.
import AFQ
import AFQ.utils.bin as afb
import pdb

def _parse_qsiprep_params_dict(params_dict):
    arg_dict = afb.func_dict_to_arg_dict()
    kwargs = {}
    special_args = {
        "CLEANING_PARAMS": "clean_params",
        "SEGMENTATION": "segmentation_params",
        "TRACTOGRAPHY_PARAMS": "tracking_params"}

    for key in params_dict.keys():
        for special_key in special_args.keys():
            if key in arg_dict[special_key].keys():
                if special_args[special_key] not in kwargs:
                    kwargs[special_args[special_key]] = {}

                kwargs[special_args[special_key]][key] = afb.toml_to_val(params_dict[key])

    for ignore_param in afb.qsi_prep_ignore_params:
        kwargs.pop(ignore_param, None)

    return kwargs

params_dict = {}
params_dict['n_seeds'] = 1500000
params_dict['random_seeds'] = True
params_dict['n_points'] = 200

kwargs = _parse_qsiprep_params_dict(params_dict)
print(kwargs)

OUTPUT
{'tracking_params': {'n_seeds': 1500000, 'random_seeds': True}, 'clean_params': {'n_points': 200}}
PS. It seems that tracking_params are stored under TRACTOGRAPHY_PARAMS not in TRACTOGRAPHY (line 29)


⚠ ClamAV has degraded performance

In 26eed5a, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 4159 ms

Resolved: ClamAV performance has improved in aa0fa2e.


Document potential micropurchase jobs for review/publication

As the product owner for cloud.gov, I want to publish descriptions of potential micropurchase jobs so that I can advertise them, discuss them, and tune their scope based on potential takers of the work.
Here's a good previous example:
https://github.com/18F/cg-atlas/issues/19
Acceptance Criteria:

There are markdown files in this repository with similar content for each of the draft list of jobs we're thinking about per attachment...

Micropurchase ideas.pdf

Another micropurchase-able thing: Adapt the UK Notify IP-restricting route-service to be more generically applicable, and show how to both restrict-to and block-from particular IP ranges.

And another: Harbor-as-a-Service.


Document docker exec concerns

Resolves #233

Thank you for the contribution @MattF-NSIDC ! I made a few minor adjustment, but it looks great. I'll merge it after the required (but not relevant) tests pass.


deps: update displaydoc to 0.2.5

Updates displaydoc to 0.2.5 to fix a future incompatibility warning (see https://github.com/twitch-rs/twitch_api/issues/410).
Since we do merges, I've added a fix for the unexpected_cfgs lint in a separate commit.

Same here, audit workflow was disabled


There is no Tab for Android yet ?

I see TabBarIOS but i can not find how to make something like that for Android. Is there way to make like that ?
I like this plugin but it often break with each update of react.
https://github.com/brentvatne/react-native-scrollable-tab-view

We don't have a core component like that, see: https://facebook.github.io/react-native/docs/known-issues.html#content
I suggest using the module you linked to and improving it.


[FEAT] Add Changelog

Is your proposal related to a problem?

With new features being added regularly, we need a better way of communicating the latest developments with users.
Describe the solution you'd like

A new section in the docs to highlight the latest features being added to splink. This could take the form of a formal changelog simply linking to the relevant parts of the docs or a more rounded blog where we can highlight the top stories and then link to parts of the docs with more detail.
Describe alternatives you've considered

Additional context

Waiting for material by mkdocs to release their blog capability into the open source version then we will start a blog as part of the doc site. Currently envisioning a wrap up of the main Splink developments every 3 months, but this could change.

Blog capability has just been released. I will mock up something in the next week to test it out. This is just the basic functionality, so (once we get it up and running and establish if it adds value) we will need to consider whether we want to stick with just the open-source version. Things like RSS feed integration will likely always be a premium feature and if we want this to be accessible as possible, we can decide if sponsorship is something we want to pursue.
There are other teams across MoJ who are using mkdocs for technical documentation (and have even created an open-source gov-style mkdocs theme) so if usage of mkdocs becomes more prevalent we may be able to justify become a sponsor as an organisation, but getting sign off for these things is always a bit tricky.


LUT configuration update problem

Can the resource_dir.ini file not be updated directly? Maybe adding a comment after updating the file will solve this problem, so you don't have to set the path after each update

For the modified resource_dir.ini, the file is not overwritten when updated with the git pull command.
If you update the plug-in in another way, implement a backup of the modified configuration file.

This can be solved if the updated file and the created file have some difference in name, just change it during the first configuration

Already changed the *. ini files default suffix to *. ini. example,  and need to change it to *. ini for the first configuration.


Doesn't work with auto layout.

I can't add subviews with auto layout.

I realise this issue is closed, but I have just started looking at this component and am having the same issue.
I am using autolayout to pin a label to a specific position and this seems to break the entire layout engine.
Any suggestions?

@yacir Thanks for getting back to me. Checkout a quick illustration here: https://github.com/Roost/CollectionViewSlantedDemo-AutoLayout
As you will see from the demo, the background image is sized incorrectly, and the label is not positioned as expected as per the auto layout constraints. Furthermore, changing the size of the item further highlights the issue.
By simply unchecking "Use Auto Layout" on CustomCollectionCell.xib, the cells revert to the expected behavior.

@Roost check this project ;)
CollectionViewSlantedDemo-AutoLayout-fixed.zip

@yacir Excellent, that's perfect.
May I suggest adding that information to the readme.


Introduce impl_bytelike_traits macro

There are two macros currently for fmt stuff that do similar things, arr_newtype_fmt_impl and hex_fmt_impl - the difference is not immediately obvious, its the way that the byte array is iterated. We can't remove hex_fmt_impl yet,  but in #2770 I managed to unify them. Without all the changes in #2770 its not guaranteed that we will be able to ...
As a step towards cleaning up the hashes macros introduce a new macro that adds all the traits we want for byte-like types. Use it to:

Implement stuff in the hash_trait_impls macro.
Implement stuff for sha256::Midstate
Remove arr_newtype_fmt_impl macro

Done as multiple steps, with api file updates so that it is clear how the public api is being effected.
Part of #2881

In 4287bf2ff35ca21cdd1f1e27341239fbcc499965:
The new as_byte_array method should be const.
Other than that d3f36fe5a76dde18bb9238cd54f9612d6d5e5943 looks great! I wonder if later we will want to extend this macro to also add the as_byte_array etc inherents.

Only change is adding the const.

#2893 should fix CI.

Note to self, pick this up after #3010 merges.

Re-basing this after all the recent work on hashes was quite serendipitous. Turns out we missed a few things from siphash::Hash. This is now a nice little clean up.

There are bugs in siphash still. Will try again another time.

Incredible. In addition to the bugs in CI that failed to notice this, my local CI had different bugs that prevented me from noticing it:

I was not running doctests https://github.com/nix-community/crate2nix/issues/194 (now I found a way to run them "manually" much the way I run clippy and other things that crate2nix does not support)
The schemars test is not a workspace but instead a special directory, and I can't run those because I don't have the infrastructure to get all their deps in place. IMO they should actually be a workspace, as should the embedded tests etc.

This also probably shows we need more testing of the public API of the crates in the rust-bitcoin repo other than just using them in bitcoin. (In addition to relying on rustdocs.)

@Kixunil this PR has been updated to no longer implement the macro on Midstate.
It is very large and does make a nontrivial change (introducing the impl_bytelike_traits macro) but it's mostly a set of bugfixes that are critical and blocking improvements to my local CI. I'd like to get it in. (I am running my local CI setup on it now; it is at 10 hours and counting because my system scales to large numbers of commits very poorly, at least on this repo which has many crates with many features) but should finish in the next few hours.
I think the new macro needs some iteration -- it's called with a list of unlabeled data instead of looking like code, for one thing -- but we can do that in a followup PR. So would you be willing to review/ack this.
Alternately, @tcharding can you pull out the simple bugfix stuff into separate PR(s) which are trivial and which I can one-ACK merge? Preferably new PR(s) with a smaller number of commits than 9 :)

I can but the very first commit needs an ACK from @Kixunil anyways so I'm not sure it will speed things up. Patch 2, 3, and 4 are totally boring and I think they won't slow review down. However if we require changes to patch 5, 6, or 7 anyways I'll split it up. I'm hopeful that we can merge as is and do follow ups, lets see.

I am running my local CI setup on it now; it is at 10 hours and counting because my system scales to large numbers of commits very poorly

I found and fixed the bug in my Nix derivation (which was doing a bunch of unnecessary recomputation and taking close to 2 seconds for every one of 5020 jobs), so a run like this should take less than 2 hours rather than 20+.
So going forward these many-commit PRs should be a minor annoyance for me rather than a "stop doing local CI for an entire day" situation.

On ice so #3200 can go in with just the bug fixes.

Some considerations:

The "reverse" stuff is bitcoin specific, invalidating it being put in hex, right?
internals would then have to depend on hex, so far I had it in mind that internals was a leaf crate but I guess that because hex is not bitcoin specific then it will never be allowed to depend on bitcoin-internals so this is ok.
If we want hashes to stabalize we need to be careful not to change the output of this macro (and others) eventually. Having them over in internals makes me nervous that we will want the change somewhere else and forget hashes, I'm inclined to think that as we stabalize we will want to share macros between crates less or somehow mark them as 'used for  stable code generation, don't modify the output'

But if none of that matters, I can move it.

The "reverse" stuff is bitcoin specific, invalidating it being put in hex, right?

I prefer keeping it out until there's a good reason (e.g. perf problems reversing the hashes). But I'm not strongly opposed to having it in hex - it should be mostly harmless.

internals would then have to depend on hex, so far I had it in mind that internals was a leaf crate but I guess that because hex is not bitcoin specific then it will never be allowed to depend on bitcoin-internals so this is ok.

That's not a problem, internals is allowed to depend on whatever, though it needs to be gated. The idea of enforcing a crate to stay a leaf indefinitely is not useful.

or somehow mark them as 'used for  stable code generation, don't modify the output'

This was my intention for a while and I still think it's a good approach.

Can we do it as a follow up (#3320) or do you guys want it done now?

I'd prefer as a followup. It's more risky than I'd considered.

Sweet ,lets go!

Can't serde simply call as_ref instead of indexing? Should be a trivial change.

This PR does not add SliceIndex, it maintains the current behavour and removal of SliceIndex is unrelated to this PR. FTR I've tried it before and I tried it again this morning. Unless I am being brain dead it is not trivial to remove the SliceIndex impl.
This PR has been stalled for almost three months, can we just get it in and move on?

Holy f***, there is already a draft PR to do the SliceIndex stuff: https://github.com/rust-bitcoin/rust-bitcoin/pull/3296#issuecomment-2333315297

Currently is broken. Not the first patch was added after I struggled with the compiler only to find the same problem in patch 3.
First one gets this error:
error: expressions must be enclosed in braces to be used as const generic arguments
   --> hashes/src/util.rs:192:49
    |
192 |           $crate::impl_bytelike_traits!($newtype, <$newtype as $crate::Hash>::LEN, <$newtype as $crat...

Then if adding {} as suggested on gets this error:
error[E0308]: mismatched types
   --> hashes/src/util.rs:192:51
    |
192 |           $crate::impl_bytelike_traits!($newtype, { <$newtype as $crate::Hash>::LEN }, <$newtype as $crate::Hash>::DISPLAY_BACKWARD);

Before I rebased I did not see this problem, and I hard reset to double check nothing had changed - well something has changed but I can't work out what.

Hey @apoelstra if you get a second could you take a look at this please, non-urgent though. If you check out the first commit and try and build you'll get
error: expressions must be enclosed in braces to be used as const generic arguments

Looks simple, but I cannot workout how to fix it. The compiler suggestion just leads to another error.
Thank in advance.

@tcharding super weird. But this "fixes" it

diff --git a/hashes/src/internal_macros.rs b/hashes/src/internal_macros.rs
index c231b237b..625b8d8d0 100644
--- a/hashes/src/internal_macros.rs
+++ b/hashes/src/internal_macros.rs
@@ -20,7 +20,7 @@
 /// `from_engine` obviously implements the finalization algorithm.
 macro_rules! hash_trait_impls {
     ($bits:expr, $reverse:expr $(, $gen:ident: $gent:ident)*) => {
-        $crate::impl_bytelike_traits!(Hash, $bits / 8, $reverse $(, $gen: $gent)*);
+        $crate::impl_bytelike_traits!(Hash, { $bits / 8 }, $reverse $(, $gen: $gent)*);

         impl<$($gen: $gent),*> $crate::GeneralHash for Hash<$($gen),*> {
             type Engine = HashEngine;
diff --git a/hashes/src/macros.rs b/hashes/src/macros.rs
index fe658b17b..7a0cfdbb5 100644
--- a/hashes/src/macros.rs
+++ b/hashes/src/macros.rs
@@ -41,7 +41,7 @@ fn from_str(s: &str) -> $crate::_export::_core::result::Result<Self, Self::Err>
         $crate::hex::impl_fmt_traits! {
             #[display_backward($reverse)]
             impl<$($gen: $gent),*> fmt_traits for $ty<$($gen),*> {
-                const LENGTH: usize = $len;
+                const LENGTH: usize = ($len); // parens required due to rustc parser weirdness
             }
         }

IMO there is some bug in the rustc parser somewhere.

const LENGTH: usize = ($len); // parens required due to rustc parser weirdness

Nice find,  out of interest what led you to try this? I tried putting {} on all the other places that use $len but I did not try to paranthesise this one.

The compiler suggested adding {}s as I did. At first I tried putting braces around all the uses of $len but it didn't like that, so I put it on the call site as it suggested. Then it complained that I needed to add ()s at the call site. I tried that and it went back to wanting braces. So instead I tried adding them at the usage sites, and that worked.

So instead I tried adding them at the usage sites, and that worked.

Nice! I did everything else you did but did not make this leap.


[Feature Request] Custom Implementation of BearerTokenAuthenticator

Background
We think the security of the private key used for JWT signing is important, and we want to use Cloud KMS on Google Cloud with this library.
The current implementation of AppStoreServerAPIClient only signs the JWT with the on-memory private key, and we'd like this library to accept injecting arbitrary implementation of JWT signing.
Proposal
We want to add setBearerTokenAuthenticator to the AppStoreServerAPIClient class and refactor BearerTokenAuthenticator into an interface.
The existing BearerTokenAuthenticator would be renamed OnMemoryBearerTokenAuthenticator, and it would not be instantiated when the signingKey parameter is null.
This proposal will enable users to integrate any JWT signing backend, and you don't need to maintain the detailed implementation of the JWT signing.
We are eager to contribute by creating a pull request to implement this feature when you agree to this proposition.
Thanks in advance!

@umireon I had a project going to already significantly overhaul some aspects of the client so threw this in there, let me know if https://github.com/apple/app-store-server-library-java/pull/109 would meet your needs

Yes, that PR implements the exact feature that we need with your library! Thanks!

Version 3.0.0 was just released

Thank you for implementing this ❤


[Wallet] Creating wallet with wrong PIN

Preface
While I was fixing https://github.com/celo-org/celo-monorepo/pull/2932, I've discovered the follwoing:

We ask user to type PIN and then confirm
We store this PIN in cache
Creating account
a. If PIN is still in cache we are using it to store private key on disk
b. If PIN is not in cache anymore (app closed or timeout) we ask user to enter the PIN and using it to store private key on disk.
https://github.com/celo-org/celo-monorepo/blob/1b5b5aeca366e0873b22abb0abe222de9e3bd7a4/packages/mobile/src/web3/saga.ts#L190-L200

Expected Behavior
In step 3b we should use the same PIN, user entered earlier or again ask two times to ensure, that user did not do a mistake.
Current Behavior
In step 3b we are using whatever PIN user has entered discarding previously entered PIN.

Possible solution I see - ask user to enter PIN only after verifying invite code/restoring from words.

@i1skn can this be closed?

I can't reproduce this, is this still possible after the komenci flow changes? The account is created immediately after setting the PIN, so there doesn't seem to be much room to cause this issue. I tried commenting out the account creation after setting the PIN and restarting the app so that the cache is clean, but the user starts from the education screen in that case, so they have to enter their PIN and confirm it again.

I will double check if I can still reproduce it

This issue has been fixed with the changes to onboarding flow, cause we are creating an account immediately after setting the PIN now as @gnardini pointed out.


Moved some functions and variables out of tf.test.TestCase in image ops.

See #1328
This is the start of a refactoring. I want to avoid a diff of 150 lines at once.

@mainak431
You are owner of some files modified in this pull request.
Would you kindly review the changes whenever you have the time to?
Thank you very much.


Fatal Error

Hi, I'm trying to open the Publish and scriber page but i'm getting Error:
Fatal error: Class 'phpMQTT' not found in C:\xampp\htdocs\websockets\examples\subscribe.php on line 12 .
Links:
Publish : http://localhost/websockets/examples/publish.php
Subscribe : http://localhost/websockets/examples/subscribe.php

I am restarting this project so having a clear out of old issues (If you believe this is still valid please reopen)


Fix simplify function call

Call the function rather than redeclaring its signature, probably just a copy-paste error.

Thanks!


caddy-gen: generate endpoint for metrics

Note that prometheus of docker registry is available on 2.7. We'll upgrade registry later.


[flutter_tools] copy flutter_texture_registrar.h header for Windows shell

Description
Once flutter/engine#19405 has landed we need to ensure that the newly introduced header flutter_texture_registrar.h gets copied on Windows.
Related Issues
#38601
flutter/engine#19405
Tests
I added the following tests:
I created a new plugin with Windows platform support and verified that the respective header file gets copied as expected.
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
[x] I signed the [CLA].
[x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
[x] I read the [Tree Hygiene] wiki page, which explains my responsibilities.
[x] I updated/added relevant documentation (doc comments with ///).
[x] All existing and new tests are passing.
[x] The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Did any tests fail when you ran them? Please read [Handling breaking changes].

[x] No, no existing tests failed, so this is not a breaking change.
[ ] Yes, this is a breaking change. If not, delete the remainder of this section.

[ ] I wrote a design doc: https://flutter.dev/go/template Replace this with a link to your design doc's short link
[ ] I got input from the developer relations team, specifically from: Replace with the names of who gave advice
[ ] I wrote a migration guide: https://flutter.dev/go/breaking-changes-template Replace this with a link to a pull request that adds the migration guide to https://flutter.dev/docs/release/breaking-changes

Note that this can't be landed yet, however; see https://github.com/flutter/engine/pull/19405#issuecomment-758770197

Note that this can't be landed yet, however; see https://github.com/flutter/engine/pull/19405#issuecomment-758770197

Waiting on the recipe change that added the new header to the archive to propagate, then for the engine to roll. Once that happens I'll verify that the header is in fact in the archive, then we can land this.

Waiting on the recipe change that added the new header to the archive to propagate, then for the engine to roll. Once that happens I'll verify that the header is in fact in the archive, then we can land this.

Waiting on the recipe change that added the new header to the archive to propagate, then for the engine to roll. Once that happens I'll verify that the header is in fact in the archive, then we can land this.

The latest engine archive includes the header. Can you rebase this to the latest master? That should fix the Windows build_test failure (which was due to trying to copy the missing header).

Waiting on the recipe change that added the new header to the archive to propagate, then for the engine to roll. Once that happens I'll verify that the header is in fact in the archive, then we can land this.

The latest engine archive includes the header. Can you rebase this to the latest master? That should fix the Windows build_test failure (which was due to trying to copy the missing header).

Waiting on the recipe change that added the new header to the archive to propagate, then for the engine to roll. Once that happens I'll verify that the header is in fact in the archive, then we can land this.

The latest engine archive includes the header. Can you rebase this to the latest master? That should fix the Windows build_test failure (which was due to trying to copy the missing header).

Done. All checks have passed.

Waiting on the recipe change that added the new header to the archive to propagate, then for the engine to roll. Once that happens I'll verify that the header is in fact in the archive, then we can land this.

The latest engine archive includes the header. Can you rebase this to the latest master? That should fix the Windows build_test failure (which was due to trying to copy the missing header).

Done. All checks have passed.


when will the new version release?

such as the title

We have to wait for Apache approval, and be setting up Maven center repository


Are you going to publish this project to PIP?

The original cannot be installed on 3.9

This project is already on pip, and seems to work fine under python 3.9. Could you be more specific?

it is ok, I found it  had new name parse-torrent-title not name 👍


doc tags ignored if they appear immediately after @example

:bug: Bug Report
The .jsii manifest doesn't include doc tags if they appear immediately after @example.
Repro:
/**
 * @example foo
 * @boom tag
 */
export class Foo {
}

The .jsii manifest for Foo should include  the @boom doctag but it doesn't.
Affected Languages

[x] TypeScript or Javascript
[x] Python
[x] Java
[x] .NET (C#, F#, ...)

General Information

JSII Version: <!-- Output of jsii --version --> 0.20.7

Yep, known limitation of the TypeScript JSDoc parser.
The following:
/**
 * A function
 * @example
 * const x = require('@aws-cdk/aws-x');
 * frob(new x.Thing());
 */

Would be parsed as:
"@example" => "const x = reuqire('"
"@aws" => "-cdk/aws-x'); frob(new x.Thing());"

I can try and see whether the \n is maintained, that'll help parsing.

This used to work in the past so I guess something changed in how the docparser is used.

Correct.

So in order to properly detect the WHOLE example, I've had to absorb non-standard @tags into the @example itself.

Something we COULD do is tokenization, and absorb @-signs encountered within quotes, and treat @-signs encountered outside quotes as the start of a new tag.
Would solve most of our existing examples needs (would NOT solve the email-address-like counterexample I showed before).
But it seemed more complex than dictating that examples need to come last.
There is no way to error, because I can't tell whether the @domain is legitimately part of the example, or the start of a new tag that shouldn't have been there. If I could do that, I wouldn't need too error, because I could just parse it correctly.


Coloring in sap.ui.unified.Calendar

OpenUI5 version:
Any version
Browser/version (+device/version):
Any version
Any other tested browsers/devices(OK/FAIL):
URL (minimal example if possible):
User/password (if required and possible - do not post any confidential information here):
Steps to reproduce the problem:

We have option to mark the days in the calendar using special day feature. But we have limited color codes for day types.
Please find the below link related to color pallets for charts. Similar to the below, do we have any feature to use the various color shades to mark the days in the calendar using special days function ?
https://experience.sap.com/fiori-design-web/values-and-names/

What is the expected result?
What happens instead?
Any other information? (attach screenshot if possible)

Hello @chinthamanjunath
Correct my if I am wrong but you want to style the background of the day cell?
The bottom border can be styled using the color property of the DateTypeRange.
As the background of the cell is a considering the theme applied I don't think it can (and should) be styled outside the control, or you should care to have it with a correct  contrast ratio in all themes.
Regards,
Martin


Add create/update/delete observation endpoints from v1 API

Since the older Rails-based (v0) API is considered deprecated and may be retired at some point in the future, it would be good to add some of the create/update/delete features from the v1 API, starting with observations.

[ ] POST /observations
[ ] PUT /observations/{id}
[ ] DELETE /observations/{id}
[ ] Doctrings + type annotations
[ ] Usage examples
[ ] Sample response data
[ ] Unit tests

Usage details here: https://github.com/inaturalist/iNaturalistAPI/issues/257


Explain which log files are available via DB Console.

Note: we will fix the known limitations in v21.2.1, but for now it needs to be documented.

This change is


Block editing nationality once a candidate has submitted

Hide the actions for nationality, right to work and immigration status rows if the candidate has submitted their application. And prevent access to the pages via the URL.

 Block editing nationality once a candidate has submitted


Remove autorestci storage account and the publishing of the coverage reports

These autorest coverage reports are no longer used so we can disable the publishing and delete the storage account
We essentially need to remove the publishing from all the places that access storage-coverage-pass https://github.com/search?q=org%3AAzure+storage-coverage-pass&type=code we should also remove storage-coverage-user from KV and variable groups.

This work has been completed.


[spaceship] something wrong with select_team or login

New Issue Checklist

[ ] Updated fastlane to the latest version
[ ] I have read the Contribution Guidelines

Issue Description
All is right but no "app" api when I used 0.21.0 version, but after I updated to 0.25.1 version, I login an apple id which used to generate free ceriticate by xocde, it can not get teams any more, reponse is just "[]" in the log file.
but version 0.21.0 is ok, even more I can add device udid by api successsfully!
btw, it's ok, if I login an purchased iOS Developer ID.
Environment
space version :0.25.1

Can you please share the full log?

it's at 0.25.1
[11:05:36]: >> POST: https://developerservices2.apple.com/services/QH65B2/listTeams.action
[11:05:39]: << POST: https://developerservices2.apple.com/services/QH65B2/listTeams.action: {"teams"=>[], "creationTimestamp"=>"2016-04-12T03:05:39Z", "resultCode"=>0, "userLocale"=>"en_US", "protocolVersion"=>"QH65B2", "requestUrl"=>"https://developerservices2.apple.com:443/services/QH65B2/listTeams.action", "responseId"=>"ef683e66-4c57-4850-9f66-20c8bf580430"}
it's at 0.21.0
[19:10:39]: >> POST: https://developerservices2.apple.com/services/QH65B2/listTeams.action?clientId=XABBG36SBA {client: XABBG36SBA}, {myacinfo: DAWTKNV282bf99648d75cf4d2140774311516592adf9a363ce57011567e26149e735ece48d30902ab00af87add12700118c42920a6b94e789fd9506f0960bba9fa5dd30e51c642a9d0b7281f9730f25695f01992aace6310fb116b9d66e419729fdf330164a92b13d9670bbe0ba2aac119e489a88094a5c5413aab922fe74ac721a2f723efde6eac2aaac86b5540eced503858e92d6d44f3d008b9e2b9479ab65c7ceae338ea0cc8019b568dc8efbb981281956d3e40023a015b96d761586c63b2bf2ecc3c1c92249a45d27c9ad6443f6887c1365f057e27246e94785288a6539561526d9b5f4939a7f5e190d0c917e7680a74fecf72ea3a79cfc779d2dadaca470bd6ea04120b8d8beb6da1e5d4d60922b759193a5d70c7d05f69bd9ed02934605f215eMVRYV2}, {protocolVersion: QH65B2}, {requestId: e2b5b7f4-3511-43bf-b0b8-aa82a5bdc70c}, {userLocale: ["en_US"]}
[19:10:41]: << POST: https://developerservices2.apple.com/services/QH65B2/listTeams.action?clientId=XABBG36SBA: {"teams"=>[{"status"=>"active", "name"=>"vance xu", "teamId"=>"EJJMCSCDJ3", "type"=>"Individual", "extendedTeamAttributes"=>{}, "memberships"=>[{"membershipId"=>"RVZ47W4H75", "membershipProductId"=>"fp22", "name"=>"Xcode Free Provisioning Program", "status"=>"activeNeverExpire", "inIosDeviceResetWindow"=>false, "inRenewalWindow"=>false, "dateStart"=>#<DateTime: 2016-04-09T09:31:05+00:00 ((2457488j,34265s,0n),+0s,2299161j)>, "platform"=>"ios"}], "currentTeamMember"=>{"teamMemberId"=>"87W5YEK53Y", "personId"=>1254690211, "firstName"=>"vance", "lastName"=>"xu", "developerStatus"=>"active", "privileges"=>{}, "roles"=>["XCODE_FREE_USER"]}}], "creationTimestamp"=>"2016-04-11T11:10:41Z", "resultCode"=>0, "userLocale"=>"en_US", "protocolVersion"=>"QH65B2", "requestId"=>"e2b5b7f4-3511-43bf-b0b8-aa82a5bdc70c", "requestUrl"=>"https://developerservices2.apple.com:443/services/QH65B2/listTeams.action?clientId=XABBG36SBA", "responseId"=>"ab79d089-5658-4b3d-a41a-d717a1d0205a"}
login same account

OK, and which commands are you passing to spaceship?

@vancex Thanks for following up. We are going to look into this over here and follow up soon 👍 Let me know if you discover anything on your end while investigating. Thanks!

Encountered same issue today. Occurred with a specially created account for fastlane. I noticed that when going to the developer site I needed to accept some terms for that account. After this was done the issue was still there. What I still needed to do is invite that new account to my development team. Issue was solved then.

@vpolouchkine @TKBurner
sorry, I found that this spaceship doesn't support xcode api, account created by xcode can not get teamid by this spaceship api.
My fault~

@vancex Glad to hear you've managed to find the culprit.

@vancex Thanks for letting us know. Glad to hear you got up and running. Hope that you have a wonderful day! 👍


[core] Introduce avgCompactionTime metrics

Purpose

Linked issue: #4063

Tests

API and Format

Documentation

@JingsongLi Please help to review this PR, thanks


Database connection in DatabaseSessionHandler is not set correctly.

Describe the bug
The sessions table needs to be on the central database when using Jetstream
Steps to reproduce
Once Jetstream is installed, comment out the
// require DIR.'/auth.php';
from web.php in order to make use of the routes defined in Jetstream/Fortify (I've added the middleware in jetstream.php and fortify.php)
Expected behavior
The database connection should have the credentials from tenant instead of central domain
Dump from Illuminate\Session\DatabaseSessionHandler, line 64:

Dump from Stancl\Tenancy\Middleware\IdentificationMiddleware, line 36:

Your setup

Laravel version: 8.12
stancl/tenancy version: 3.4

The sessions table needs to be on the central database when using Jetstream

Hmm why exactly?

For me the issue is the other way around:
Jetstream is using the session table of central database instead of the tenant's one for authentication, when it is in tenant context.
I think it should use teanant's session table for this purpose.
I traced it back into laravel\framework\src\Illuminate\Session\SessionManager.php:
    protected function getDatabaseConnection()
    {
        $connection = $this->config->get('session.connection'); // <- uses central instead of tenant, when in tenant context
        return $this->container->make('db')->connection($connection);
    }

It seems tenancy is not initialized when this code runs.
I try to find a solution for this, maybe someone know one already?

Well, its the same way for me as well.
Let me try to explain again: It tries to find the sessions table on the central database, no matter if I’m in the tenant context or not.

I think I got mine working now with database session driver, by following these steps:

Enable Universal routes

add (as described) 'universal' to config of jetstream, fortify, livewire, and route.
skip adding 'InitializeTenancyByDomain' because it will be initialized globally by the next step

apply Rory's fix: https://stackoverflow.com/a/66261097
this changes middleware InitializeTenancyByDomain to be loaded globally except for central domains to fix a endless loop at login caused by a not separated session

Jetstream support is not an offered feature currently, there are some needed changes to the dependency injection in Jetstream etc. It was discussed on our Discord and I'll be implementing it in the new boilerplate.
Is there any bug specifically in the package? As in something that should be working a certain way per the docs but isn't?
If so, please open an issue for that. This has a lot of Jetstream-specific stuff that's hard to debug efficiently.


Fix auto_round import

Type of Change
bug fix
Description
RTN is not related with auto_round, so we should make this dependency independent.
Expected Behavior & Potential Risk
the expected behavior that triggered by this PR
How has this PR been tested?
how to reproduce the test (including hardware information)
Dependency Change?
any library dependency introduced or removed

@yiliu30 , What do you think of using LazyImport in autoround.py?

@yiliu30 , What do you think of using Lazyimport in autoround.py?

The neural_compressor/torch/algorithms/weight_only/autoround.py is for auto-round, it can import auto_round from anywhere in the code.
Importing these components lazily would be a better way?
https://github.com/intel/neural-compressor/blob/ba7564833ccf660688350235115ad9429efb078d/neural_compressor/torch/algorithms/weight_only/__init__.py#L15-L21
autoround = LazyImport("neural_compressor.torch.algorithms.weight_only.autoround.autoround_quantize")

If LazyImport is utilized, the autonomy of the algorithm code cannot be ensured, as it triggers the inclusion of additional INC API code.


Missing assets

Hello,
Thank you again for your wonderful efforts.
The login page doesn't seem to be loading assets. Do you have an idea why?
Please see errors below:

Best,
N

@marvoh Did you follow the steps?

Yes, I followed all those steps including stopping and starting the npm command like you did on your YouTube video but without success

@marvoh Could you please share your dev environment?
PHP version:
Nodejs version:
NPM version:
Laravel Version:

The problem is a dependency issue.
When I run npm install the following is the result:

A quick workaround is to run yarn install instead which is much better at resolving dependencies.
Laravel version: v11.5.0
PHP version: 8.3.6
Nodejs version: 20.12.2
NPM version: 10.6.0
Thank you for your quick response and desire to fix issues that we encounter from time to time

@marvoh Could you try on Nodejs 16/18?

I am sorry I haven't managed to try this but I think it could be the problem.
I'm wondering also whether it could be related to a new issue I am facing.
I have included the data table as follows in my blade template <x-tabulator :table="$table"></x-tabulator> and it renders almost okay when there is no data.
When I add data to the database, I realise that the assets (css / js) are not loaded automatically. Is this the expected behaviour therefore I add them manually?
Thank you again for the good work.

I'm sorry I have not managed to test it. I will let you know how it goes but it's currently working with yarn

@marvoh According to your scenario, it seems ok from the package end. Maybe it's a nodejs issue.


Make the parser operate on Iterator

What property can be improved?
maintainability due to loose coupling, testability
Right now, the Parser relies on a specific Scanner implementation, making the latter harder to replace.
This is especially significant in the test code which requires an own Scanner subclass.

The Parser should build upon a more general interface as Iterator as a sequence of Tokens is all the parser actually needs.


The requested URL returned error: 404 Not Found

E:\workshop\github-vm-clone\workshop-vm>vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'box-cutter/ubuntu1604-desktop' could not be found. Attempting to find and install...
default: Box Provider: virtualbox
default: Box Version: >= 0
==> default: Loading metadata for box 'box-cutter/ubuntu1604-desktop'
default: URL: https://vagrantcloud.com/box-cutter/ubuntu1604-desktop
==> default: Adding box 'box-cutter/ubuntu1604-desktop' (v2.0.26) for provider: virtualbox
default: Downloading: https://vagrantcloud.com/box-cutter/boxes/ubuntu1604-desktop/versions/2.0.26/providers/virtualbox.box
default: Download redirected to host: cdn.boxcutter.io
default:
An error occurred while downloading the remote file. The error
message, if any, is reproduced below. Please fix this error and try
again.
The requested URL returned error: 404 Not Found

thanks for the report it seems like https://vagrantcloud.com/box-cutter/boxes/ubuntu1604-desktop/versions/2.0.26/providers/virtualbox.box isn't available anymore.
I have to look for another box that comes with ubuntu desktop.


Depth First Search Python

Have you read the Contributing Guidelines on Pull Requests?
Yes.
Description
Depth-First Search Traversal in Python was corrected and updated.
Checklist

[x] I've read the contribution guidelines.
[x] I've checked the issue list before deciding what to submit.
[x] I've edited the README.md and link to my code.

Related Issues or Pull Requests
Issue #2398

@iamrajiv @ankitaggarwal23 @gat786 Please review when possible, and let me know if any changes are required.

@gat786 Thank you very much for the useful feedback. Thank your taking the time to give a detailed review.
I have made the requested changes.

@HarshCasper When you get time, can you add the level tag, please?


Number system Converter.

Describe the project you want to add with tech stack

Using HTML, CSS and Javascript, I want to create a website that performs conversion between several number systems.
Expected behavior

As soon as you give a number to the calculator; and give the number system in which the number is, the base of the number system in which the given number is to be converted will be asked. Hence, with all this information desired result will be displayed.
Screenshots (optional)

Additional context (optional )

I am a GSSoC23 contributor;
Please assign me this issue.
@Saurabh-kumar09


Detect missing processes

Intended to resolve YugaByte/yugabyte-db#1643 and YugaByte/yugabyte-db#1729.
For yb-ctl, add various process existence checks:

Ensure that no master or tserver processes exist at the beginning of
performing a create
Ensure that master and tserver processes exist after
ClusterControl.start_daemons when performing a create or start
Ensure that master and tserver processes continue to exist while performing
RPCs in ClusterControl.wait_for_cluster
Allow PIDs referenced by postmaster.pid to refer to already dead postgres
processes

@WesleyW - would be good to take a look at the fixes here and see if we need to incorporate any in yugabyte-db. cc @bmatican @schoudhury

@jj-kim please take a look at the test failures at https://travis-ci.org/YugaByte/yugabyte-installation/jobs/556584742


Fuse ElementwiseOps to its consumers

Request description
There might be a performance benefit if we push element-wise ops into its consumers directly. It may improve parallelism at the expense of added overhead to the consumers.
Here is an example graph from miniLM.

We may be able to move the ops in dispatch 4 into dispatch 14, 5, and 6.
Another instance:

dispatch 0 can be pushed into its consumers. But in this case, the input is coming from arg1, we can early execute dispatch 0 to overlap its computation with other dispatches. It is a trade-off between the computation overlapping and live range increase which contributes the maximum memory usage.
What component(s) does this issue relate to?
Fusion

This is low priority for now since the trade off is not easily determined due to the lack of performance query in the compiler stack and evaluation.

For your first example, we definitely dont want to do that. The result of the linalg.generic is an operand of linalg.matmul. THe access pattern for operands has a O(N) reuse. Doing this kind of fusion will result in that many recomputations of the linalg.generic operation. It also increases the memory bandwidth requirement since you will have O(number of args of generic * N) increase in bandwidth usage. You always want to fuse such operations with its producers.


Feature request: A browser-friendly version of or-tools (emscripten)

With web development frameworks such as Vue, React and Angular, a web-based version (WASM) of or-tools would make a big difference for us developers. The ability to solve many optimization problems directly in the browser would provide a better user experience and the opportunity to do work in the browser when offline.
Unfortunately, I'm not a star on tools like https://emscripten.org/, otherwise I would have made the first attempt myself. But can it be a simple task for someone else in this community?
Thank you for your hard work on this project!

As expected, you, or somebody else can try.  :-)
But I will not spend some time on it. I prefer the client server approach.
--Laurent
Le ven. 25 oct. 2019 à 22:33, Nils Franzen<EMAIL_ADDRESS>a
écrit :

With web development frameworks such as Vue, React and Angular, a
web-based version (WASM) of or-tools would make a big difference for us
developers. The ability to solve many optimization problems directly in the
browser would provide a better user experience and the opportunity to do
work in the browser when offline.
Unfortunately, I'm not a star on tools like https://emscripten.org/,
otherwise I would have made the first attempt myself. But can it be a
simple task for someone else in this community?
Thank you for your hard work on this project!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1670?email_source=notifications&email_token=ACUPL3M3BNH2CW3WA4RVH7DQQNJYHA5CNFSM4JFI2YQKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HUPJYYQ,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3NRZ4K4X67RI4PG3OLQQNJYHANCNFSM4JFI2YQA
.

There's a reason solvers and local search algorithms aren't implemented in high level languages. They're not suited for solving NP-hard problems.
If you really need to do this client side, you could look into packages like https://www.npmjs.com/package/simulated-annealing or something similar.
If you just need a simpler interface for modelling optimization problems client-side, check out the MiniZinc project.

An experimental/poc wasm-version of or-tools is available at https://github.com/kjartanm/wasm-or-tools (and wasm is not exactly high level :) )

Copy of the mailing list topic: Proof of concept WASM-build of OR-Tools
Few comments for myself (and sorry, I should have done this feedback  20 days ago when you reply in the issue):

Build on top of stable branch (v7.8)
https://github.com/kjartanm/wasm-or-tools/commits/stable

Basically use the CMake based build with an emscripten CMAKE_TOOLCHAIN_FILE -> GOOD
ed: While we still not support cmake toolchain files (i.e. not tested and out of our current scope), it is the way to go to support Android NDK and emscripten in the near future..
So any clean patch (i.e. without breaking change) to ease this future usage will be reviewed and integrated if possible !

Have some dirty hack to "fix" dependencies (glog and protobuf)
https://github.com/kjartanm/wasm-or-tools/commit/ef4b4fadea2c9550c0e81890490d844281ef65c7
note: For Glog why did we need to use "HAVE_SYMBOLIZE 0" ? and the hack of raw_looging -> provide a clean patch to glog to better support toolchain and/or emscripten target should be better IMHO...
note: For Protobuf, it  should be better to check for CMAKE_CROSSCOMPILING and in this case disable the build of the protoc binary during the fetch_content() (i.e. set protobuf_BUILD_PROTOC_BINARIES=OFF when CMAKE_CROSSCOMPILING is true using generator expression ?) AND build an HOST version instead...
Then we can replace the protobuf::protoc by a host_protoc alias target etc...

According to the emscripten doc:

The preprocessor define EMSCRIPTEN is always defined when compiling programs with Emscripten.
ref: https://emscripten.org/docs/compiling/Building-Projects.html#detecting-emscripten-in-preprocessor
So it should be better to properly fix the code by checking this define instead of this hack ;)
e.g. https://github.com/kjartanm/wasm-or-tools/commit/17a87e1e582af581eb191609effe8363ee91c920#diff-afde8dfcb72625759b3232fdcf38f378L164-L166

Using make_third_party, i.e. the Make based build, to retrieve all dependencies headers is not a "clean" CMake way to say the least ;)
Also I don't deeply understand why you should do it...
note: Our goal is to replace (i.e. remove) the Makefile based build by the CMake based build...

Instead of modifying the default value of few CMake options you should pass it in your call to CMake.
i.e. in cmake-build.sh:

-    cmake -DCMAKE_TOOLCHAIN_FILE=${EMSCRIPTEN_CMAKE_PATH} ..
+   cmake -DCMAKE_TOOLCHAIN_FILE=${EMSCRIPTEN_CMAKE_PATH} -DBUILD_DEPS=ON -DBUILD_SAMPLES=OFF -DBUILD_EXAMPLES=OFF -DBUILD_TESTING=OFF ..

ref: https://github.com/kjartanm/wasm-or-tools/commit/ed4d8709b2a3dcc023a611b09844259b76edfa28

wasn-build.sh should be refactored as a CMake file using a function and you shouldn't have to list by hand all dependent libraries of ortools (out of sync and should be already provided by the target attribute)
ref: https://github.com/kjartanm/wasm-or-tools/blob/17a87e1e582af581eb191609effe8363ee91c920/wasm-build.sh#L19-L87
note: need to check if there is already an "emscripten" CMake module providing this kind of tools (i.e. wrapper function for "emcc" and "emscripten") on the internet...

On archlinux:
pacman -S emscripten
cat /usr/lib/emscripten/cmake/Modules/Platform/Emscripten.cmake

This would be very useful. Is there an easy way to compile or-tools with emscripten?

@luiscastro193 now or-tools, on master branch, support CMAKE_TOOLCHAIN_FILES for C++, see https://github.com/google/or-tools/blob/master/tools/cross_compile.sh

Ok so I followed these steps:

Cloned the branch
Edited line 209 of the file at the build function as emcmake cmake -S. -B"${BUILD_DIR}" "${CMAKE_DEFAULT_ARGS[@]}" "${CMAKE_ADDITIONAL_ARGS[@]}"
Set export TARGET=x86_64 and export PROJECT=or-tools
Executed ./cross_compile.sh build

However, the building process fails at some point: output.txt
Am I doing something wrong?

I added
  case ${TARGET} in
    wasm64)
      expand_wasm_config
      declare -r QEMU_ARCH=DISABLED ;;
    x86_64)

to have wasm as a target, and
function expand_wasm_config() {
  declare -r TOOLCHAIN_FILE=${PROJECT_DIR}/emscripten/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake
  CMAKE_ADDITIONAL_ARGS+=( -DCMAKE_TOOLCHAIN_FILE="${TOOLCHAIN_FILE}" -DBUILD_SAMPLES=OFF -DBUILD_EXAMPLES=OFF -DBUILD_TESTING=OFF)
}

to set the toolchain and turn off some parts of the process.
It runs for a good while, but kicks the bucket around 85%, after Built target ortools_constraint_solver.

The blockers for compiling seems to be the same 2 minor points that I addressed in my PoC-build (https://github.com/kjartanm/wasm-or-tools -- I cleaned it up a bit, so it is less hacky than to begin with :) )
It can be handled using EMSCRIPTEN preprocessor variable. Would something like this be an option?
glop/lp_solver.cc, 160:
  // Note that we only activate the floating-point exceptions after we are sure
  // that the program is valid. This way, if we have input NaNs, we will not
  // crash.
#ifndef __EMSCRIPTEN__
  ScopedFloatingPointEnv scoped_fenv;
  if (absl::GetFlag(FLAGS_lp_solver_enable_fp_exceptions)) {
#ifdef _MSC_VER
    scoped_fenv.EnableExceptions(_EM_INVALID | EM_ZERODIVIDE);
#else
    scoped_fenv.EnableExceptions(FE_DIVBYZERO | FE_INVALID);
#endif
  }
#endif

base/raw_logging.cc, 28:
#if !defined(_MSC_VER) && !defined(__EMSCRIPTEN__)
#include <sys/syscall.h>  // for syscall()
#define safe_write(fd, s, len) syscall(SYS_write, fd, s, len)
#else
#if !defined(__EMSCRIPTEN__)
#include <io.h>  // _write()
// Not so safe, but what can you do?
#define safe_write(fd, s, len) _write(fd, s, len)
#else
#define safe_write(fd, s, len) write(fd, s, len)
#endif
#endif

Yes send us a PR.
Le mer. 17 févr. 2021 à 08:41, Kjartan Müller<EMAIL_ADDRESS>a
écrit :

The blockers for compiling seems to be the same 2 minor points that I
addressed in my PoC-build (https://github.com/kjartanm/wasm-or-tools -- I
cleaned it up a bit, so it is less hacky than to begin with :) )
It can be handled using EMSCRIPTEN preprocessor variable. Would
something like this be an option?
glop/lp_solver.cc, 160:
// Note that we only activate the floating-point exceptions after we are sure
// that the program is valid. This way, if we have input NaNs, we will not
// crash.
#ifndef EMSCRIPTEN
ScopedFloatingPointEnv scoped_fenv;
if (absl::GetFlag(FLAGS_lp_solver_enable_fp_exceptions)) {
#ifdef _MSC_VER
scoped_fenv.EnableExceptions(_EM_INVALID | EM_ZERODIVIDE);
#else
scoped_fenv.EnableExceptions(FE_DIVBYZERO | FE_INVALID);
#endif
}
#endif
base/raw_logging.cc, 28:
#if !defined(_MSC_VER) && !defined(EMSCRIPTEN)
#include <sys/syscall.h>  // for syscall()
#define safe_write(fd, s, len) syscall(SYS_write, fd, s, len)
#else
#if !defined(EMSCRIPTEN)
#include <io.h>  // _write()
// Not so safe, but what can you do?
#define safe_write(fd, s, len) _write(fd, s, len)
#else
#define safe_write(fd, s, len) write(fd, s, len)
#endif
#endif
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1670#issuecomment-780368864,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3NBSXU2JKWZCVCQR5TS7NXI7ANCNFSM4JFI2YQA
.

I have sent a PR for glop/lp_solver.cc and base/raw_logging.cc, but I also have a suggestion for cross_compile.sh that is currently in master. I have understood it so that you only want PRs on stable?

No, on master
Le sam. 20 févr. 2021 à 10:47, Kjartan Müller<EMAIL_ADDRESS>a
écrit :

I have sent a PR for glop/lp_solver.cc and base/raw_logging.cc, but I
also have a suggestion for cross_compile.sh that is currently in master.
I have understood it so that you only want PRs on stable?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1670#issuecomment-782596934,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3PSVASPUOUZXDWP633S76AMFANCNFSM4JFI2YQA
.


'dnd-core' and 'disposables' '.babelrc' file incompatibility issue with babel 6

I was having an issue using webpack with babel 6 in a react/redux project and using react-dnd. The only solution I had was to actually go into node_modules and delete the '.babelrc' file from both 'dnd-core' and 'disposables'. Webpack now works and the drag and drop components function as desired but this issue took quite a while to realize where it was going wrong because it was in the node_modules.

@darthtrevino I think this is still an issue.

please update this .npmignore of the project of disposables at the same time.


Documentation CI fails

Starting today the documentation and wheel CI builds are failing, e.g. https://dev.azure.com/pramodskumbhar/nmodl/_build/results?buildId=4098&view=logs&j=e663e406-bc69-5454-a50c-2daf70096c50&t=44a6ba8e-413d-536f-de25-37de658d0279:
Traceback (most recent call last):
  File "setup.py", line 145, in <module>
    install_requires=install_requirements,
  File "/root/nmodl/nmodl_build_venv36/lib/python3.6/site-packages/skbuild/setuptools_wrap.py", line 750, in setup
    return setuptools.setup(*args, **kw)
  File "/root/nmodl/nmodl_build_venv36/lib/python3.6/site-packages/setuptools/__init__.py", line 152, in setup
    _install_setup_requires(attrs)
  File "/root/nmodl/nmodl_build_venv36/lib/python3.6/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires
    dist.fetch_build_eggs(dist.setup_requires)
  File "/root/nmodl/nmodl_build_venv36/lib/python3.6/site-packages/setuptools/dist.py", line 815, in fetch_build_eggs
    replace_conflicting=True,
  File "/root/nmodl/nmodl_build_venv36/lib/python3.6/site-packages/pkg_resources/__init__.py", line 782, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (Sphinx 5.0.0 (/root/nmodl/.eggs/Sphinx-5.0.0-py3.6.egg), Requirement.parse('sphinx<5,>=3.1'), {'myst-parser'})

This is probably related to sphinx 5.0.0 being released yesterday: https://pypi.org/project/Sphinx/5.0.0


Improve simulated activity timeline rendering

Closes #608

Render spans even if their directives are not in view.
Make span text sticky to the left.
Display an arrow to indicate partially out of view spans when needed.

Is this intended?
There are two grandchild spans, but the first one is outside of the time span. It looks like grandchild is maybe one single span, but split into two parts. I suppose that makes sense to still show it, but I just wanted to confirm.

I wonder if the left grandchild is like 0-1px wide or just on the edge or something? Any easy way to repro this one?

Oh I see the issue, the span time bounds are not taking span children into account. Will fix

@duranb fixed, give it a whirl.


Campaign encounter creation

Associate an encounter with a campaign

i think i have a pretty good idea of what a campaign is, how are we defining an encounter? I'd tend toward a relationship like campaign HAS-A session HAS-A encounter?

It would be A Campagin (game) HAS-MANY encounters.  Those encounters may or may not be "encountered" during a session depending on the character actions and directions

oh i see. i was coming at it from a player's POV. that makes sense form the DM's perspective though


ROX-9476: Improve error handling on sensor start - use utils.Should

Description
Use utils.Should instead of logging the error during the sensor start. This will lead to a component crash when it is started in dev environment (non release build).
Checklist

[ ] Investigated and inspected CI test results
~- [ ] Unit test and regression tests added~
~- [ ] Evaluated and added CHANGELOG entry if required~
~- [ ] Determined and documented upgrade steps~
~- [ ] Documented user facing changes (create PR based on stackrox/openshift-docs and merge into rhacs-docs)~

Testing Performed
Manual dev test

Just to make it clear, I'm definitely for merging a one-line PR, but please make sure to reopen the ticket if it gets auto-closed.

@porridge thanks for your input! I'm going to check the Start behavior of the components and fix in the next PR. Looks like status auto update doesn't work after changing the JIRA workflow.


Tracking Firefox's built in dark theme

Firefox is starting to make some improvements to the built-in dark theme.  This issue will be used to track Firefox's progress and discuss the future directions of ShadowFox as these features are implemented.
New features:
🔻 FF61
Note: Unless uplifted, these changes will not make it into the next ESR cycle

Panels (navbar menus, drop downs, browser notifications etc.)
Awesome bar and search bar drop down
New tab page

Opinions on the new thememing:
The new themeing is still a little rough around the edges, but that's somewhat to be expected as they are only available in the Nightly channel at the time of writing this - hopefully there will be improvements on the way.
  Personally, I think they look a little out of place given the color of the navigation bar:

Same goes for the awesome bar drop down.  Additionally, I think having the "one-off" search engines as a lighter color draws your eye towards them instead of the results, which is the natural place to look:

Possible discussion topics:

Future direction of ShadowFox.  Namely, what should be done with features that are now styled by Firefox?

???

I'll continue to update this thread, but if anyone has any opinions/thoughts/concerns etc., feel free to comment below.  Otherwise, I'll move forward with whatever I feel is best.

I've also noticed Mozilla's plans and welcome them. Nonetheless I think you don't need to worry about ShadowFox. It'll take ages for them to release a complete dark skin - if it will ever be dark everywhere...

Looks like the some of the issues I had with the menus and address bar drop downs has been partially addressed:

@overdodactyl The built-in dark theme is becoming more and more comprehensive. I now only import:

color_variables.css (in both userChrome.css and userContent.css)
remove_white_flash.css
secure_connection_colors.css
about_pages.css
amo_store.css
directory_listings.css
manifest.css
raw_githubusercontent.css
and view_source.css

This set seems to pretty much do the trick for me these days.
Re: Future direction of ShadowFox, have you considered paring down ShadowFox with respect to the native dark theme? The import statement system is good, but it could be time to cut some code for all users. The small triangular element that is above menu popups is broken for me with ShadowFox, and the sidebar splitter is different than normal. Perhaps these still work on Mac, but they don't seem to work on Linux or Windows. I'm guessing the native theme will maintain better cross-platform support. I still love ShadowFox for theming about:pages, the AMO store, and add-ons, but perhaps its scope could be reduced which would make it easier to maintain?
Furthermore, I'm a little confused by the common files. Could you describe them? The only one that seems obvious to me is color_variables.css. Why do the other four need to be imported by both userChrome.css and userContent.css? radios_checkboxes.css doesn't seem to do anything on Linux, and the names of library.css and trees.css are confusing. I think trees.css is dropdown items?
Thanks for your work and for any insight you can provide!

Hi @lhindir - thanks for bringing up the concerns/thoughts here.

Re: Future direction of ShadowFox, have you considered paring down ShadowFox with respect to the native dark theme?

Yes, it's definitely been on my mind and something I've been wanting to do.  The goal of ShadowFox from the beginning was to be a supplemental tool to Firefox's default theme, not to style/re-style everything.  I've been pretty busy over the last couple months moving to a new state and starting up a new job, so unfortunately I haven't been able to put as much time into this project as I would like.  Pairing down on the code here would certainly help with maintenance, compatibility and future-proofing.
I'm also thinking of trying to simplify the workflow/build process in hopes of making it easier for others to contribute.
The common files are features that can be used either by user userContent or userChrome.  trees.css are the list like items such as about:config entries, but also pop up in places like about:settings. Same kind of thing for radio boxes (some are in settings pages and menus, others in dialogues such as page settings).  Most people access the library through the menu bar, but some also open it in a tab using the url chrome://browser/content/places/places.xul, so it's included in both userChrome and userContent.


Event page

Event page finished, almost identical to figma. I made one change, since I thought it could look better. When pressing readmore, instead of having more text being shown, a "modal" is popped up for more info. I thought this would look better, but mby not. Can fix it in that case.

Oh and also: I think the modal looks fine, but the close button is a bit hard to press. You have to press exactly on one of the lines that makes it up. Would be nice if you could close the modal anywhere in the square that the button covers.

All issues resolved, except for the images one. The bar picture seems to be used in FAQ. I'm a bit lazy so I could do it in another PR. I think I need to download the original pictures for the other pictures used as well, otherwise the the images scale badly when trying to use the image from FAQ.


docs: use netlify env to set NIMBELLA_LOGIN_TOKEN

Netlify CLI supports setting env variables, so we can avoid the context switch to the UI

Thanks @erezrokah.


Please add window shadow enums, 'CS_HREDRAW', 'CS_VREDRAW', 'CS_DROPSHADOW' and more...

Is your feature request related to a problem? Please describe.
To make the call easier.
Describe the additions or enhancements you'd like
To make the call easier.
public static void SetShadowStyle(this IntPtr handle)
{
    if (handle == default)
        throw new InvalidDataException(nameof(handle));

    var gclStyle = GetClassLongFlag.GCL_STYLE;
    var newLong = new IntPtr((uint)GetClassLong(handle, gclStyle) | CS_DROP_SHADOW);

    SetClassLong(handle, gclStyle, newLong);
}

Previous work
If work on these imports have been done, please provide links.

The window class styles are available in Vanara.PInvoke.User32.WindowClassStyles. Using the search bar, you can search for the enum value (e.g. CS_DROP_SHADOW) and find the file it is in. The directory structure will then indicate which NuGet package (assembly) contains the definition.

Sorry, I may not have searched carefully at the time, I did.

All the Vanara libraries use the spellings found in the header files of the Windows API. https://learn.microsoft.com/en-us/windows/win32/winmsg/window-class-styles


Problem decoding Greyhound Data upon Read

I'm trying to work with the data returned by the greyhound read query.  It looks like I'm losing precision somewhere along the line.  Having written my own code I had assumed my struggles were a bug of mine. However I just discovered your own sample python code ( https://github.com/hobu/greyhound/pull/35/files ) that appears to have the same issue. Playing with the settings in this file, I run it to produce this query:
http://[myipaddress]:8080/resource/[MyDemoResource]/read?bounds=[-13634057.2874,4552379.08325,-13634052.2874,4552384.08325]&compress=false
It returns 30813 points from a 5 meter cube of my data. The las file that's generated looks like this:

You can probably see 30-4 points within the bounds of this data in this image above. When I analyze the data itself, there are really only 6 unique values in each of the X, and Y coordinates.  In other words, each point visible in this image is actually about 880 superimposed vertices.
The part that I don't understand is that when I aim Potree at this same area, I see the full point density and it looks nothing like this. It's filled in, and almost solid in appearance until you really zoom in on it.
For what it's worth, this is the point breakdown per depth level (I only have data in 11-21):
11:1,12:2,13:15, 14:61, 15: 255, 16:966, 17:3534, 18:10399, 19:13031, 20:2514, 21:35
My schema:
{"baseDepth":7,"bounds":[-13635560,4545640,-6040,-13623480,4557720,6040],"boundsConforming":[-13635553.321355114,4546474.442458361,-346.13140000000004,-13623524.425497472,4556869.877397481,309.4769],"numPoints":1185748768,"offset":[-13629520,4551680,0],"reprojection":{"in":"","out":"EPSG:3857"},"scale":0.001,"schema":[{"name":"X","size":4,"type":"signed"},{"name":"Y","size":4,"type":"signed"},{"name":"Z","size":4,"type":"signed"},{"name":"Intensity","size":2,"type":"unsigned"},{"name":"ReturnNumber","size":1,"type":"unsigned"},
I feel like I'm missing something fundamental in trying to interpret the returned data. Do you have any ideas what I might be missing?
Thank you.

Can you reproduce this against something public so I can tweak the query around?  I have some ideas but no concrete suggestions without being able to fully reproduce.  For example with http://data.greyhound.io/resource/autzen.
Another thing that you might not be aware of you might like - instead of /read, you can use /static, which will statically render your query.  Query parameters will be forwarded.  For example: http://data.greyhound.io/resource/autzen/static?bounds=[634950,848860,637300,851210]&depthEnd=10.  This is nice for quick query testing.

Even though you didn't provide a solution, your answer was enough to inspire me to dig a little deeper.  So I've fixed my issue.  My sample data is mobile LiDAR that I indexed with a scale of .001.  I'm not sure if Greyhound defaults internally to serving data at .01 scale, but I went ahead and provided scale and offsets into my query. I also adjusted my x and y bounds parameters (as your documentation says I should do) like so:
x = (x - offset[0]) / scale[0]
y = (y - offset[1]) / scale[1]
(I'm not providing z in my bounds)
That fixed it.
What's curious is that your documentation specifically says "If scale and offset values are passed, and they are exactly equivalent to those present in the info query, then this results in a no-op transformation on the server since the data is already in the desired local coordinate space under-the-hood"
Yet, that's exactly what I did and it fixed my issue.
Thanks again.

👍 on resolving your query.  I'll close this since it seems to be working for you.  Feel free to reopen if you ever recreate your original query issue against something public so I can take a look with concrete data.


nyc merge for large number of files throws "Invalid string length" error

We run 20+ parallel runs of cypress that generate 20 different coverage${ID}-final.json stored in cypress/coverage
Expected Behavior
$(npm bin)/nyc merge cypress/coverage cypress/coverage/coverage-final.json should merge them into a single coverage file without issue.
Observed Behavior
$(npm bin)/nyc merge cypress/coverage cypress/coverage/coverage-final.json throws this error:
RangeError: Invalid string length

Likely issue
Might be happening due to https://github.com/nodejs/node/issues/35973 and thus lowering the number of runs to 14 works as the size of the json is lower than 500MB

I ran into the same issue. I fixed it by reducing the size of the JSON files. For example, for my use case, we didn't need code coverage for files in node_modules directory. Perhaps the same or something similar can apply to your case.

A big reason merge can hit this error quickly is that it stringifies pretty json with a lot of white space meaning huge json
https://github.com/istanbuljs/nyc/blob/main/lib/commands/merge.js#L44
Going from JSON.stringify(map, null, 2) to simple JSON.stringify(map) adds a lot of leeway.


additional instructions to go quick start and update to go landing page

Proposed changes
Added instructions for GOPATH to quick starts, updated Go link on languages landing page, added link to godocs on go.md

@spara Are you planning to revisit this, or would you like to close?

this work is planned to be revisited after @davidwrede's updates to the getting started guides

Can we close this and move it to pulumi-hugo?

not published


Add Optimization Of Recursive Style Use To Avoid Re-parsing All Of Format

Each level of fmtster formatting re-parses the format string. I will admit to not being an expert on whether the constexpr design of fmt::formatter<>::parse() causes much of this to be done at compile time, but I think it's possible that this will be less likely when Add Recursive Brace Support is done, so that formatting can more easily be done at run time. A method of passing the pre-parsed data (currently things like FmtsterBase::mStyleSetting, FmtsterBase::mTab, etc.) to avoid reparsing, if the values didn't change, would be helpful.

With the work being done in https://github.com/HarmanFOSS/fmtster/tree/enhancement/27-new-JSON-style_union, the approach to this is coming into focus.
First, since the actual configuration structure is effectively a large integer, and there is a fmtster::XXXStyleHelper associated with the configuration to provide the expanded strings (et al), the efficiency sought here can be realized by storing an already initialized XXXStyleHelper and quickly associating it with a matching fmtster::XXXStyle(.value) object.
An interesting approach would be to maintain a std::map<fmtster::internal::XXX_VALUE_T, fmtster::internal::XXXStyleHelper>.
It will be necessary to investigate whether indexing would use the XXX_VALUE_T directly as desired and provide custom comparitor if necessary.
To keep the size of this from growing too large, an additional std::list<fmtster::internal::XXX_VALUE_T> could also be maintained for the same entries as the map. This would be used to facilitate a LRU method of eviction to maintain the most recently used N XXXStyleHelper objects.
In the short term, this may be implemented with a single XXXStyleHelper maintained (for each serialization format). And even if the above approach is taken, it may be determined that maintaining a single object is as efficient as is necessary. Or, the actual count could be overridden by the user, perhaps by adding a #define XXXStyleHelperCacheCount 5 before including the <fmtster.h> header.

enhancement/19-lru is a working branch for this. The approach there uses an LRU cache design to re-use helper objects associated with style values/objects/unions.

Pull request: https://github.com/HarmanFOSS/fmtster/pull/32


Convert to lit-element and TypeScript

Depends on vaadin/component-mixins#39
Depends on vaadin/component-mixins#40
⚠ Breaking Changes
This should include two breaking changes:

make vaadin-radio-group vertical by default (#116)
use orientation instead of theme="vertical"

Mixins
The following mixins have to be implemented:

@vaadin/checked-state-mixin (vaadin/component-mixins#40)
@vaadin/slotted-label-mixin (vaadin/component-mixins#39)

The following mixins are ready and should be used:
vaadin-radio-button

@vaadin/active-state-mixin
@vaadin/control-state-mixin
@vaadin/disabled-state-mixin
@vaadin/focus-visible-mixin

vaadin-radio-group

@vaadin/orientation-mixin
@vaadin/keyboard-direction-mixin
@vaadin/roving-tabindex-mixin
@vaadin/slotted-items-mixin

Done by #146


Consistent Hash Partition size

Configure consistent hash Partition size
Description
The current Consistent hash DefaultPartitionN = 16.   To prepare for future growth this number needs to be >= the max cluster size for the life of a DB.

Make this a parameter within the config
Default this to a higher value to prepare for cluster growth i.e. 360

Success criteria (What criteria will consider this ticket closable?)
We can create a cluster with more that 16 nodes, and no more that the set partition size.
Conflicts
Changing the default partition size will invalidate any current  DB's slice->node mapping.  We should note this breaking change in the version upgrade notes

I'm trying to do this, but it breaks a bunch of tests when you make the change. Some tests are creating slices on nodes which don't own the slice after the change, but that doesn't seem to be the only source of issues, so I'm having to dig further.

I ran into the same thing when I change database to index along with changing the index name in the tests from d to i. It was a bit tedious to adjust all the tests so that they worked again.
I'm not sure about the other issue you're having.

@travisturner I'm not sure the test is entirely correct in its current state (although it is passing).
Do these lines look right to you? given the slices that are being set up above?
https://github.com/pilosa/pilosa/blob/master/client_test.go#L102


[Idea]: Minimum Index Sum of Two List

Idea Title
Minimum Index Sum of Two List
Idea Description
Given two arrays of strings list1 and list2, find the common strings with the least index sum.
A common string is a string that appeared in both list1 and list2.
A common string with the least index sum is a common string such that if it appeared at list1[i] and list2[j] then i + j should be the minimum value among all the other common strings.
Return all the common strings with the least index sum. Return the answer in any order.
Input:
list1 = ["Shogun","Tapioca Express","Burger King","KFC"],
list2 = ["Piatti","The Grill at Torrey Pines","Hungry Hunter Steakhouse","Shogun"]
Output: ["Shogun"]
Explanation: The only common string is "Shogun".
Potential Benefits

[ ] Improves user experience
[X] Enhances learning
[X] Adds a new feature
[ ] Improves code quality
[ ] Other

Implementation Suggestions (Optional)
No response

pls assign this to me .. i would like to contribute


Instructions do not provide intended result.

I followed the instructions provided and keep getting a message that the "Refresh has failed". The Power BI app is not able to load my data. The instructions in the doc suggest the tennant ID should have the hyphens removed, even though the dialog box in the app that asks for the tennant ID has dashes in the example format. I have tried both ways without success. Please advise.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 65732911-21e9-0902-51fa-0d3ae0caaa7a
Version Independent ID: 3682fff5-36fc-1d69-96ef-88f7f73225d5
Content: Enable Microsoft 365 usage analytics - Microsoft 365 admin
Content Source: microsoft-365/admin/usage-analytics/enable-usage-analytics.md
Service: o365-administration
GitHub Login: @efrene
Microsoft Alias: efrene

@shane-obrien Thank you for your feedback. This looks more like a technical issue as it working on multiple other tenants. I'd recommend you to contact Microsoft via the need help section so that a Support engineer will be able to assist you.
Hope this helps!
Thanks
Sri

@msbemba Thank you very much for the contribution and sharing this explanation. @shane-obrien Hope this comment is helpful for you. Considering the nature of the issue, as advised, please open a service ticket in your tenant and follow with them for the resolution. We proceed to close it as here we are limited to documentation fixes and improvements only.
We will be very happy if you can share the outcome or resolution with us if you see documentation update is required. We will certainly update this documentation with that valuable information. Thanks again, for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


Speed up bazaar by simulating bzr root with bash

The PS1 line took forever (~1s) to display in bzr branches, because bzr is slow.
I increased the speed by eliminating the call to bzr, and replacing it with a modified bash script which I found here:
http://unix.stackexchange.com/questions/6463/find-searching-in-parent-directories-instead-of-subdirectories

This version should work now.
The bzr part should be a lot quicker too.


Some strings are displayed as binary

Hey! I have a problem displaying strings, some are displayed as binary.
I have definitions like:
field(:owner, Ch, type: "FixedString(100)")

and it displays something like <<49, 50, 67, 90, 68, 69, 107, 57, 114, 78, 98, 88, 106, 85, 74, 77, 77, 55, 97, 72, 111, 85, 57, 112, 99, 54, 57, 84, 104, 68, 50, 51, 121, 83, 76, 116, 119, 66, 102, 105, 119, 56, 54, 49, 0, 0, 0, 0, 0, 0>>
While some other strings are displayed properly.
Could you please tell how to fix this problem?

@arrowcircle
The binary is not displayed as text because it contains null bytes padding in the end to make it 100 bytes.


Update test_mdgraph for gfx90a and gfx10

@Slimakanzer IIRC #809 enables 1x1 fused asm kernel for gfx90a. And #772 enables fused Winograd for gfx10. Please update tests as necessary and check locally. Thanks.
/cc @Kirpich30000
Originally posted by @atamazov in https://github.com/ROCmSoftwarePlatform/MIOpen/issues/813#issuecomment-808487766

via #813