"Data size too large" error with byte-stream-split encoded data since Arrow 12.0.0

Describe the bug, including details regarding any error messages, version, and platform.
Arrow 12.0.0 has a regression where it can crash when reading byte-stream split encoded data written by itself or older versions of Arrow:
import numpy as np
import pyarrow as pa
import pyarrow.parquet as pq

x = pa.array(np.linspace(0.0, 1.0, 1_000_000), type=pa.float32())
table = pa.Table.from_arrays([x], names=['x'])
pq.write_table(table, 'data.parquet', use_dictionary=False, use_byte_stream_split=True)

table = pq.read_table('data.parquet')
print(table)

This crashes with:
Traceback (most recent call last):
  File "/home/.../write_read_data.py", line 9, in <module>
	table = pq.read_table('data.parquet')
			^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adam/.local/share/virtualenvs/arrow12-nieuEBn0/lib64/python3.11/site-packages/pyarrow/parquet/core.py", line 2986, in read_table
	return dataset.read(columns=columns, use_threads=use_threads,
		   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/adam/.local/share/virtualenvs/arrow12-nieuEBn0/lib64/python3.11/site-packages/pyarrow/parquet/core.py", line 2614, in read
	table = self._dataset.to_table(
			^^^^^^^^^^^^^^^^^^^^^^^
  File "pyarrow/_dataset.pyx", line 546, in pyarrow._dataset.Dataset.to_table
  File "pyarrow/_dataset.pyx", line 3449, in pyarrow._dataset.Scanner.to_table
  File "pyarrow/error.pxi", line 144, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 115, in pyarrow.lib.check_status
OSError: Data size too large for number of values (padding in byte stream split data page?)

But the above code works fine with pyarrow 11.0.0 and and 10.0.1.
It appears that #34140 caused this regression. I tested building pyarrow on the current main branch (commit 42d42b1194d8a672e13dac10a8102573f787f70d) and could reproduce the error, but it was fixed after I reverted the merge of that PR (commit c31fb46544b9c8372e799138bad9223162169473).
Component(s)
Parquet

If I run this in a debugger, I can see ByteStreamSplitDecoder<DType>::SetData is called 4 times. For the first 3 times, num_values is 262,144 and len is 1,048,576, then on the 4th time len is again 1,048,576 but num_values is only 213,568 (for a total of 1,000,000) values.

I'll try to reproduce this problem. In ByteStreamSplitDecoder::SetData, it expects that num_values is greater or equals to the num of values in ByteStreamSplit, so it expects num_values * static_cast<int64_t>(sizeof(T)) >= len. It would be == if there is no null value in the page

Seems that using ParquetTableReader to just read that file is ok, and size as 100_000 or 200_000 is ok, but 300_000 would failed.  I guess ParquetFileFormat with a huge page size is not ok.

I've found out the reason, I think it's not related to ByteStreamSplit, it's caused by page.size(), page reuse one buffer when ReadNextPage, and it size might be greater than expect size( page.uncompressed_size() ). However, ByteStreamSplit could expose that problem because it checks the boundary


Disk space requirements on this page are 150GB, are 300GB in actual setup guide

From the 2004 lab setup guide:

The Windows and Office Deployment Lab Kit is a testing environment based on Windows Server 2019 and Windows 10, Version 2004. It must be imported and created onto a Hyper-V server installed on Windows Server 2012 R2 or Windows Server 2016 or Windows 10. It is recommended to use a Windows Server OS and should be fully updated. The Hyper-V Host must preferably meet the following specifications:

Hyper-V role installed
Administrative rights on the host
300 gigabytes of free disk space or more
High-throughput disk subsystem
Up to 32 gigabytes of memory
Preferably a High-end processor for faster processing
An External virtual switch in Hyper-V connecting to the external adapter of the host machine for internet connectivity, example External 2

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 21195f1c-0f99-502b-c6e7-a87d35237a81
Version Independent ID: 1848a7e1-4919-8e28-27ea-cfdbef125609
Content: Windows and Office deployment lab kit - Microsoft 365 Enterprise
Content Source: microsoft-365/enterprise/modern-desktop-deployment-and-management-lab.md
Service: o365-solutions
GitHub Login: @JoeDavies-MSFT
Microsoft Alias: josephd

@JeremyTBradshaw I have asked the owner of the Windows and Office deployment lab kit download to comment on this issue. Thanks for letting us know.

@JeremyTBradshaw Thank you for your feedback. While 150 GB of free disk space is required, 300 GB is recommended as per the prerequisite. I have made changes to the document via PR#https://github.com/MicrosoftDocs/microsoft-365-docs/pull/3531
CC: @JoeDavies-MSFT
Thanks
Sri

Closing since it's address in @msbemba 's PR.  Thanks!


Improve error messaging for derive_param_tte()

What happened?
Any reason why derive_param_tte() argument source_datasets doesn't like underscores?
Session Information
Using {admiral} 1.1.1
Reproducible Example
This code works (assuming I have a valid adsl_death dataset and I've set rand_censor):
death <- event_source(
  dataset_name = "adsl",
  filter = DTHFL == "Y",
  date = DTHDT
)

test_os <- derive_param_tte(
  dataset_adsl = adsl_death,
  start_date = RANDDT,
  event_conditions = list(death),
  censor_conditions = list(rand_censor),
  source_datasets = list(adsl = adsl_death),
  set_values_to = exprs(
    PARAMCD = "OS"
  )
)

This code errors:
death <- event_source(
  dataset_name = "adsl_death",
  filter = DTHFL == "Y",
  date = DTHDT
)

test_os <- derive_param_tte(
  dataset_adsl = adsl_death,
  start_date = RANDDT,
  event_conditions = list(death),
  censor_conditions = list(rand_censor),
  source_datasets = list(adsl_death = adsl_death),
  set_values_to = exprs(
    PARAMCD = "OS"
  )
)

The error given is:
Error in `fun(..., .envir = .envir)`:
! Could not evaluate cli `{}` expression: `source_names`.
Caused by error in `eval(expr, envir = envir)`:
! object 'source_names' not found
Type .Last.error to see the more details.

@rossfarrugia , I think the issue is not related to underscores but an issue in the error message. The correct error message should be:
Error in `derive_param_tte()`:
! The dataset names must be included in the list specified for the `source_datasets` argument.
ℹ Following names were provided by `source_datasets`: "adsl_death"
ℹ But, `censor_conditions[[1]]$dataset_name = adsl`

I assume in your example rand_censor is referring to adsl instead of adsl_death. Could you check?

Yes! That's it @bundfussr - great spot. It was an error on my side but the error message could definitely be improved, as I'd have spent hours trying to figure that out.

yes, works now by giving better error message when re-running the same code 👍

Yes! That's it @bundfussr - great spot. It was an error on my side but the error message could definitely be improved, as I'd have spent hours trying to figure that out.

hours!!  oh no!!


Uninstall 05 | DCOS-18197: add delete status while service is deleting and disable actions

Add delete status while service is deleting and disable actions.
Closes DCOS-18197
Checklist

[ ] Did you add a JIRA issue in a commit message or as part of the branch name?
[ ] Did you add new unit tests?
[ ] Did you add new integration tests?
[ ] If this is a regression, did you write a test to catch this in the future?

@bstavroulakis how does the PR relate to https://github.com/dcos/dcos-ui/pull/2440?

@orlandohohmeier I closed #2440, since it is broken down into 3 now #2457, #2458, #2459

Putting this one on hold till https://github.com/dcos/dcos-ui/pull/2459/files#discussion_r144799299 is addressed.

@ahoskins Yeap, I agree


Feature request: Add the ability to change/unset the keyboard bindings

The current implementations lacks a settings window with which one could change or unset the keyboard bindings to switch the windows on a workspace. Since the keyboard bindings are already stored in the gsettings the only thing which is missing is a UI to alter the settings.
If this is something what should be added, I am happy to help implementing it.

Good idea, feel free to start the implementation if you like

Will do, but I do not know when I have time.


fix(packages/app): add lang attribute to the page element

Part of #2507

Description of what you did:

My PR is a:

[ ] 💥 Breaking change
[x] 🐛 Bug fix
[ ] 💅 Enhancement
[ ] 🚀 New feature

Please confirm that your PR fulfills these requirements

[x] Multiple commits are squashed into one commit.
[x] The commit message follows Conventional Commits, which allows us to autogenerate release notes; e.g. fix(plugins/plugin-k8s): fixed annoying bugs
[x] All npm dependencies are pinned.

i think this is ok for now? but i wonder if it needs to be the user's locale?

can you add a comment to the issue specifying which of the listed items is resolved?


Twine issue uploading

Curfrently an issue uploading using Twine, the credentials do not seem to be populated.

Issue caused by secrets being stored in wrong location, issue now resolved.


Unable to add dynamic content when entering in an expression.

Describe the Bug with repro steps

Add an action. ex: "sharepoint get file content using path"
In any parameter, click 'insert expression' (fx)
click 'dynamic content'
try clicking any option from a previous step, value does not get entered into the expression box.

What type of Logic App Is this happening in?
Consumption (Portal)
Which operating system are you using?
Windows
Are you using new designer or old designer
New Designer
Did you refer to the TSG before filing this issue? https://aka.ms/lauxtsg
No
Workflow JSON
No response
Screenshots or Videos

Browser
Firefox
Additional context
Applies to all parameters in every block I have tried.

We'll be hot fixing this today. We got a fix in for this before the release but somehow it didn't make it into the release branch. Thank you for reporting this!


Add mixedbread-ai/mxbai-rerank-base-v1

Hi,
Please add mxbai-rerank-base-v1.
Thanks!

Ah, sorry about the duplicate! Must've missed that


solarized-everything-css

Hi there,
As a fellow GitHub CSS themer, I thought I'd offer this:
https://github.com/alphapapa/solarized-everything-css
I set up a Makefile that uses Stylus to build the stylesheets, and it makes re-theming so much easier than writing raw CSS.  We might even be able to collaborate on the github.styl file, and you could use a dark.styl file to change the colors and fonts.  :)

Hi i will get back to you soon, I'm quite busy this week

Sorry closing this.


feat: incremental annotation processor

updated the gradle file as well for android studio 4.1

Thanks for this PR!
Looks added meta-data for gradle incremental annotation process.
However, in this doc https://docs.gradle.org/5.0/userguide/java_plugin.html#sec:incremental_annotation_processing it says

They must generate their files using the Filer API.
If they use Filer#createResource, Gradle will recompile all source files. See gradle/issues/4702

If jenny.outputDirectory is not passed, Jenny will use the Filer#createResource API, which according to the rule it won't be incremental.
Also, If jenny.outputDirectory is not passed, Jenny will not use the Filer API, which disobeys the first rule.
I'm thinking if this could be a problem.

Thanks for this PR!
Looks added meta-data for gradle incremental annotation process.
However, in this doc https://docs.gradle.org/5.0/userguide/java_plugin.html#sec:incremental_annotation_processing it says
1. They must generate their files using the Filer API.

2. If they use Filer#createResource, Gradle will recompile all source files. See gradle/issues/4702

If jenny.outputDirectory is not passed, Jenny will use the Filer#createResource API, which according to the rule it won't be incremental.
Also, If jenny.outputDirectory is not passed, Jenny will not use the Filer API, which disobeys the first rule.
I'm thinking if this could be a problem.

https://github.com/CraZyLegenD/FreshAdapter
You can check out my implementation it works perfectly without recompilation

But my thinking is that this repo is gonna fall under isolating instead of aggregating

Thanks! I'll try it.

Thanks! I'll try it.

But my thinking is that this repo is gonna fall under isolating instead of aggregating

Yes, it should, but there is a "jenny_fusion_proxies.h" file that needs to be worked in the aggregate mode. Just because we need to know all the proxy files.

I tested the pull request with isolating and it recompiles everything, with aggregating as it currently is it doesn't.
Of course there are corner cases when recompilation happens and that's usually when static member variable is changed or package moved etc... which are not under your control.

I tested the pull request with isolating and it recompiles everything, with aggregating as it currently is it doesn't.

Yes, I tested it as well, and the compiler says:

Full recompilation is required because the generated resource 'jenny/proxy/jnihelper.h in SOURCE_OUTPUT' must have exactly one originating element, but had 0. Analysis took 0.0 secs.

So not only does jenny_fusion_header.h blocks the way, so does the jnihelper.h too.
Now as I tested aggregating mode works pretty well! Gradle is smart enough to figure which files are used to the annotation processor and if any of them is changed, all of them will be recompiled, but still, It's an incremental compilation (not all code are recompiled)
Great work! Thanks, man!
And also, sorry for my being busy these days.


Tabs are not grouped in Google Chrome v93.0.4577.82

I'm experiencing the same issue using Chrome Version 95.0.4638.69 on MacOS 11.6.1. Tabs are no longer joining their assigned groups automatically but can be moved manually. However, when the manually moved tab is selected again, it defaults to its previous behavior (separation and duplicate groups).
An update would be great since I love this plugin so freaking much. @jdhayford please let me know if you need beta testers for a potential fix.

This is also an issue in Edge 95

Does anyone here have a consistent repro process?  Normally, for me my extension works properly for a while and then fails when I have many tabs but there isn't one thing that sticks out.
I setup the dev env for the extension and will be watching logs when this reproduces - if I can reproduce it, I'll fix it.

I am seeing this as well. But for me, the tabs still get correctly grouped in the first Chrome window that I have opened. It's only in subsequent Chrome windows that the tabs don't group. I normally get around this by trying to have only one Chrome window open, or treat one as my "primary" chrome window (where I want tabs grouped) and know that all other windows won't have grouping.
There hasn't been any activity on this plugin in 6 months though, so I'll look for another plugin.

@jdhayford I'm seeing this issue as well. I'm also experiencing issues with the tabs not being in the correct order as well. Possibly related. Will try to take a look if I get time

It looks like #16 may have fixed this issue. I forked the repo and ran through the instructions in the readme to install it from a locally-built version and it appears to be working for me now from a quick test.
Hopefully all it needs is a release from @jdhayford to the chrome store

It looks like #16 may have fixed this issue. I forked the repo and ran through the instructions in the readme to install it from a locally-built version and it appears to be working for me now from a quick test.
Hopefully all it needs is a release from @jdhayford to the chrome store

Thanks for the comment. I will perform a new release this week 👍🏼

@jdhayford
Thanks for the great release!


[Discussion] I have an error when I do train.py

When I do train.py, I have an error ValueError: attempted relative import beyond top-level package.
Directory like "audio" is upper hierarchy from "commands" directory.
How can I do train.py ?

See ValueError: attempted relative import beyond top-level package


We need a fast path for open_mfdataset

It would be great to have a "fast path" option for open_mfdataset, in which all alignment / coordinate checking is bypassed. This would be used in cases where the user knows that many netCDF files all share the same coordinates (e.g. model output, satellite records from the same product, etc.). The coordinates would just be taken from the first file, and only the data variables would be read from all subsequent files. The only checking would be that the data variables have the correct shape.
Implementing this would require some refactoring. @jbusecke mentioned that he had developed a solution for this (related to #1704), so maybe he could be the one to add this feature to xarray.
This is also related to #1385.

Would these two options be necessarily mutually exclusive?
I think parallelizing the read in sounds amazing.
But isnt there some merit in skipping some of the checks all together, if the user is sure about the structure of the data contained in the many files?
I am often working with the aforementioned type of data (many files either contain a new timestep or a different variable, but most of the dimensions/coordinates are the same).
In some cases I am finding that reading the data "lazily" consumes a significant amount of the time in my workflow. I am unsure how hard this would be to achieve, and perhaps it is not worth it after all.
Just putting out a few ideas, while I wait for my xr.open_mfdataset to finish :-)

@jbusecke - No. These options are not mutually exclusive. The parallel open is, in my opinion, the lowest hanging fruit so that's why I started there. There are other improvements that we can tackle incrementally.

Awesome, thanks for the clarification.
I just looked at #1981 and it seems indeed very elegant (in fact I just now used this approach to parallelize printing of movie frames!) Thanks for that!

I am currently motivated to fix this.
Over in @rabernat mentioned

allowing the user to pass join='exact' via open_mfdataset. A related optimization would be to allow the user to pass coords='minimal' (or other concat coords options) via open_mfdataset.

@dcherian I'm sorry, I'm very interested in this but after reading the issues I'm still not clear on what's being proposed:
What exactly is the bottleneck? Is it reading the coords from all the files? Is it loading the coord values into memory? Is it performing the alignment checks on those coords once they're in memory? Is it performing alignment checks on the dimensions? Is this suggestion relevant to datasets that don't have any coords?
Which of these steps would a join='exact' option omit?

A related optimization would be to allow the user to pass coords='minimal' (or other concat coords options) via open_mfdataset.

But this is already an option to open_mfdataset?

The original issue of this thread is that you sometimes might want to disable alignment checks for coordinates other than the concat_dim and only check for same dimensions and dimension shapes.
When you xr.merge with join='exact', it still checks for alignment (see https://github.com/pydata/xarray/pull/1330#issuecomment-302711852), but does not join the coordinates if they are not aligned. This behavior (not joining) is also included in what @rabernat envisioned here, but his suggestion goes beyond that: you don't even check for alignment, because it is expensive.
So join='exact' does not fix this issue here. In fact, possibly adding this option to open_mfdataset could be another issue.

So I think it is quite important to consider this issue together with #2697. An xml specification called NCML already exists which tells software how to put together multiple netCDF files into a single virtual netcdf. We should leverage this existing spec as much as possible.
A realistic use case for me is that I have, say 1000 files of high-res model output, each with large coordinate variables, all generated from the same model run. If we want to  for for which we know a priori that certain coordinates (dimension coordinates or otherwise) are identical, we could save a lot of disk reads (the slow part of open_mfdataset) by never reading those coordinates at all. Enabling this would require a pretty low-level change in xarray. For example, we couldn't even rely on open_dataset in its current form to open files, because open_dataset eagerly loads all dimension coordinates into indexes. One way forward might be to create a new Store class.
For a catalog of tricks I use to optimize opening these sorts of big, complex, multi-file datasets (e.g. CMIP), check out
https://github.com/pangeo-data/esgf2xarray/blob/master/esgf2zarr/aggregate.py

One common use-case is files with large numbers of concat_dim-invariant non-dimensional co-ordinates.  This is easy to speed up by dropping those variables from all but the first file.
e.g.
https://github.com/pangeo-data/esgf2xarray/blob/6a5e4df0d329c2f23b403cbfbb65f0f1dfa98d52/esgf2zarr/aggregate.py#L107-L110
    # keep only coordinates from first ensemble member to simplify merge
    first = member_dsets_aligned[0]
    rest = [mds.reset_coords(drop=True) for mds in member_dsets_aligned[1:]]
    objs_to_concat = [first] + rest

Similarly https://github.com/NCAR/intake-esm/blob/e86a8e8a80ce0fd4198665dbef3ba46af264b5ea/intake_esm/aggregate.py#L53-L57
def merge_vars_two_datasets(ds1, ds2):
    """
    Merge two datasets, dropping all variables from
    second dataset that already exist in the first dataset's coordinates.
    """

See also #2039 (second code block)
One way to do this might be to add a master_file kwarg to open_mfdataset. This would imply coords='minimal', join='exact' (I think; prealigned=True in some other proposals) and would drop non-dimensional coordinates from all but the first file and then call concat.
As bonus it would assign attributes from the master_file to the merged dataset (for which I think there are open issues) : this functionality exists in netCDF4.MFDataset so that's a plus.

Is this issue really closed?!?
🎉🎂🏆🥇

YES!
(well almost)
The PR lets you skip compatibility checks.
The magic spell is xr.open_mfdataset(..., data_vars="minimal", coords="minimal", compat="override")
You can skip index comparison by adding join="override".
Whats left is extremely large indexes and lazy index / coordinate loading but we have #2039 open for that. I will rename that issue.
If you have time, can you test it out?

This is big if true!
But surely to close an issue raised by complaints about speed, we should really have some new asv speed tests?

=) @TomNicholas PRs welcome!

PS @rabernat
%%time
ds = xr.open_mfdataset("/glade/p/cesm/community/ASD-HIGH-RES-CESM1/hybrid_v5_rel04_BC5_ne120_t12_pop62/ocn/proc/tseries/monthly/*.nc",
                        parallel=True, coords="minimal", data_vars="minimal", compat='override')

This completes in 40 seconds with 10 workers on cheyenne.

Wooooow. Thanks. Ill have to give this a whirl soon.

Let's close this since there is an opt-in mostly-fast path. I've added an item to #4648 to cover adding an asv benchmark for mfdataset.

Let's close this since there is an opt-in mostly-fast path. I've added an item to #4648 to cover adding an asv benchmark for mfdataset.

PS @rabernat
%%time
ds = xr.open_mfdataset("/glade/p/cesm/community/ASD-HIGH-RES-CESM1/hybrid_v5_rel04_BC5_ne120_t12_pop62/ocn/proc/tseries/monthly/*.nc",
                        parallel=True, coords="minimal", data_vars="minimal", compat='override')

This completes in 40 seconds with 10 workers on cheyenne.

@dcherian,  thanks for your solution. In my experience with 34013 NetCDF files, I could open 117 Gib in 13min 14s. Can I decrease this time again?

PS @rabernat
%%time
ds = xr.open_mfdataset("/glade/p/cesm/community/ASD-HIGH-RES-CESM1/hybrid_v5_rel04_BC5_ne120_t12_pop62/ocn/proc/tseries/monthly/*.nc",
                        parallel=True, coords="minimal", data_vars="minimal", compat='override')

This completes in 40 seconds with 10 workers on cheyenne.

@dcherian,  thanks for your solution. In my experience with 34013 NetCDF files, I could open 117 Gib in 13min 14s. Can I decrease this time again?

That's 34k 3MB files! I suggest combining to 1k 100MB files, that would work a lot better.

That's 34k 3MB files! I suggest combining to 1k 100MB files, that would work a lot better.

combin
@dcherian, thanks for your reply. I am a beginner in xarray. How can I do it?  Furthermore, when I want to extract a subset from this dataset such as:

subset = ds.where((ds.lat > 25) & (ds.lat < 40) & (ds.lon > 44) & (ds.lon < 63), drop=True)
It returns this warning:
`PerformanceWarning: Slicing is producing a large chunk. To accept the large
chunk and silence this warning, set the option
>>> with dask.config.set(**{'array.slicing.split_large_chunks': False}):
...     array[indexer]
To avoid creating the large chunks, set the option
>>> with dask.config.set(**{'array.slicing.split_large_chunks': True}):
...     array[indexer]
value = value[(slice(None),) * axis + (subkey,)]`

combin
@dcherian, thanks for your reply. I am a beginner in xarray. How can I do it?  Furthermore, when I want to extract a subset from this dataset such as:

subset = ds.where((ds.lat > 25) & (ds.lat < 40) & (ds.lon > 44) & (ds.lon < 63), drop=True)
It returns this warning:
`PerformanceWarning: Slicing is producing a large chunk. To accept the large
chunk and silence this warning, set the option
>>> with dask.config.set(**{'array.slicing.split_large_chunks': False}):
...     array[indexer]
To avoid creating the large chunks, set the option
>>> with dask.config.set(**{'array.slicing.split_large_chunks': True}):
...     array[indexer]
value = value[(slice(None),) * axis + (subkey,)]`


Fix datetime encoding precision loss regression for units requiring floating point values

This PR proposes a fix to #8271.  I think the basic issue is that the only time we need to update the needed_units is if the data_delta does not evenly divide the ref_delta.  If it does evenly divide it--as it does in the example in #8271--and we try to update the needed_units solely according to the value of the ref_delta, we run the risk of resetting them to something that would be coarser than the data requires.  If it does not evenly divide it, we are safe to reset the needed_units because they will be guaranteed to be finer-grained than the data requires.
I modified test_roundtrip_float_times to reflect the example given by @larsbuntemeyer in #8271.  @kmuehlbauer let me know if this fix makes sense to you.

[x] Closes #8271
[x] Tests added
[x] User visible changes (including notable bug fixes) are documented in whats-new.rst

@spencerkclark Oh wow, yes, the tests didn't catch this. Your fix makes perfect sense.


fix: allow disabling throttling in project monitors

fix #689
PR fixes the types and allows throttling to be disabled via

CLI - --no-throttling
Synthetics Config File - monitor.throttling: false
Monitor API - monitor.use({ throttling: false })

Needs Kibana fix to work End to End - https://github.com/elastic/kibana/issues/148344

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-01-03T19:30:11.332+0000

Duration: 15 min 9 sec

Test stats :test_tube:

Test
Results

Failed
0

Passed
206

Skipped
2

Total
208

:robot: GitHub comments
Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

@vigneshshanmugam with https://github.com/elastic/kibana/issues/148344 not in 8.6, does it matter than this is merged and will be included in beta.40 which will be released before 8.7?  (or does it just fail silently and not disable throttling with earlier versions of Kibana?)


cannot get the example to run

I am reporting a bug or issue
cannot get the example to run
Expected behavior:
app starts sketch
Observed behavior:
react-sketchapp/examples/basic-setup on master via ⬢ v7.7.1
➔ npm run render

><EMAIL_ADDRESS>render /Users/adamclark/repos/react-sketchapp/examples/basic-setup
> skpm build --watch --run

🖨  Copied src/manifest.json in 9ms
Executing post-build scripts
⛏  Built ./my-command.js in 1407ms
events.js:163
    throw er; // Unhandled 'error' event
    ^

Error: spawn /Applications/Sketch.app/Contents/Resources/sketchtool/bin/sketchtool ENOENT
  at exports._errnoException (util.js:1029:11)
  at Process.ChildProcess._handle.onexit (internal/child_process.js:193:32)
  at onErrorNT (internal/child_process.js:367:16)
  at _combinedTickCallback (internal/process/next_tick.js:80:11)
  at process._tickCallback (internal/process/next_tick.js:104:9)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>render: `skpm build --watch --run`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>render script 'skpm build --watch --run'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the basic-setup package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     skpm build --watch --run
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs basic-setup
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls basic-setup
npm ERR! There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:

How to reproduce:
using sketch 43.2, node 7.7.1, and npm 4.5
git clone https://github.com/airbnb/react-sketchapp.git
cd react-sketchapp/examples/basic-setup && npm install

npm run render

Sketch version:
sketch 43.2
Please attach screenshots, a zip file of your project, and/or a link to your github project
just trying to run the example

🤔 sounds like Sketch might be installed in a different location - can you try the solution in #27 and report back?

nice this looks promising. will let you know and thank you for the quick response!

the config was throwing js-yaml errors but moving sketch from /downloads to /applications was all it took 👌

ahah, that makes sense. perhaps skpm could check for the presence of sketchtool before trying to run it 🤔

🤔 and throw an error referencing location of sketch not being in /applications. not sure if that covers all the basis that error but for #27 and my issue it would've been a quick fix!


Adds documentation and example conf files to support SSL.

Inadvertently added some local changes to this PR. Will close and resubmit from a rebased master.


Change AlertCondition.severity to optional, allow any string

Summary

Updates AlertCondition.severity to be an optional field, and allow any string value

Description
The concept of severity is likely to be varied across both AlertConditions and NotificationTargets.  The current implementation seems to act as a hint to the NotificationTarget about what type of notification should be sent (ticket or page), but it's likely that teams and services will have their own custom definitions of what severity means for a given alert breach or incident.
For example, a team might using a numeric scaling of incident severity (Sev0, Sev1, Sev2) etc. with each severity level determining the course of notification action (Sev0 - page immediately and escalate in 5 min if unacked, Sev1 - page immediately and repeat every 30min unacked, etc.)
By decoupling the logic between AlertCondition severity and the type of Notification sent, the spec can be flexible enough to adapt to a wider variety of implementations and expectations.
Other potential considerations would be moving the severity field to AlertCondition.metadata or potentially even AlertCondition.metadata.labels

@weyert  Would love your feedback on this update!

Sounds fine to me, but I can imagine it could still be a required field? If it's required they have define a severity level (that could be none) and consider the severity?

Sounds fine to me, but I can imagine it could still be a required field? If it's required they have define a severity level (that could be none) and consider the severity?
It seems a little odd to support none on a required field, at that point is it really required?
I think I'd prefer to keep it optional, some common alerting platforms (ex. Grafana) don't have a built-in concept of severity and it feels odd to force it.  I'll ask around the Slack as well to see if other folks are interested in chiming in!

I think most teams would have a concept of severity, and it certainly feels like something that should be part of best practice, because if everything is urgent then nothing has priority.
To @nobl9-mikec's point about Grafana, there could easily be a plugin or similar for Oslo that created the Grafana dashboards/alerts and just used different webhooks/alert configurations within Grafana depending on the value of this variable.
Leaving it as free-form text makes sense, as where some orgs use Sev01 etc, others use P1, and others again High or Urgent, but I'd definitely be tempted to make it required.

Awesome, thank you for the additional feedback @proffalken!
Returning severity to required per based on you and @weyert's feedback


Shuttle drone improvements

Made grid spawning a generic thing and made mining shuttle use it.
Same with drone console.
Code still kinda flakey but better.
Disable escape pods on debug + mapinit tests.

This at least helps for maps with salvage expedition consoles.
:cl:

add: Mining shuttle will spawn roundstart.

does this make mining shuttle gridfill airlock in my pr useless

does this make mining shuttle gridfill airlock in my pr useless

uhhhhh yeah which pr is it, my understanding is emisse doesn't want mining mapped docked coz there isn't space on maps for it.

:rocket:


Tests are failing in yggdrasil installation

check this run https://github.com/threefoldtech/tfgrid-sdk-ts/actions/runs/6680221000/job/18153127199

Update:
Still investigating the issue and trying to find what's causing the issue, tried to manually install yggdrasil on multiple vms deployed from the grid but it didn't work and it crashed the vms since the vms already had yggdrasil installed and configured. Currently trying to manually install it on local ubuntu image using docker but i'm still facing some issues with the docker images.

Verified,
The test workflow ran successfully today, and the yggdrasil was installed with no errors. The yggdrasil was specified to use the 0.4.7 version.
https://github.com/threefoldtech/tfgrid-sdk-ts/actions/runs/6700526217/job/18206601169


Support links w/o going through the search page

I might not understand exactly how your app works, so just set me straight if I'm way off base here...
Because there's a pretty straight-forward conversion from open.spotify.com URLs to match.audio URLs, I thought I'd be able to infer the match.audio URL. This would be super handy because a chat bot could detect an open.spotify.com URL and respond with the inferred match.audio URL. All of that could happen without hitting the match.audio server. So there'd be no need for a API endpoint like discussed in #11.
This doesn't work at the moment unless the spotify URL has been searched for already. So there appears to be some server-side work that needs to happen before a match.audio URL is useful. Could this server side work lazily the first time a match.audio URL is hit?
I'd be happy to take a crack at this if it's not a batshit crazy idea.
Thanks for the project, by the way. I'm surprised how well you got the matching to work... :+1:

It should actually work lazily, if it doesn't it's a bug. I'll have to investigate later.

Going to https://match.audio/spotify/track/3PnEn4pp5hZ7KovNzLGP3o, for instance, gets me this:

This got nuked as part of the Koa refactor, I needed to simplify things at the time and intended to add it back. I guess I forgot :D


Add import for collections

On line number 478 of this file, deque was used from the collections module but there was no import and thus it was showing an error. So, the import was added.

Can you please mention if this is a genuine pull request?


New-CrescendoCommand needs better doc

Happened to see below question. Its easy to get in trouble when there is not enough examples on how to use New-CrescendoCommand in a script.
https://forums.powershell.org/t/help-with-crescendo/18359/11

Thanks you for the link @kvprasoon -- I'm working on improved documentation for the release, and a Getting Started guide.


Adjust position of custom fee and suppress Sentry error

Try out this version of Leather — download extension builds.

This PR:

adjust the position of the slot on the custom fee field
rolls back a change we made that introduced this sentry error

Pretty sure this doesn't suppress the error.

The error is on L66 and because event.currentTarget is undefined. In what situations might it be undefined, and how can we catch that error?

I'm not familiar with the business logic here but since we made updates and changed this, the sentry error gets triggered every time you type in this field:
https://github.com/leather-wallet/extension/assets/2938440/e498facb-aa47-4e77-9884-8b3e103760db
It could be there is something broken we need to fix but it's not something new that's broken, just this warning about it

Thanks for sharing demo video. Strange. Why would changing the event to non-async change the event.currentTarget value 🤔
Either way if errors gone 👍🏼


clair: enable tests

clair: enable tests
Tests logs at: https://termbin.com/leb9

Have you tried the tests on darwin?


Changelog after v1.2.0

I'm trying a new thing. Previously, I created an issue like this to track changes for the upcoming version. By making a PR against the readme, we remove any need to do any markdown cleanup with the release notes, and we rely on git to keep a history of changes to this changelog.
If we have any TODOs for upcoming releases, they can still be added as task list items to this PR thread. Only completed things will go in the changelog.

[ ] Add --system config commands, elevate install config to system scope when root/packaged #1177 (@javabrett)

👍

Closing this, the changelog script in #1281 handled this like a champ.

Woo-hoo! 🙌


Add known issue for chocolateyGUI

Description Of Changes

Added a known issue for ChocolateyGUI related to chocolatey/choco#2886

Motivation and Context
The bug can't be trivially fixed and may some time to resolve, so we should document the known issue for the time being.
Testing

[x] I have previewed these changes using the Docker Container or another method before submitting this pull request.

Change Types Made

[ ] Minor documentation fix (typos etc.).
[x] Major documentation change (refactoring, reformatting or adding documentation to existing page).
[ ] New documentation page added.
[ ] The change I have made should have a video added, and I have raised an issue for this.

Issue #

Change Checklist

[ ] Requires a change to menu structure (top or left hand side)/
[ ] Menu structure has been updated

Related Issue

chocolatey/choco#2886

@vexx32 if you can rebase this, I'll merge it.

Sorted! Sorry about that, thought I updated my local main branch, but apparently not.


Postpone takes original date into account instead of current date

Describe the bug
When postponing a todo, it should calculate the new date based on the following formula: new date = current date + pattern. It seems that currently the new date is calculated based on the original date (e.g., due date) rather than today.
Expected behavior
Data should be calculated based of the current date.

Tests failed - update required


Raidboss: Timelines not starting on zone seal lines

Running the beta XIV plugin and V24.0 of cactbot. In two separate instances, the zone seal lines failed to start the timeline. (The timeline module was indeed running, since later syncs correctly matched.)
[20:57:56.000]   ChatLog 00:0839::The Chancel will be sealed off in 15 seconds!
2000.0 "--sync--" sync / 00:0839::The Chancel will be sealed off/ window 2000,0

Not sure what's causing this, but I'll try to find more information.

One thing I did notice is that make_timeline sets things to be window 0,1 for engage lines, and I think that's incorrect.  It needs to be window 1,0, I think?

You know the internals better than I do, but my intuition is that it's correct as it is? The timeline start point is 0, so in theory anything with a lookahead window that's non-negative should be synced correctly.
That being said, after multiple updates to cactbot and Overlayplugin, this bug seems to have disappeared, or at least I can't reproduce it now. This is probably safe to close unless/until someone else has a similar issue.


IndexError: tuple index out of range

Trying to integrate this into my repository, I came across the following issue:
basename='speech_commands_v0.02.tar.gz'
Loading training set: 100%|█████████████████████████████████████████████████████████████████| 84843/84843 [02:13<00:00, 637.32it/s]
Traceback (most recent call last):
  File "run_train.py", line 143, in <module>
    train(parser.parse_args())
  File "train.py", line 151, in train
    ) = create_dataset_fn(args.dir_name, seed=data_seed, batch_size=args.batch_size, **kwargs)
  File "dataloading.py", line 820, in create_speech_commands_10plus2_dataset
    train_ds = FewShotSpokenTermClassification("command", "raw_datasets/", "speech_commands_v0.02", "SpeechCommands", None, False, "training")
  File "few_shot_spoken_term_classification/FewShotSpokenTermClassification.py", line 120, in __init__
    train_or_val_subset = [i for i in range(len(train_or_val_data._data)) if train_or_val_data._data[i][5] in unknown_words + ['_silence_'] + train_words]
  File "few_shot_spoken_term_classification/FewShotSpokenTermClassification.py", line 120, in <listcomp>
    train_or_val_subset = [i for i in range(len(train_or_val_data._data)) if train_or_val_data._data[i][5] in unknown_words + ['_silence_'] + train_words]
IndexError: tuple index out of range

Printing one such tuple in question, print(train_or_val_data._data[0]) gives: (tensor([[-0.0658, -0.0709, -0.0753,  ..., -0.0700, -0.0731, -0.0704]]), 16000, 11, '0165e0e8', 0)
Perhaps the library was tested under another version of AudioLoader? I am using the latest git version.

This is fixed right now right?


Create PhEval Documentation

Using cookie-cutter model as a template, create the structure for the documentation pages.

https://monarch-initiative.github.io/pheval/


main.jsbundle does not exist. This must be a bug with

Description
Failed to construct transformer:  Error: error:0308010C:digital envelope routines::unsupported
at new Hash (node:internal/crypto/hash:71:19)
at Object.createHash (node:crypto:133:10)
at stableHash (/Users/codelantic/Desktop/CL/spotonpay_mobile/node_modules/metro-cache/src/stableHash.js:19:8)
at JsTransformer.getCacheKey (/Users/codelantic/Desktop/CL/spotonpay_mobile/node_modules/metro/src/JSTransformer/worker.js:478:7)
at getTransformCacheKey (/Users/codelantic/Desktop/CL/spotonpay_mobile/node_modules/metro/src/DeltaBundler/Transformer/getTransformCacheKey.js:39:29)
at new Transformer (/Users/codelantic/Desktop/CL/spotonpay_mobile/node_modules/metro/src/DeltaBundler/Transformer.js:147:28)
at /Users/codelantic/Desktop/CL/spotonpay_mobile/node_modules/metro/src/Bundler.js:54:29 {
opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
library: 'digital envelope routines',
reason: 'unsupported',
code: 'ERR_OSSL_EVP_UNSUPPORTED'
}
error Cannot read properties of undefined (reading 'transformFile'). Run CLI with --verbose flag for more details.
TypeError: Cannot read properties of undefined (reading 'transformFile')
at /Users/codelantic/Desktop/CL/spotonpay_mobile/node_modules/metro/src/Bundler.js:91:34
at Generator.next ()
at asyncGeneratorStep (/Users/codelantic/Desktop/CL/spotonpay_mobile/node_modules/metro/src/Bundler.js:14:24)
at _next (/Users/codelantic/Desktop/CL/spotonpay_mobile/node_modules/metro/src/Bundler.js:34:9)

[[ false != true ]]
[[ ! -f /Users/codelantic/Library/Developer/Xcode/DerivedData/SpotonPay-ayrwuibscfwqvvfpjawgxchgstzr/Build/Intermediates.noindex/ArchiveIntermediates/SpotonPay/BuildProductsPath/Release-iphoneos/SpotonPay.app/main.jsbundle ]]
echo 'error: File /Users/codelantic/Library/Developer/Xcode/DerivedData/SpotonPay-ayrwuibscfwqvvfpjawgxchgstzr/Build/Intermediates.noindex/ArchiveIntermediates/SpotonPay/BuildProductsPath/Release-iphoneos/SpotonPay.app/main.jsbundle does not exist. This must be a bug with'
error: File /Users/codelantic/Library/Developer/Xcode/DerivedData/SpotonPay-ayrwuibscfwqvvfpjawgxchgstzr/Build/Intermediates.noindex/ArchiveIntermediates/SpotonPay/BuildProductsPath/Release-iphoneos/SpotonPay.app/main.jsbundle does not exist. This must be a bug with
echo 'React Native, please report it here: https://github.com/facebook/react-native/issues'
React Native, please report it here: https://github.com/facebook/react-native/issues
exit 2

! Need Help
Version
0.63.4
Output of npx react-native info
System:
OS: macOS 12.5.1
CPU: (8) arm64 Apple M1
Memory: 99.97 MB / 8.00 GB
Shell: 5.8.1 - /bin/zsh
Binaries:
Node: 18.7.0 - ~/.nvm/versions/node/v18.7.0/bin/node
Yarn: 1.22.19 - ~/.nvm/versions/node/v18.7.0/bin/yarn
npm: 8.18.0 - ~/.nvm/versions/node/v18.7.0/bin/npm
Watchman: 2<IP_ADDRESS> - /opt/homebrew/bin/watchman
Managers:
CocoaPods: 1.11.3 - /opt/homebrew/bin/pod
SDKs:
iOS SDK:
Platforms: DriverKit 21.4, iOS 15.5, macOS 12.3, tvOS 15.4, watchOS 8.5
Android SDK: Not Found
IDEs:
Android Studio: 2021.3 AI-213.7172.25.2113.9014738
Xcode: 13.4.1/13F100 - /usr/bin/xcodebuild
Languages:
Java: 11.0.16 - /usr/bin/javac
Python: Not Found
npmPackages:
@react-native-community/cli: ^4.14.0 => 4.14.0
react: 16.13.1 => 16.13.1
react-native: 0.63.4 => 0.63.4
react-native-macos: Not Found
npmGlobalPackages:
react-native: Not Found
Steps to reproduce
Hello,
i get this issue when i try to archive my project for testFlight furthermore i have this issues "Failed to construct transformer:  Error: error:0308010C:digital envelope routines::unsupported"..
this is common in android and ios
Snack, code example, screenshot, or link to a repository

Exact same probleme here with M1 Pro, Ventura
Already tried to :
-> Upgrade / Downgrade NodeJS Version
-> Clear pods, yarn cache etc...
-> export NODE_OPTIONS=--openssl-legacy-provider but not really viable and not working on Xcode
-> .....
Tried all things about this problem, still not resolved :/

Exact same probleme here with M1 Pro, Ventura
Already tried to : -> Upgrade / Downgrade NodeJS Version -> Clear pods, yarn cache etc... -> export NODE_OPTIONS=--openssl-legacy-provider but not really viable and not working on Xcode -> .....
Tried all things about this problem, still not resolved :/

Found any solution ? I am also facing the same issue.

+1

I have solved this issue by downgrading my Node version to 16.

I have same problem after upgrading please help.

+1

Fixed by adding an alias to my zsh and lauching it when i need to archive an project with this issue:
vim ~/.zshrc
clearwatch() {
 cd /usr/local/var/run/watchman/NAME_OF_YOUR_COMPUTER-state && rm log pid sock state
}

source ~/.zshrc

@DanielT777 what 'bout
clearwatch() {
 rm -rf /usr/local/var/run/watchman/*-state
}

npm start works on vs code cli works but, npm run android opens terminal app of macos does not work for me.

downgrading to node 16.0 cleared the  failed to construct transformer problem.

I've tried downgrading to 16.0 in addition to everything mentioned on this post, and what I could find on the internet. If anyone has found a solution not mentioned in this post, would greatly appreciate it

solution?

I had the same problem, when upgrade my node version to the latest - 16.20.1. To fix this problem I've just downgraded to 16.13.0

I found the solution I upgraded react native to latest version so 0.72 upgraded node js to 20.4 current latest and recrawled watchman I'm using a m1 MacBook Air .

Any solutions ?

Same here!
Now my workaround is add 'export NODE_OPTIONS=--openssl-legacy-provider' to Bundle react native code and image phase, and raise metro by 'yarn start' rather than Xcode run button.
Can someone give a better solution?

Is there any workable solution for this? I have tried all the solutions suggested above. But still getting the same error. :(

Finally, I was able to solve the problem by downgrading the Node version from 17 to 16 using nvm. Previously had an issue with user PREFIX for the node version. Able to run the project removing the old PREFIX and adding the new Node version PREFIX using nvm commands.


Import: numeric field expanded import options

Numeric fields should have an expanded set of options:

Current behaviour

Numeric fields should allow for empty entries and entries that can be parsed correctly
Numeric fields generate a _valid field additional to the field itself indicating whether a given row is valid or not

New behaviours:

Validation:

Accept empty values (current behavior)
Strict

Expect all numbers to parse correctly or fail

Relaxed

Treat all casting failures as values to flag as invalid

Flag field

export flag field
don't export flag field (set to default value only)

Please also think about how to represent that in the schema.
Conversation from 18/03:
Here are all the combinations that I can think of:

no _valid field, fail on any non-numeric values (strict)
no _valid field, fail on any non-empty non-numeric values, write default value for empty numeric values (normal)
no _valid field, write default value for all non-numeric values (relaxed)
no strict with _valid field
_valid_field, fail on any non-empty non-numeric values, write default value for empty numeric values (normal)
_valid field, write default value for all non-numeric values (relaxed) - CURRENT MODE

Here is what I think of how the validation and flag field is presented in the schema:
"a_numeric_field": {
"field_type": "numeric",
"value_type": "uint16",
"default": 100
"validation_mode": "strict" (or "normal", "relaxed")
"if_export_flag_field": True (False)
},

"default" -> "invalid_value"
"validation_mode": "strict", "allow_empty", "relaxed" (allow_empty by default)
"create_flag_field": True, False (True by default)
"flag_field_name": string ("_valid" by default) (ignored if 'create_flag_field' is False)
What do you think of these?

Sounds good.


Keep getting model_id constraint violation when saving

Do exactly what you described...but keep getting this
SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'model_id' cannot be null (SQL: insert into trix_rich_texts (model_id, model_type, field, content, updated_at, created_at) values......
What can be the problem?
Thanks,

@kevin-xyz2002  I am getting the same error, did you find any solution for the same ?


consider removing first few lines of output from pqrs cat --json?

I would love to pipe the contents of pqrs cat --json <filename into jq or other tools. Currently the header from pqrs cat prevents me from doing this.

######################################################
File: ../myfolder/myfile.parquet
######################################################

Is there a chance we can either remove this entirely or remove it specifically when used with the --json flag?
My current version:
❯ pqrs --version
pqrs 0.2.0

Potentially related to --csv --no-header flag that was recently added: https://github.com/manojkarthick/pqrs/pull/31

maybe we can add --quiet argument to cat command, if this is valid, I can make contribute this feature

maybe we can add --quiet argument to cat command, if this is valid, I can make contribute this feature

Hi @klion26, for sure - happy to review and release the feature if you contribute.


Better puzzle size calculation

Current the puzzle size calc multiple it just a tad too much...
longest = max(10, longest_word_length)
# calculate multiplier for larger word lists so that most have room to fit
multiplier = len(all_words) / 15 if len(all_words) > 15 else 1
# level lengths in config.py are nice multiples of 2
l_size = log2(len(level)) if level else 1  # protect against log(0) in tests
size = round(longest + l_size * 2 * multiplier)

Maybe up division to 18 from 15 🤷♂

Simply change the 15 to an 18 right there in the code or move to something like config.word_density?

Yeah this was really just a reminder for me. And yes, it could be as simple as swapping the numbers, I just planned to explore some other ways to calculate it if I had time.


Case sensitive

Any way to incorporate case insensitive option/flag?

Actually, duh, just need to set both needle and haystack to lowercase :)

Yeah. It's up to the consumer to prepare the data in the same case


Android Support

As this Package is a great solution,
please update it so it also supports Android.
the other solutions all lack some important feature.

I've researched it a bit and it seemed to me it could be not doable. Maybe GBoard itself has some API though, but I doubt it. PRs are very welcome!!


Changed language_id to language for ricecooker

This change is to maintain consistency of the .json wire format for internal API when creating Files and Nodes.
The key name is language and its content is the language code (a string) from the internal representation lookup table.


TypeError: null attempting to click Submit button using Safari - iOS 15.3

Preflight checklist

[X] I could not find a solution in the existing issues, docs, nor discussions.
[X] I agree to follow this project's Code of Conduct.
[X] I have read and am following this repository's Contribution Guidelines.
[X] This issue affects my Ory Cloud project.
[X] I have joined the Ory Community Slack.
[X] I am signed up to the Ory Security Patch Newsletter.

Describe the bug
TypeError: null is not an object (evaluating 'y.name') is what is shown in the Developer Tools console, found in Safari. One must use Safari's developer tools to see the error outputs from an Xcode simulator.

Reproducing the bug

Install Ory Element's React SPA example. I used the instructions from README.md file
Fire up Vite via npm run dev -- --port 3000 and Ory Tunnel via ory tunnel http://localhost:3000 --project <project-slug> --dev
Install a simulator with iOS 15.3 or 15.2 via Xcode
Push the logical home button so that you can go to the Safari browser
Navigate to http://<IP_ADDRESS>:3000
At this point, I advise having your Mac's Safari open so you can see the log output for your simulator
Press Submit on the sample login form

Relevant log output
[Error] TypeError: null is not an object (evaluating 'y.name')
	onSubmit (index.mjs:1791)
	callCallback2 (react-dom_client.js:3674)
	dispatchEvent
	invokeGuardedCallbackDev (react-dom_client.js:3699)
	invokeGuardedCallback (react-dom_client.js:3733)
	invokeGuardedCallbackAndCatchFirstError (react-dom_client.js:3736)
	executeDispatch (react-dom_client.js:7016)
	processDispatchQueueItemsInOrder (react-dom_client.js:7036)
	processDispatchQueue (react-dom_client.js:7045)
	dispatchEventsForPlugins (react-dom_client.js:7053)
	batchedUpdates$1 (react-dom_client.js:18909)
	batchedUpdates (react-dom_client.js:3579)
	dispatchEventForPluginEventSystem (react-dom_client.js:7176)
	dispatchEventWithEnableCapturePhaseSelectiveHydrationWithoutDiscreteEventReplay (react-dom_client.js:5478)
	dispatchEvent (react-dom_client.js:5472:93)
	dispatchDiscreteEvent (react-dom_client.js:5449)
	dispatchDiscreteEvent

Relevant configuration
Do not sign into Ory after entering the `ory tunnel` command. Everything else was done as prescribed via the README instructions.

Version
v0.0.1-beta.7
On which operating system are you observing this issue?
macOS
In which environment are you deploying?
Ory Cloud
Additional Context
Please take a look at CanIUse. There you can see that there is a bug with the SubmitEvent API: submitter functionality starting on iOS 15.3.
My coworker found the problematic code resides in the following file:
src/react-components/ory/helpers/user-auth-form.tsx, ~line 83
Thank you

Our users reported this problem on iOS 15.3 too.

My user reported this problem:
https://github.com/user-attachments/assets/bc5208ec-7124-4b60-8b66-93b6d0d7e83a


Unable to set gain at 5v/div

Reported in 1.4.0

Seems to be working now.... Not sure what version of WaveformsLive it is...

Fixed in firmware version 1.37.0.  Fix does not depend on a specific version of WaveForms Live.


Question about localization simulation through dreamview

System information

OS Platform and Distribution (e.g., Linux Ubuntu 14.04): ubuntu 16.04
Apollo installed from (source or binary): source
Apollo version (1.0, 1.5, 2.0, 2.5, 3.0): 2.5

Steps to reproduce the issue:
Hello. Nowadays, I am doing Apollo simulation through the dreamview. And I have a question about dreamview system. My goal is running the localization module so below is my command.
bash scripts/bootstrap.sh start
bash scripts/localization.sh start
And I play new bag file which has no '/apollo/localization/pose' topic messages but others all exactly same with demo_2.0.bag file. In this case, I think that the car would move the same route with demo_2.0.bag file case. (Because I only filter /apollo/localization/pose topic) But, the car is running differently with demo_2.0.bag case when I play with loop function. Could you tell me this above situation is right? Or did I modify something wrongly?
Another problem is when I play bag file without /apollo/localization/pose topic, new /apollo/localization/pose topics are created when time is 27 / 34.895545. Why such situation happened...?
In advance, thank you for helping me.
Supporting materials (screenshots, command lines, code/script snippets):

Do you see any data update from the ros topic /apollo/localization/pose?

@unacao
I cannot see any messages from the ros topic /apollo/localization/pose right after running the rosbag play.
And after almost 30s, /apollo/localization/pose topic messages are created.

I tried to reproduce the problem and seems like there is no /apollo/localization/pose output? Assigning to localization related developers for a further look. @zhouyao4321

Localization module needs to be initialized before it broadcasts localization results. In demo bag, there are some inputs for localization module. So it's possible that the localization module starts to work after 30 seconds. But the localization module may not work properly, since when circling playing bag, the end of bag is connected to beginning again, which may lead to undefined localization output.

@unacao
Yes right. After filter /apollo/localization/pose messages and do ./scripts/localization.sh start and do rosbag. Then, you can see the problem.
@zhouyao4321
Thank you! Then, is there any method to solve this problem? I just want to play bag file perfectly with localization module.

Closing this issue as it appears to be resolved. Thanks!


Don't create folder with package name

I've been trying to add jquery.js and bootstrap.css into my project. I would like to have this files exactly in:

src/js/jquery.js
src/css/bootstrap.css

Right now there is no way to do this because bower-installer will always create a folder with the package name si this would became

src/js/jquery/jquery.js
src/css/bootstrap/bootstrap.css

The closest I have get is using the "mapping" option (with the target name starting with "../", eg "../jquery.js") which creates the files in the desired locations but keeps an empty folder with the package name.

I have faced the same problem.

I replaced the latest code on /usr/local/lib/node_modules/bower-installer and modifed
var defaultInstallOptions = {
  "includePackageNameInInstallPath": false
};

in lib/installer.js file. Then I got expected result.

As https://github.com/blittle/bower-installer/pull/128 pointed, a more elegant way is provide this piece of code
    "options" : {
      "includePackageNameInInstallPath": false
    }

in your install block of bower.json file.

Thank you very much!

@tikoflano @liudonghua123 I downloaded the latest master and copied it over the version in /usr/local/lib/node_modules/bower-installer and tried adding the options key and even editing the lib/installer.js though neither works.
Here's my bower.json file:
{
  "name" : "test",
  "version": "0.1",
  "dependencies": {
    "medium-editor": "latest"
  },
  "install": {
    "options": {"includePackageNameInInstallPath": "false"},
    "path": "mallowstreet/assets/js",
    "sources": {
      "medium-editor": {
        "mapping": {"bower_components/medium-editor/dist/js/medium-editor.min.js": "medium-editor.min.js"}
      }
    }
  }
}

Any ideas?

@derekoakley, have you solved this problem?

Sorry @Manidos, unfortunately I don't use bower anymore. I think I used a node module to copy the files in the end.


Use https for gravatar image

To remove not secure resource warning

Looks good to merge, pending CI.


Writing JSON from npz fails because int64 is not JSON serializable

I am trying to convert old npz test data to JSON in order to make CI testing of skymap_scanner more transparent. Although such conversion seems to fail.
python examples/npz_to_json.py --input-file skymap_scanner/tests/data/results_npz/millipede_wilks/run00127907.evt000020178442.HESE_1.npz

[...]

TypeError: Object of type int64 is not JSON serializable

json doesn't understand numpy types, you can call array.tolist() before trying to serialize

Makes sense. I was just wondering whether or not this used to work in the past, considering that the CI test seems to output JSON just fine.
I guess when we build SkyScanResult from the runtime scanner output it does not use the numpy type for the internal representation but it does when loading a npz.

The python exception is annoyingly cryptic about where the problem is. However since the JSON file is sequentially written just up to where the error occurs, it appears that the problem are the metadata (likely the nside field).


Add option to create Unbound forwarding configs.

Add --forward option to create Unbound configs for forwarding requests to LanCache DNS servers.
Using forward-first: yes allows downloads to continue to work even when LanCache is down, by falling back to using Unbound to resolve URLs.

Seems plausible, Can you confirm this has been tested on a running unbound instance and upload confirmation logs somewhere showing the config file generated, then I'll merge this?

Similarly -- I'm currently running a functional Lancache system with just AdGuard and the lancachenet/monolithic image.
Instead of using "Settings" > "DNS Settings" to sideload a text file to forward queries for cache-domains to a lancache DNS server which returns the lancache IP, I use "Filters" > "DNS Rewrites" to just point at the lancachenet/monolithic docker image's IP address.

This removes the need for another DNS server, as AdGuard can just do it by itself ...as can Unbound. Should I write that up someplace? Is there interest?


Chunk-Components

Background
Sometimes it makes sense to attach additional data to a chunk. Whether it is flags, entire components or additional arrays to naturally expand the possibilities. Chunk components would therefore make a lot of sense.
Solution
We could simply give each chunk an additional array in which these components could be stored. Just like with the entity component arrays. This would require an additional API to access them. The current one would probably not need to be changed or touched. It would be an extension.
chunk.AddChunkComponent(...);
chunk.GetChunkComponent();
chunk.HasChunkComponent();
...

This would likely permit #190 by allowing addition and change watermarks to be stored alongside chunks and letting queries filter out types with watermarks older than their last run watermark


US 01.04.01

User Story
As an organizer, I want to upload an event poster to provide visual information to attendees.
Acceptance Criteria

Given:

A possible way to upload an event poster in organizer view.

Action

Shows the visual information to attendees.

Then

Store the uploaded event poster in the event object.

Additional components

Storyboard demonstrating the feature.
Wiki entry describing feature.
UML for frontend components.


helm: support configurable automountServiceAccountToken

Please provide a description of this PR:
Keeps the automountServiceAccountToken defaulting to true just as before for ServiceAccounts/Deployments.
It makes it configurable though for cases such as with Azure AKS Security policy which scans for any pod/SA that have it set to true and instead recommended as part of their security benchmarks to disable auto-mounting and injecting the service account tokens as extra volumes/volumeMounts explicitly for each workload that requires it: Kubernetes clusters should disable automounting API credentials
A similar thread on this from cert-manager and hardened values for such setups & rationale behind it.

Hi @stefan-caraiman. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

hi @hanxiaop  thanks for your suggestion, but we are not able to use Kustomize in our setup due to a number of internal limitations.
I've went ahead and forked this chart internally and we are able to use Istiod as well with custom extraVolumes/extraVolumeMounts and automountServiceAccountToken: false.
Example manifest:
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "21"
  creationTimestamp: "2023-02-23T12:24:51Z"
  labels:
    app: istiod
    argocd.argoproj.io/instance: istio
    istio: pilot
    istio.io/rev: default
    operator.istio.io/component: Pilot
    release: istio
  name: istiod
  namespace: istio-system
  resourceVersion: "389809518"
  uid: 19a10f8f-ea78-4de5-9803-0c8e709985ec
spec:
  progressDeadlineSeconds: 900
  replicas: 3
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      istio: pilot
  strategy:
    rollingUpdate:
      maxSurge: 100%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      annotations:
        ambient.istio.io/redirection: disabled
        kubectl.kubernetes.io/restartedAt: "2023-08-07T16:52:40+03:00"
        prometheus.io/port: "15014"
        prometheus.io/scrape: "true"
        sidecar.istio.io/inject: "false"
      labels:
        app: istiod
        istio: pilot
        istio.io/rev: default
        operator.istio.io/component: Pilot
        sidecar.istio.io/inject: "false"
    spec:
      automountServiceAccountToken: false
      containers:
      - args:
        - discovery
        - --monitoringAddr=:15014
        - --log_output_level=default:info
        - --log_as_json
        - --domain
        - cluster.local
        - --keepaliveMaxServerConnectionAge
        - 30m
        env:
        - name: REVISION
          value: default
        - name: JWT_POLICY
          value: third-party-jwt
        - name: PILOT_CERT_PROVIDER
          value: istiod
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: SERVICE_ACCOUNT
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.serviceAccountName
        - name: KUBECONFIG
          value: /var/run/secrets/remote/config
        - name: METRIC_GRACEFUL_DELETION_INTERVAL
          value: 2m
        - name: METRIC_ROTATION_INTERVAL
          value: 2m
        - name: PILOT_TRACE_SAMPLING
          value: "100"
        - name: PILOT_ENABLE_ANALYSIS
          value: "false"
        - name: CLUSTER_ID
          value: Kubernetes
        - name: GOMEMLIMIT
          valueFrom:
            resourceFieldRef:
              divisor: "0"
              resource: limits.memory
        - name: GOMAXPROCS
          valueFrom:
            resourceFieldRef:
              divisor: "0"
              resource: limits.cpu
        - name: PLATFORM
        image: ourregistry.azurecr.io/istio/pilot:1.20.3
        imagePullPolicy: Always
        name: discovery
        ports:
        - containerPort: 8080
          protocol: TCP
        - containerPort: 15010
          protocol: TCP
        - containerPort: 15017
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /ready
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 1
          periodSeconds: 3
          successThreshold: 1
          timeoutSeconds: 5
        resources:
          limits:
            cpu: "1"
            memory: 1Gi
          requests:
            cpu: 100m
            memory: 384M
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsNonRoot: true
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
          name: serviceaccount-token
          readOnly: true
        - mountPath: /var/run/secrets/tokens
          name: istio-token
          readOnly: true
        - mountPath: /var/run/secrets/istio-dns
          name: local-certs
        - mountPath: /etc/cacerts
          name: cacerts
          readOnly: true
        - mountPath: /var/run/secrets/remote
          name: istio-kubeconfig
          readOnly: true
        - mountPath: /var/run/secrets/istiod/tls
          name: istio-csr-dns-cert
          readOnly: true
        - mountPath: /var/run/secrets/istiod/ca
          name: istio-csr-ca-configmap
          readOnly: true
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: istiod
      serviceAccountName: istiod
      terminationGracePeriodSeconds: 30
      volumes:
      - name: serviceaccount-token
        projected:
          defaultMode: 292
          sources:
          - serviceAccountToken:
              expirationSeconds: 3607
              path: token
          - configMap:
              items:
              - key: ca.crt
                path: ca.crt
              name: kube-root-ca.crt
          - downwardAPI:
              items:
              - fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
                path: namespace
      - emptyDir:
          medium: Memory
        name: local-certs
      - name: istio-token
        projected:
          defaultMode: 420
          sources:
          - serviceAccountToken:
              audience: istio-ca
              expirationSeconds: 43200
              path: istio-token
      - name: cacerts
        secret:
          defaultMode: 420
          optional: true
          secretName: cacerts
      - name: istio-kubeconfig
        secret:
          defaultMode: 420
          optional: true
          secretName: istio-kubeconfig
      - name: istio-csr-dns-cert
        secret:
          defaultMode: 420
          optional: true
          secretName: istiod-tls
      - configMap:
          defaultMode: 420
          name: istio-ca-root-cert
          optional: true
        name: istio-csr-ca-configmap

This will not break the existing charts, just introduces the option to overwrite the default true for automountServiceAccountToken

@stefan-caraiman thanks for the explanation, the usage with additional mounts makes sense to me. Maybe @jacob-delgado is more familiar with the scenario usage, and I'll let him review it.

Manually mounting the same SA to avoid having Kubernetes do it automatically seems ridiculous. The intent of policies like https://portal.azure.com/#blade/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2F423dd1ba-798e-40e4-9c4d-b6902674b423 is, as I understand it, to provide guidelines to reduce access where its not needed. To take those guidelines and then do the same thing anyways, in a more convoluted and hard to understand way, is worse than what its telling you not to do.
I don't mind the gateway changes. I would maybe even make it default to false. But the Istiod change seems like a bad idea. LMK if I am missing context. If you want to implement ideas that are "bad", you can always do so with things like https://istio.io/latest/docs/setup/additional-setup/customize-installation-helm/ without getting approval.

@howardjohn thank you for your comment, but a counter argument could be that Explicit is better than implicit (to quote the Python Zen), so I don't see why you would consider it "bad" to have the option to customize/explicit define it.
I see it more "bad" to have the K8s api calls "just work" because of the built-in ServiceAccount admission controller which actually does the exact same thing aka modifying the pods and adding those volumes/volumeMounts when  automountServiceAccountToken: true (the default behaviour which most users are not aware on why/how it works).
There was thread on this some time ago in this Kubernetes issue in regards to actually making automountServiceAccountToken default to false (and other community charts which do it by default as well even when requiring the SA token inside the pod via volumes)
I have a case which does not allow additional customizations via Kustomize only overwriting/setting values (setting Istio as a Helm chart through ArgoCD) and I would rather not fork the project in order to customize a hardcoded value 😅

I am fine hardcoding automountServiceAccountToken: true instead of unsetting it. But setting it to false only to add a bunch of manual volume mounts is not solving any problems aside from tricking a security scanner.

As previously mentioned, it is not for the sake of tricking security scanners, I would rather have it explicitly defined ( the Pod definition will also look the same either with explicit "manual" volumes or implicit option which get modified by the ServiceAccount admission controller).
But if you consider it bad practice you can close this MR


Feat (#2284): Feature: Add Table of Contents to the Readme

Description

Adds a table of contents to readme.md
Changes header styles for the different sections to introduce better distinction between the sections
Adds a hierarchy to some of the sections for better readability
See issue #2284
View the readme.md in my fork here to sample this change: https://github.com/KameronKeller/bruno/blob/feature/table-of-contents/readme.md

Original vs. Suggested Update:

Suggested Heading Change:

Contribution Checklist:

[x] The pull request only addresses one issue or adds one feature.
[x] The pull request does not introduce any breaking changes
[x] I have added screenshots or gifs to help explain the change if applicable.
[x] I have read the contribution guidelines.
[x] Create an issue and link to the pull request.

Note: Keeping the PR small and focused helps make it easier to review and merge. If you have multiple changes you want to make, please consider submitting them as separate pull requests.
Publishing to New Package Managers
Please see here for more information.

I'd suggest converting the README to AsciiDoc to be able to make use of a table of contents macro.
I'm afraid the manually created TOC might not be maintained too well, resulting either in a broken or inconsistent state at some point in time.

Thank you Andreas, I will look into what it would take to create a macro for the table of contents.

Thank you Andreas, I will look into what it would take to create a macro for the table of contents.

@KameronKeller Unfortunately, I currently don’t have the time to prepare a specific example for the Bruno repository but I can point you to an example in one of my repositories.
With a README in AsciiDoc format, all it takes to get a table of contents are these two lines:
:toc: preamble
:toclevels: 3

https://github.com/andreassiegel/tessa/blob/2ae3c8c046189e4ae7f67705c31f4a1e03c80c77/README.adoc?plain=1#L7-L8
Actually, the first one would be sufficient to automatically get the table of contents.
The second one then defines how deep the TOC should be.

Merged ! Thanks @KameronKeller !


Capture Logging/Silence tests in setUp

For comparison, here's the new output: https://travis-ci.org/pyfarm/pyfarm-agent/jobs/77930978
And here's the old output: https://travis-ci.org/pyfarm/pyfarm-agent/jobs/77850694
There's still a couple of places we're emitting output, but those will not be as easy to capture because of the context in which the logger is being used.

@guidow, PTAL

Looks good.


feat: [NPM] add dataplane health metrics

Introduce dataplane health metrics, and pave the way to add other metrics endpoints easily.
TODO: Consider how to initialize only the necessary metrics/endpoints.

moving to #1220


[SPARK-50176][CONNECT][FOLLOWUP][3.5] Fix ReattachableExecuteSuite failure

What changes were proposed in this pull request?

https://github.com/apache/spark/pull/48725 closes a session completely during ReattachableExecuteSuite causing 'sleep' to be unavailable in subsequent test cases.
https://github.com/apache/spark/pull/43546 fixes the issue by re-creating the 'sleep' udf in each test case needing the udf, and this PR back-ports part of it.
Why are the changes needed?

In order to make the test green again.
Does this PR introduce any user-facing change?

No.
How was this patch tested?

Existing tests.
Was this patch authored or co-authored using generative AI tooling?

No.

@LuciferYang Hi, this should fix the reattach execute suite failure in branch-3.5.

Merged to branch-3.5.

late LGTM


Will TensorFlow run on an AMD FX processor?

I'm looking at buying a new PC to run TensorFlow on.  The system I'm looking at has an AMD FX processor and a GTX 960 with 2GB of memory.  Will TensorFlow run fine on that processor and be able to take advantage of the GTX 960 for CUDA processing?  The O/S I'm planning on is Ubuntu Linux.

It's an x86 processor so it should be fine on CPU, and I believe we support the GTX960 (It's a Maxwell GPU with cuda compute capability 5).

I don't think AMD FX CPU is a problem. But GTX 960 with 2 GB might be an
issue.
Thanks.
Junli
On Mon, Dec 7, 2015 at 9:03 PM, roschler<EMAIL_ADDRESS>wrote:

I'm looking at buying a new PC to run TensorFlow on. The system I'm
looking at has an AMD FX processor and a GTX 960 with 2GB of memory. Will
TensorFlow run fine on that processor and be able to take advantage of the
GTX 960 for CUDA processing? The O/S I'm planning on is Ubuntu Linux.
—
Reply to this email directly or view it on GitHub
https://github.com/tensorflow/tensorflow/issues/440.

--

Junli Gu--谷俊丽
Coordinated Science Lab
University of Illinois at Urbana-Champaign

Yeah, it would be supported by TensorFlow, but is definitely on the low-side compared to most compute GPUs.

Thanks everyone.  Would it be enough of an improvement to get a GTX 960 with 4GB, or is anything less than something like a GTX TitanX going to be underpowered?

@roschler , I would get at least a 980TI, but if I had to do it all over again, I would definitely get the titanx

@LeavesBreathe - Ok, that's what I'll do.  Thanks.

Does GTX 960 work on pip installation or must it be installed from sources? I have a GTX 960 build, but doesn't seem to get recognized by tensorflow.


searching in a field for an exact phrase containing spaces

I have tags on all my web pages, and have built the lunr index with the tags field. Some tags are multiword phrases. If I search for tags:hand drawn maps I get completely wrong result. However, if I search for tags:hand* I get the correct result. I have tried tags:+hand +drawn +maps and tags:"hand drawn maps" but without any success. Suggestions?
update: I should add that I make my index and conduct my query like so
    idx = lunr(function () {
        this.field('title', { boost: 10 }),
        this.field('tags'),
        this.field('body'), { boost: 20 },
        this.field('created'),
        this.ref('file'),

        pages.forEach(function (doc) {
            this.add(doc)
        }, this)
    });

    searchResult = idx.search(q).map(function(result) {
            return {
                ref : result.ref,
                disp : result.ref.replace(/-/g, ' ')
            }
        });

When performing a search using lunr.Index#search the query string you use is parsed into a lunr.Query object. By default the parser assumes that whitespace indicates a term boundary, that is a search for "foo bar" is interpreted as a search for the terms foo and bar. What you want is a search for the term foo bar.
You can bypass the query parsing entirely by using the lunr.Index#query method which allows you to specify the term exactly how you want. Alternatively you can continue using lunr.Index#search but escape the spaces in the query string. Finally you could use wildcards to match the spaces, though this will match any character so is probably not a great solution.
I have put together a fiddle showing the three approaches. I think using lunr.Index#query is probably the best choice.

Thanks for explaining in detail @olivernn !
I was wondering about one thing:
How come if I don't wrap the tag in an array, when adding, it no longer works?
(line 7, see updated fiddle: https://jsfiddle.net/hey72dmg/14/)
It couldn't find anything in the docs.

@icidasset yeah, sorry I didn't explain that well, and the relevant document is a bit hidden.
Lunr uses a lunr.Token internally, these are mostly just wrappers around a string. lunr.tokenizer is used to create lists of tokens from the fields of the documents being indexed.
If a field is an array, Lunr assumes that the items in the array are already 'tokens', for everything else it assumes it has to split the field into tokens itself. It does this splitting on whitespace (among other characters).
So, when you pass an ['foo bar baz'] Lunr assumes you have already done the splitting and the token is "foo bar baz", when you just pass "foo bar baz" it does the splitting it self and you get "foo", "bar", "baz".

/Users/athenacapsis/Documents/Making and Knowing/making-knowing-edition/scripts/test_search.js   Why doesnt my test work?

const fs = require('fs');
// load lunr with fr support
var lunr = require('lunr');
require("lunr-languages/lunr.stemmer.support")(lunr)
require('lunr-languages/lunr.multi')(lunr)
require("lunr-languages/lunr.fr")(lunr)
const jsdom = require("jsdom");
const { JSDOM } = jsdom;
var documents = [{
"name": "Lunr",
"text": "Like Solr, but much smaller, and not as bright."
}, {
"name": "React",
"text": "A JavaScript library for building user interfaces."
}, {
"name": "Lodash",
"text": "A modern JavaScript utility library delivering modularity, performance & extras."
}]
var idx = lunr(function () {
  this.ref('name')
  this.field('text')

  documents.forEach(function (doc) {
    this.add(doc)
  }, this)
})

/* USING QUERY METHOD****/
let results =   idx.query(function(){
const searchTerm = 'A JavaScript library for building user interfaces.';   //'library'
console.log(search term is ${searchTerm})
this.term(['A JavaScript library for building user interfaces.'])
});
console.log(results length is ${results.length})
  const resultText = [];
  let docWithTheTerm;
  results.forEach( result =>{
  docWithTheTerm = documents.find( d =>{ return d.name === result.ref})
        if(docWithTheTerm !== undefined)
        {
              console.log(docWithTheTerm.text)
              resultText.push(docWithTheTerm);
        }
  })


Introduce a fail fast plugin

TL;DR
Introduce a fail fast plugin which can be used as a default implementation for cases where Flyte deployments are not set-up to support certain tasks.
Type

[ ] Bug Fix
[ ] Feature
[x] Plugin

Are all requirements met?

[x] Code completed
[x] Smoke tested
[x] Unit tests added
[x] Code documentation added
[ ] Any pending items have an associated Issue

Complete description
In certain cases it would be useful to leverage configurable plugins as a whitelist mechanism without necessarily having a fallback implementation. In this case a fail-fast plugin implementation would be useful to have as the default plugin implementation for a task type and whitelisted projects/domains could be added for the actual plugin handler.
Tracking Issue
https://github.com/lyft/flyte/issues/552
Follow-up issue
NA

Codecov Report

Merging #130 into master will increase coverage by 0.10%.
The diff coverage is 94.73%.

@@            Coverage Diff             @@
##           master     #130      +/-   ##
==========================================
+ Coverage   61.28%   61.38%   +0.10%
==========================================
  Files         105      106       +1
  Lines        5943     5962      +19
==========================================
+ Hits         3642     3660      +18
- Misses       1928     1929       +1
  Partials      373      373

Impacted Files
Coverage Δ

go/tasks/pluginmachinery/bundle/fail_fast.go
94.73% <94.73%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bf3f0f6...d5a9eea. Read the comment docs.

ping @kumare3


Tests are broken

FAILED: tests/CMakeFiles/common.dir/block.cpp.o
/usr/bin/x86_64-pc-linux-gnu-g++  -I/var/tmp/portage/dev-libs/dstu-engine-1.3.2/work/dstu-engine-1.3.2/keylib  -march=native -O2 -pipe -mmmx -msse4.2 -mssse3 -mfpmath=sse -fomit-frame-pointer         -frecord-gcc-switches -std=c++14 -MD -MT tests/CMakeFiles/common.dir/block.cpp.o -MF tests/CMakeFiles/common.dir/block.cpp.o.d -o tests/CMakeFiles/common.dir/block.cpp.o -c /var/tmp/portage/dev-libs/dstu-engine-1.3.2/work/dstu-engine-1.3.2/tests/block.cpp
/var/tmp/portage/dev-libs/dstu-engine-1.3.2/work/dstu-engine-1.3.2/tests/block.cpp: In function 'void DSTUEngine::printBlock(const void*, size_t)':
/var/tmp/portage/dev-libs/dstu-engine-1.3.2/work/dstu-engine-1.3.2/tests/block.cpp:49:41: error: ISO C++ forbids declaration of 'type name' with no type [-fpermissive]
   49 |     const auto* ptr = static_cast<const uint8_t*>(data);
      |                                         ^~~~~~~
/var/tmp/portage/dev-libs/dstu-engine-1.3.2/work/dstu-engine-1.3.2/tests/block.cpp:49:41: error: expected '>' before 'uint8_t'
/var/tmp/portage/dev-libs/dstu-engine-1.3.2/work/dstu-engine-1.3.2/tests/block.cpp:49:41: error: expected '(' before 'uint8_t'
   49 |     const auto* ptr = static_cast<const uint8_t*>(data);
      |                                         ^~~~~~~
      |                                         (
/var/tmp/portage/dev-libs/dstu-engine-1.3.2/work/dstu-engine-1.3.2/tests/block.cpp:49:41: error: 'uint8_t' was not declared in this scope
   49 |     const auto* ptr = static_cast<const uint8_t*>(data);
      |                                         ^~~~~~~
/var/tmp/portage/dev-libs/dstu-engine-1.3.2/work/dstu-engine-1.3.2/tests/block.cpp:7:1: note: 'uint8_t' is defined in header '<cstdint>'; did you forget to '#include <cstdint>'?
    6 | #include <iomanip>
  +++ |+#include <cstdint>
    7 | #include <stdexcept>
/var/tmp/portage/dev-libs/dstu-engine-1.3.2/work/dstu-engine-1.3.2/tests/block.cpp:49:49: error: expected primary-expression before '>' token
   49 |     const auto* ptr = static_cast<const uint8_t*>(data);
      |                                                 ^
/var/tmp/portage/dev-libs/dstu-engine-1.3.2/work/dstu-engine-1.3.2/tests/block.cpp:49:56: error: expected ')' before ';' token
   49 |     const auto* ptr = static_cast<const uint8_t*>(data);
      |                                                        ^
      |                                                        )

Fixed in 1.3.3.


Support cloud storage

Original PR: https://github.com/openvinotoolkit/cvat/pull/2620

Motivation and context

Support work with remote cloud storage without copying data to CVAT
Related issue: #863
How to generate temporary credentials (S3) e.g.
import boto3
# generate temporatry credentials
sts_client = boto3.client('sts', aws_access_key_id="...", aws_secret_access_key="...")
tokens = sts_client.get_session_token()
aws_access_key_id = tokens.get('Credentials').get('AccessKeyId')
aws_secret_access_key = tokens.get('Credentials').get('SecretAccessKey')
aws_session_token = tokens.get('Credentials').get('SessionToken')

# test a credentials validity
s3 = boto3.client('s3',
	aws_access_key_id=aws_access_key_id,
	aws_secret_access_key=aws_secret_access_key,
	aws_session_token=aws_session_token
)
s3.list_buckets()

How has this been tested?

Manually with swagger
REST API

GET /api/v1/cloudstorages
POST /api/v1/cloudstorages
GET /api/v1/cloudstorages/{id}
GET /api/v1/cloudstorages/{id}/content
PATCH /api/v1/cloudstorages/{id}

Supported cloud providers:

[x] Azure Blob container (implemented, not tested)
[x] AWS S3 bucket (implemented, tested)
[ ] Google Drive

Iterations:

Support images
Support video
Support archive

Checklist

[x] I submit my changes into the develop branch
[x] I have added description of my changes into CHANGELOG file
[ ] I have updated the documentation accordingly
[ ] I have added tests to cover my changes
[x] I have linked related issues (read github docs)
- [ ] I have increased versions of npm packages if it is necessary (cvat-canvas,
cvat-core, cvat-data and cvat-ui)

License

[x] I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.
[x] I have updated the license header for each file (see an example below)

# Copyright (C) 2020 Intel Corporation
#
# SPDX-License-Identifier: MIT

@Marishka17 , I will merge the PR as is. We discussed a couple of issues:

it is not convenient for users to generate 3 tokens manually. Probably the only solution is to use the original tokens. Potentially it can create some security problems. Need to check with appropriate security guys in our org.
I have a bucket with a couple of images and manifest.jsonl. When I try to get content of the bucket, it doesn't work with "out of index" error.

{"version":"1.1"}
{"type":"images"}
{"name":"000000000139","extension":".jpg","width":640,"height":426,"meta":{"related_images":[]},"checksum":"b242df990b7315d5ad71c8cd3bc8bfdb"}
{"name":"000000000285","extension":".jpg","width":586,"height":640,"meta":{"related_images":[]},"checksum":"3bc71e9139761c5308587dd47c08695f"}
{"name":"000000000632","extension":".jpg","width":640,"height":483,"meta":{"related_images":[]},"checksum":"bf1e99429d341e28fcfcfc36b2edff06"}
{"name":"000000000724","extension":".jpg","width":375,"height":500,"meta":{"related_images":[]},"checksum":"e2a379e1649fdcc25a5f3829778525f8"}
{"name":"000000000776","extension":".jpg","width":428,"height":640,"meta":{"related_images":[]},"checksum":"3b7f7967c63f61a6139b25280ca0b372"}
{"name":"000000000785","extension":".jpg","width":640,"height":425,"meta":{"related_images":[]},"checksum":"647645423232ef03691064eda55f9f36"}
{"name":"000000000802","extension":".jpg","width":424,"height":640,"meta":{"related_images":[]},"checksum":"9f244e7c14cadca4195093a06bcbaf16"}
{"name":"000000000872","extension":".jpg","width":621,"height":640,"meta":{"related_images":[]},"checksum":"5516b8d3bad2c08f2c86a2ca95d0d062"}
{"name":"000000000885","extension":".jpg","width":640,"height":427,"meta":{"related_images":[]},"checksum":"ec166d3d69fc7475d4a2a4445b4bf35a"}


Control endpoint exposure via address space controller config

Always create endpoints for services by default
Add configuration option to address space controller config to remove exposure
Add service host to cert CN

Next step is to allow individual endpoints to be expose.
Fixes #1398

@lulf In the case that addressSpace.getEndpoints() is an empty, I guess there is still the exception from #1398. Although the workaround for #661 wouldn't involve an empty endpoints list anymore, I think it would be good to handle that case - could also be by just logging an error about that being an invalid address space config.

@calohmn Some tests were failing, hoping the next test run will fix that.

@lulf did a test with bash deploy.sh -n hono -t kubernetes -o singletenant -a standard.
There is an issue now with the keycloak-controller not creating the realm. The following block gets repeated in keycloak-controller:
2018-06-27 10:10:58 INFO  KeycloakManager:81 - Actual: [master], Desired: [hono]
2018-06-27 10:10:58 INFO  KeycloakManager:91 - Creating realm hono
2018-06-27 10:10:58 INFO  KeycloakManager:37 - getConsoleEndpoint: checking {name=messaging,host=null,service=messaging,servicePort=amqps,cert={provider=selfsigned,secretNameexternal-certs-messaging}}
2018-06-27 10:10:58 INFO  KeycloakManager:37 - getConsoleEndpoint: checking {name=amqp-wss,host=null,service=messaging,servicePort=https,cert={provider=selfsigned,secretNameexternal-certs-messaging}}
2018-06-27 10:10:58 INFO  KeycloakManager:37 - getConsoleEndpoint: checking {name=mqtt,host=null,service=mqtt,servicePort=secure-mqtt,cert={provider=selfsigned,secretNameexternal-certs-mqtt}}
2018-06-27 10:10:58 INFO  KeycloakManager:37 - getConsoleEndpoint: checking {name=console,host=null,service=console,servicePort=https,cert={provider=selfsigned,secretNameexternal-certs-console}}
2018-06-27 10:10:58 INFO  KeycloakManager:39 - getConsoleEndpoint: found console endpoint {name=console,host=null,service=console,servicePort=https,cert={provider=selfsigned,secretNameexternal-certs-console}}
2018-06-27 10:10:58 INFO  KeycloakManager:53 - getConsoleEndpointStatus: checking {name=messaging,host=null,serviceHost=messaging.hono.svc,port=31729}
2018-06-27 10:10:58 INFO  KeycloakManager:53 - getConsoleEndpointStatus: checking {name=amqp-wss,host=null,serviceHost=messaging.hono.svc,port=32145}
2018-06-27 10:10:58 INFO  KeycloakManager:53 - getConsoleEndpointStatus: checking {name=mqtt,host=null,serviceHost=mqtt.hono.svc,port=30891}
2018-06-27 10:10:58 INFO  KeycloakManager:53 - getConsoleEndpointStatus: checking {name=console,host=null,serviceHost=console.hono.svc,port=32121}
2018-06-27 10:10:58 INFO  KeycloakManager:55 - getConsoleEndpoint: found console endpoint status {name=console,host=null,serviceHost=console.hono.svc,port=32121}
2018-06-27 10:10:58 INFO  KeycloakManager:102 - Address space default console endpoint host not known, waiting

@calohmn I think travis fails on the same thing, will have a look


Add two basic templates (google-native + raw kotlin)

Task
Resolves: #350
Description
Example usage
Now:
pulumi new https://github.com/VirtuslabRnD/pulumi-kotlin/tree/350-add-templates/templates/google-native
Once it's merged:
pulumi new https://github.com/VirtuslabRnD/pulumi-kotlin/tree/main/templates/google-native
Potentially, after moving to VirtusLab GH org:
pulumi new https://github.com/VirtusLab/pulumi-kotlin/tree/main/templates/google-native
With "Deploy with Pulumi" button:

Source:
[![Deploy with Pulumi](https://get.pulumi.com/new/button.svg)](https://app.pulumi.com/new?template=[{template-url}](https://github.com/VirtuslabRnD/pulumi-kotlin/tree/350-add-templates/templates/google-native))

To do

[ ] Decide if we should communicate this now or after we move to VirtusLab GH organization
[ ] Update docs
[ ] (maybe?) Add Deploy with Pulumi buttons (docs)

Suggestions needed

[ ] Should we suffix all these with -kotlin? For example, which name is better google-native-kotlin or google-native
[ ] How should we name the basic (almost empty) template. kotlin vs basic vs default vs empty

(maybe?) Add Deploy with Pulumi buttons (docs)

I like them!

Should we suffix all these with -kotlin? For example, which name is better google-native-kotlin or google-native
How should we name the basic (almost empty) template. kotlin vs basic vs default vs empty

Looking at https://github.com/pulumi/templates/, most of them have a suffix with the language:

On one hand, this doesn't really concern us, since our templates are hosted in this repository (and this repository has kotlin in its name), buuut (as seen in your screenshot) the actual template name right now is google-native with no mention of Kotlin:

I think I would add -kotlin to the name? And then the basic template can stay called kotlin.
Other stuff:

One thing I'm not crazy about in the google-native template is that we're using the same code snippet as in out examples/E2E tests, so this code is duplicated (if we did the same with GCP it would be even worse, since we also have the same snippet in the README). Maybe something more minimal? In the gcp-java template they just do a bucket (https://github.com/pulumi/templates/blob/master/gcp-java/src/main/java/myproject/App.java), maybe we could also do that?
Another concern of mine is that while with the examples it's not a huge issue if we don't use the most up-to-date version of each library, I think it would make sense to do that in the templates? Everything else updates automatically, but these we would have to update manually and I don't love that. I'm not saying that's a deal breaker, because I don't know how to automate it nicely.

Btw, I just realized another thing. Our examples directory uses SNAPSHOT versions of our libraries (and it's not even the most recent ones, since we rely on those gist-hosted subschemas). Each example has a README where we explain to the user how to run this code, but we don't take into account that the code won't actually run with this dependency? I think we need to create a task to address that. Maybe we need to place regular up-to-date versions in our examples and replace the version (or the entire pom.xml) before running E2E tests?


bump requests to resolve compatibility issues w urllib3 v1.24

Coverage remained the same at 60.765% when pulling ac0e9b4b812fea27bbc0be6db8a531b32ea82a62 on castrapel:requests_up into f263be96b8409bf2b089608efb401b1e1917fa09 on Netflix:master.