You have a GPU available, but only faiss-cpu is currently installed.

I am getting this warning then it uses CPU to create indexes.
WARNING! You have a GPU available, but only faiss-cpu is currently installed.
This means that indexing will be slow. To make use of your GPU.
Please install faiss-gpu by running:
pip uninstall --y faiss-cpu & pip install faiss-gpu

Will continue with CPU indexing in 5 seconds...
I have faiss-gpu installed, not faiss-cpu.

Hey, there seems to be an issue with the attribute that we use as a proxy to check whether faiss has GPU bindings or not. This should be purely an annoyance with no actual impact (i.e. the GPU will be used in spite of the message). I'll update this in the next fix release, thank you!

For me, it is more than annoyance since the embedding/index process does not use GPU.

Oh, this is particularly odd then, as the use of faiss in RAGatouille is limited to checking if it has a GPU-related attribute, and the rest is handled upstream 🤔. Could you run pip-freeze to make sure there's no conflicting faiss versions? (It's happened to me before where both faiss-gpu and faiss-cpu were present, and caused some conflicts).
If there's no conflict, it'd be really helpful if you could get your env details (OS, Python version, CUDA version, GPU used, pip freeze) and create an issue on https://github.com/stanford-futuredata/ColBERT about the GPU not being used! Thanks a lot and sorry about the issue.

Hi @bclavie,
Maybe the reason of this problem is that you have faiss-cpu as requirement while ColBERT itself has both faiss-cpu and faiss-gpu as extras.
So probably if you use the same approach for poetry then the user will be able to choose which version of faiss to install.
Thank you.

Even when I remove faiss-cpu and install faiss-gpu via conda it doesn't work but at that point I get "WARNING: faiss must be imported for indexing" so this can't be resolved just by making sure the right package is present.

Hi,
I removed faiss-cpu, installed faiss-gpu, restarted kernel an rerun all the code - it works without warning.
Thank you.

Hey all!
Sorry for the delays, a lot has been going on!
So, to be honest, maintaining faiss in a user-facing lib has been a bit of a nightmare. I've taken a few dives trying to fix it and I can somewhat consistently get it to work on my machine(s), but I'm not seeing a clear path to making it work out-of-the-box for all (or nearly all) users, so I'm trying to move away from it faster than intended, at least for most usecases (small-to-medium collections).
#173 replaces FAISS with pure PyTorch, which should remove all those compatibility issues. It gets results within a very small tolerant of FAISS-kmeans in my tests, but I'd be grateful for any feedback on results accuracy (or even overall functionality) if you have any time to do so!

Love to hear this, I've been really frustrated with how fussy faiss is. I'll try to get this running on the machine I was having trouble with and report back on compatibility. Will also try to get a benchmark set up to compare!

V0.0.8 is out and does bypass faiss in favour of PyTorch kmeans for collections with fewer than 100k documents!
I'd still consider this feature experimental, but it seems to be a pretty solid way to try out PLAID in different environments without figuring out how to make faiss play nice with this environment.
Benchmark results were pretty much identical in the few we tried, but we're very much looking for feedback if your experience doesn't (or does!) match this @RhizoNymph.

@bclavie It works great for me on the machine where faiss has been giving me problems! I wrote a short benchmark script I'm going to set up on the machine that I have faiss working on, will report back with comparisons.


CLIENTS - Whitelists && Walled Gardens

The client whitelists AKA walled gardens section is currently missing.
Needs to be added to the networks / separate section.
Waiting decision internally re. same.

We need to add the following for the time being:

Mac allow / disallow
Walled gardens

Some requests that we have an identifier for the mac address. Unsure.

Done but not merged.


Switching to pre-releases instead of cirq-unstable

Switch to PyPi pre-releases instead of cirq-unstable.
Two major reasons for this change:

in preparation for modules - for which we would have to duplicate all of them if we followed this pattern.
As raised in #3267, cirq-unstable doesn't play well with cirq installations

Fixes #3267.
A resulting state after a test-run for this looks like this on the Test PyPi:

The dev releases will still get the "this is an unstable version" notice for their description.

Thanks @viathor - I'm going to have to put a pause on this as I discovered Craig's comment here, that pointed out that tensorflow and tfq-nightly have the same issue as cirq and cirq-unstable: https://github.com/quantumlib/Cirq/issues/2431#issuecomment-547650606.
Also I need a bit more evidence that in a multi-module setting the pre-releases will play nicely together - i.e. whether I can setup for example pip install --pre cirq-pasqal should bring in the correct latest cirq... I'll convert this back to live if I'm convinced that this is the right way to go.

Isn't Craig's comment further justification for using pre-releases?

Isn't Craig's comment further justification for using pre-releases?

I interpreted it as "if this is okay with tensorflow, it should be okay for us too to live with this" based on the context that he seemed to be "pro separate package".

Oh by the way some more updates on this:

https://www.python.org/dev/peps/pep-0440/#developmental-releases:

Note
While they may be useful for continuous integration purposes, publishing developmental releases of pre-releases to general purpose public index servers is strongly discouraged, as it makes the version identifier difficult to parse for human readers. If such a release needs to be published, it is substantially clearer to instead create a new pre-release by incrementing the numeric component.
Developmental releases of post-releases are also strongly discouraged, but they may be appropriate for projects which use the post-release notation for full maintenance releases which may include code changes.

There is no impact on dependency management whether we use the same package or separate ("unstable") packages. It is straightforward to generate requirements to the given version in the setup.py for each module.

I'm more concerned about the 1st point and playing nice in the python community. More motivating conversations around this:

How PyPi does not allow for retention policy: https://discuss.python.org/t/publishing-nightly-builds-on-test-pypi-org-with-a-time-based-retention-policy/3152
scikiy-learn ended up on anaconda.org as they have the ability to delete packages: https://docs.anaconda.com/anaconda-cloud/user-guide/tasks/work-with-packages/

The only issue I have with anaconda.org is that this is how we'd install modules:
pip install --pre --extra-index https://pypi.anaconda.org/cirq-unstable/simple cirq

pip install --pre --extra-index https://pypi.anaconda.org/cirq-unstable/simple cirq-google

Oh by the way some more updates on this:

https://www.python.org/dev/peps/pep-0440/#developmental-releases:

Note
While they may be useful for continuous integration purposes, publishing developmental releases of pre-releases to general purpose public index servers is strongly discouraged, as it makes the version identifier difficult to parse for human readers. If such a release needs to be published, it is substantially clearer to instead create a new pre-release by incrementing the numeric component.
Developmental releases of post-releases are also strongly discouraged, but they may be appropriate for projects which use the post-release notation for full maintenance releases which may include code changes.

There is no impact on dependency management whether we use the same package or separate ("unstable") packages. It is straightforward to generate requirements to the given version in the setup.py for each module.

I'm more concerned about the 1st point and playing nice in the python community. More motivating conversations around this:

How PyPi does not allow for retention policy: https://discuss.python.org/t/publishing-nightly-builds-on-test-pypi-org-with-a-time-based-retention-policy/3152
scikiy-learn ended up on anaconda.org as they have the ability to delete packages: https://docs.anaconda.com/anaconda-cloud/user-guide/tasks/work-with-packages/

The only issue I have with anaconda.org is that this is how we'd install modules:
pip install --pre --extra-index https://pypi.anaconda.org/cirq-unstable/simple cirq

pip install --pre --extra-index https://pypi.anaconda.org/cirq-unstable/simple cirq-google

Oh hold on - the first point is irrelevant, it is just a strong recommendation around non-human-readable version numbers. Ignore that.
I was for a second also worried about "littering" the main package with dev releases for readability - but maybe it's not that big of a deal. Also, pre-releases like v1.0.0.rc1, v1.0.0.rc2 can still work, just pip install --pre cirq will be shadowed by the dev releases, so one will have to use the specific versions there instead explicitly.
2. In terms of space, I am actually not that worried about it. With 1.6MB per release and ~800 commits per year we are below a 1GB per year.
So the only question remains is readability of different versions on the cirq pypi page...

In the end I am ready to merge this as it will make our (less packages to deal with) and our users lives easier (no weird namespace collisions).
As I understand the per project limit is 10GB which we'll get to in 5+ years if we never clean up a release.
Cleaning up releases is manual at this point - but there is an issue for PyPi that pushes for automated cleanups (https://github.com/pypa/warehouse/issues/8792).
@quantumlib/cirq-maintainers this is a final chance to object :) Please comment, otherwise I'm ready to merge tomorrow!

All in all, I think this is a good idea. We did this for a previous open source project I maintained and it worked well iirc

As we have quorum, I'm going to go ahead and merge it. Thanks all for chiming in!


Encryption of LDAP password

What version of UAA are you running?
UAA version - 3.4.5
How are you deploying the UAA?
I am deploying the UAA

locally only using gradlew

What did you do?
We are trying to hook up LDAP as the authentication source for UAA using the Search and Bind mechanism and are following the documentation to specify the required fields (https://github.com/cloudfoundry/uaa/blob/3.4.5/docs/UAA-LDAP.md)
What did you expect to see? What goal are you trying to achieve with the UAA?
Is there a way to encrypt the password that we are providing in the configuration and more importantly when this data get saved in the Identity_Provider table in the config column, is there any way to store an encrypted password.
The newer versions of UAA also does not seem to mention about encrypting passwords for LDAP.  Any help would be appreciated.

Closing due to inactivity.

Is there any ways available in new versions of UAA to support this option ?

No don’t know any action here


Add tests to ACR Auth Policy for concurrency scenarios

Add a unit test to the collection of auth policy tests to ensure that the correct behavior happens when two messages come through at the same time.

@annelo-msft could you please share more details on what the scenario is here? Looking at the implementation here the parallelization is handled by the BearerTokenAuthenticationPolicy and ACR auth policy is only parsing the jwt token as needed.


Bump versions for pre-commit, pytest-xdist, msal and fastapi[all]

What is being addressed
Bump version for:
pre-commit from 2.21.0 to 3.0.4.
pytest-xdist from 3.0.2 to 3.1.0.
msal from 1.20.0 to 1.21.0.
fastapi[all] from 0.88.0 to 0.89.1.

/test-extended

@LizaShak, Martin has commented on the precommit PR, please review.
Also, please title this PR appropriately.

/test-force-approve

/test-force-approve

/test-force-approve


fix: encode frontmatter

This is just a quick fix for #95.
The warning still exists, but links like /tag/笔记/ won't be 404 anymore.

@billyyyyy3320  Could you please chese this?


Fallback for older browsers

We all hate IE, yet working with Compass means we barely have to think about the poor people chained to the past. This is a wonderful project, but it could learn some older-browser support from fancy-buttons.

Thanks for the feedback, I am working on adding better IE support and I hope to have them implemented soon.

what about porting stuff from fancy-buttons?


If you alt-tab off and in the editor and you drag a sprite it will act like if alt were pressed

Operating system or device, Godot version, GPU Model and driver (if graphics related):
Godot 3.0 master c880302 Windows 10
Issue description:

I didn't press alt the second time I dragged the sprite.
Steps to reproduce:

alt-tab off and in the editor
Drag a sprite into the editor's viewport without pressing alt

I can't reproduce it on Linux. Is this still reproducible on Windows?

Testing on linux mint 18.3.

Can reproduce in 3.0 thru 3.1.2.
If you press and release the Alt key once more after returning to Godot but before you release the mouse, the behaviour is correct. Presumably Godot will only acknowledge the key-up when focused.

Can't reproduce in 3.2, 3.2.3.rc3.

Issue appears to be resolved.

Closing as resolved per @tom-jk's comment.


Some question about the code.

Thanks for your great work! Here I have some questions about the code implementations, hope you can give me some advice:

When I use 'llava-v1.5-7b' or 'llava-lcs558k-scienceqa-vicuna-13b-v1.3' as my loading model, it appears that only one layer that was marked as CLIP layer: using qlora fake activation 4bit, llama layer False, block 32,8, it's that normal?
When I debug the calibrate function, I found that the 'search' step does not perform.
When inference after calibration, I found that all the layers' entropy is greater than 37, does it means that block-search do not performed?

I think maybe I don't use the right model weight file? I would like to ask about which model you load in your paper, looking forward to your reply! Thanks again.

Thank you for carefully reviewing our code.

Regarding the CLIP model's markers, you can refer to the code for modification and adjustment to adapt to different models: https://github.com/ChangyuanWang17/QVLM/blob/f623cd28904f511b89f4deb07f644392db2986f7/custom_bitsandbytes/bitsandbytes/nn/modules.py#L229

The self.search_flag indicates whether the layer is in search mode. The self.search section directly searches for quantization thresholds, which significantly impacts performance and were not utilized in the final version.

Since it is difficult to determine the threshold of layers' entropy for different input datasets, we added self.count_block % 3 == 1 as a lower bound to conduct searches every three layers.

The model we load for LLaVA-v1.3 is Science QA Checkpoints. I hope this can help you.

Thanks for your reply, which help me a lot!

I would like to ask about count_layer and count_block. Is that count_layer == 7 means that one block has six Linear4bit layers?

Apologies for the inconvenience, I just realized that modifying the current version of the code to support W8A8 is quite complex.

First, update generate_sqa_response.sh by adding the --load-8bit flag to enable loading 8-bit LLaMA weights.
Modify the CLIP model to support 8-bit loading in llava/model/multimodal_encoder/clip_encoder.py.
Update the following file to set self.activation_bit = 8 for changing the activation bit width: [link](https://github.com/ChangyuanWang17/QVLM/blob/f623cd28904f511b89f4deb07f644392db2986f7/custom_bitsandbytes/bitsandbytes/nn/modules.py#L214).

We are planning to streamline these modifications in the near future. 😊

Thanks for your reply!!!
And I am still confused about directly setting --load-8bit flag. Because I can not see the function call of Linear4bit when using --load-8bit flag, which is the main modification of your method! And it meets the following error:
AttributeError: 'NoneType' object has no attribute 'cget_col_row_stats'

Does it mean we directly use LLM.int8()? If so, I will wait for your changes in the near future to support QVLM in W8A8!
Thanks again for your patience, which helps me a lot!


Consider separating commits on separate branches in the future.


Support Whatsapp

interactive with Whatsapp

whatsapp.chat("sending message to xxx")
whatsapp.chat("what @xxx just said")

HITL with Whatsapp

pending this feature because of can't setup Whatsapp in Meta developer platform

close this issue due to https://github.com/npi-ai/npi/pull/51 has been supported


React capstone project - Metrics webapp

In this PR the following has been done:

[x] Developed reusable components including Header, Country, City, and Heading.

[x] Implemented a Home page that displays a list of countries that can be filtered by region.

[x] Developed a cities page that dynamically loads a list of cities based on the user's selected country.

[x] Implemented a Details page that fetches weather data from an API and displays it to the user.

[x] Styled the entire application to meet the design need based on the Ballhead App design idea by Nelson Sakwa on Behance.

Presentation Link

@franclobo, thank you so much 🙏


Unique constraints, first working attempt

This implements the uniqueItems constraints.  This PR has a few issues, but since we've been blocked on the uniqueItems feature for a while, I thought I'd submit it to get the ball rolling.
Issues to resolve:

Some of the custom/* tests are failing.  At least a few are related to back-pointers.  This may be the kick I need to start eliminating back pointers.

The way the unique constraints are integrated is a bit of a kludge.  There are a few possible ways to cleanly do this, and we should pick one at some point.

Commit https://github.com/katef/jvst/pull/56/commits/8a4e30e3e44c1a97d7a219dcac6b7229aad39658 fixes the failing custom/* tests.  See the commit message for details.
The way the unique constraints are worked into the VM is still kludgy, though.


Verification with GitHub (account fails)

$ yt build
info: get versions for frdm-k64f-gcc
info: get versions for mbed
info: failed to fetch Github object, re-trying with authentication...

You need to log in with Github.
Press enter to continue.
Your browser will open to complete login.

Somewhere here I've opened webpage and I've used GitHub login button to login in.
After that yotta 'collapsed':
waiting for response...
If you are unable to use a browser on this machine, please copy and paste this URL into a browser:
http://yottabuild.org/?provider=github#login/[redacted]
..Traceback (most recent call last):
  File "C:\Python27\Scripts\yt-script.py", line 9, in <module>
    load_entry_point('yotta==0.3.1', 'console_scripts', 'yt')()
  File "C:\Python27\lib\site-packages\yotta\main.py", line 171, in main
    status = args.command(args, following_args)
  File "C:\Python27\lib\site-packages\yotta\build.py", line 77, in execCommand
    install.execCommand(args, [])
  File "C:\Python27\lib\site-packages\yotta\install.py", line 58, in execCommand
    installDeps(args, c)
  File "C:\Python27\lib\site-packages\yotta\install.py", line 97, in installDeps
    test = {'own':'toplevel', 'all':True, 'none':False}[args.install_test_deps]
  File "C:\Python27\lib\site-packages\yotta\lib\component.py", line 508, in satisfyDependenciesRecursive
    test = test
  File "C:\Python27\lib\site-packages\yotta\lib\component.py", line 323, in __getDependenciesRecursiveWithProvider
    test = test
  File "C:\Python27\lib\site-packages\yotta\lib\component.py", line 223, in __getDependenciesWithProvider
    satisfyDep, specs
  File "C:\Python27\lib\site-packages\yotta\lib\component.py", line 209, in satisfyDep
    update_installed
  File "C:\Python27\lib\site-packages\yotta\lib\component.py", line 494, in provider
    r = access.satisfyVersionByInstalling(dspec.name, dspec.version_req, self.modulesPath())
  File "C:\Python27\lib\site-packages\yotta\lib\access.py", line 290, in satisfyVersionByInstalling
    v = latestSuitableVersion(name, version_required)
  File "C:\Python27\lib\site-packages\yotta\lib\access.py", line 122, in latestSuitableVersion
    vers = remote_component.availableVersions()
  File "C:\Python27\lib\site-packages\yotta\lib\github_access.py", line 262, in availableVersions
    for t in self._getTags():
  File "C:\Python27\lib\site-packages\yotta\lib\github_access.py", line 252, in _getTags
    self.tags = _getTags(self.repo).items()
  File "C:\Python27\lib\site-packages\yotta\lib\github_access.py", line 72, in wrapped
    authorizeUser()
  File "C:\Python27\lib\site-packages\yotta\lib\github_access.py", line 197, in authorizeUser
    if _pollForAuth(registry=registry):
  File "C:\Python27\lib\site-packages\yotta\lib\github_access.py", line 132, in _pollForAuth
    tokens = registry_access.getAuthData(registry=registry)
  File "C:\Python27\lib\site-packages\yotta\lib\registry_access.py", line 664, in getAuthData
    parsed_response = ordered_json.loads(body)
  File "C:\Python27\lib\site-packages\yotta\lib\ordered_json.py", line 29, in loads
    return json.loads(string, object_pairs_hook=OrderedDict)
  File "C:\Python27\lib\json\__init__.py", line 351, in loads
    return cls(encoding=encoding, **kw).decode(s)
  File "C:\Python27\lib\json\decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Python27\lib\json\decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Please could you run the command that causes this error with yotta -vvv <command>, and paste the output into an email to me (do not paste it here, as it will contain secret access keys)
cc @headlessme: looks like a non-json page was returned by the registry when polling for the completed authentication. Not sure how this could happen.

After deleting all dirs created by yotta, no change.
$ yt build
info: get versions for frdm-k64f-gcc
info: get versions for mbed
info: download mbed
info: get versions for mbed-cpputest
info: download mbed-cpputest
info: get versions for mbed-hal
info: get versions for cmsis-core
error: Failed to satisfy dependencies of c:\Work\martin\mbed-example-asynch-serial-private\yotta_modules\mbed:
error: mbed-hal does not exist in the modules registry. Check that the name is correct, and that it has been published.
error: cmsis-core does not exist in the modules registry. Check that the name is correct, and that it has been published.
error: mbed-hal not available
error: cmsis-core not available
error: Missing dependencies prevent build. Use `yotta ls` to list them.

$ yt ls
mbed-example-asynch-serial 0.0.4
|_ mbed 3.0.1
| |_ mbed-hal ~0.1.0 missing
| \_ cmsis-core ~0.1.0 missing
\_ mbed-cpputest 0.0.4

Did you login to the private registry?

@PrzemekWirkus Please see my previous comment.

Added private registry.
Can't reproduce this issue anymore (I can assume this was network error).

All things work well after step 1 & 2 :)


more food for thought

https://github.com/foxdonut/meiosis/blob/d5c8b95d4247c2d79b90e1741c612ecaaf68d56a/helpers/routing/state/index.js#L47-L62
Dear @foxdonut
I have been taking a look at your routing WIP. Array based, programmable routes are 🔥. Here are some thoughts that may make the concept easier to grok:
Initially I had problems wrapping my head around what route and routes mean or refer to in different places of the code.
We have route in state: This can be thought of an array of ...routeSegments that match against components in the view tree and to some extend against url paths as well.
Then there is the route component prop (object), that gets calculated based on a given route array in state: This is where the actual routing or navigation is happening on a per component basis. Confusingly, it contains a routes value which references the original route array.
Then on top of this there are various helper functions where it is hard to tell if they work with the component prop route or the state route / routes or both. (e. g. initRoute and nextRoute produce a prop, while parentRoute, childRoute, siblingRoute produce a state route arry.
I think the confusion may be resolved by re-naming the state prop to something different, like routing or navigation or something similar.
I would also suggest to better differentiate the two functions nextRoute and initRoute that each yield the routing prop from the other functions that yield state route arrays.
Consider the following:
export function Routing (route = [], index = 0) {
  return {
    route,
    index,
    local: route[index] || {},
    child: route[index + 1] || {},
    next () {
      return Routing(route, index + 1)
    }
  }
}

And then in the Root
import { Route, Routing } from "somewhere"
import { Beer, Beverages } from 'somewhere-else'

const componentMap = { Beer, Beverages }

function Root ({ state, actions }) {
  const routing = Routing(state.route.current)
  const Component = componentMap[routing.local.id]

  return

     <Button onClick={() => actions.navigateTo([ Route.Beer(), Route.Beverages() ])}>
        Show list of beers
     <Button>
     <Component state={state} actions={actions} routing={routing} />

}

And then in Beer
import { Beverages } from "../beverages";
import { Beverage } from "../beverage";

const componentMap = {
  Beverages,
  Beverage
}
function Beer ({ state, actions, routing }) {
  const Component = componentMap[routing.child.id]

  return (

Beer Page

      <Component
        state={state}
        actions={actions}
        routing={routing.next()}
        beverages="beers" />

  )
}

I think the mechanism will be much easier to understand just by virtue of better differentiating its moving parts.
this might also help clarify the signatures of the other helper functions that consume the routing prop to yield a new route array. (So that you don't end up referencing a routes when in fact there is only route.
const routing = new Routing([ Route.Beer(), Route.Beverages() ])

function showBeerDetails () {
  actions.navigateTo(siblingRoute(routing, [ Route.Beverage({ id: 'b2' }) ])
}

/**
 *
 * @param {Object} routing - the component routing object / prop
 * @param {Array.<{ id: String, params: Object}>} routing.route - original state route ref
 * @param {Number} routing.index - current route segment reference
 *
 * @param {{ id: String, params: Object}} sibling - route segment to inject into route
 * @param {Array.<{ id: String, params: Object }>} [children] - optional sibling children
 *
 * @return {Array.<{ id: String, params: Object }>} - new state route
 */
function siblingRoute (routing = {}, [ sibling, ...children ]) {
  return routing.route.slice(0, routing.index).concat(sibling, children)
}

A am about to try your new routing mechanism with an app of mine where I have to deal with ledgers, that contain accounts, and accounts that contain transactions, and transactions that have lots of transaction details and reference different accounts each. So this will be awesome!

Once again @smuemd your feedback is very, very helpful and I appreciate it greatly.
It's great to hear that you like the array-based routing, programmable routes, and relative navigation. It's been something I've worked on for a long time, with a handful of unsatisfactory solutions before I finally arrived at this which I think has potential.
Indeed the names needed work (Naming Is Hard) but you have saved me some effort with your suggestions. I really like them and will incorporate them into the code.
As you may have already seen, I am also working on moving the utility/reusable parts of the routing code into helper functions.
For actual path/router logic,My goal is that all you need in your application is to define your routing config and to plug in the router of your choice. I'm experimenting with feather-route-matcher as a proof-of-concept that you really only need a simple router. I still plan to also provide some support for Mithril's router since support for it is often requested.
Thank you and keep the feedback and suggestions coming, they are :1st_place_medal:

It's been something I've worked on for a long time, with a handful of unsatisfactory solutions before I finally arrived at this which I think has potential.

Yes, I've been silently following your commits into the docs/code section for a while and aware of the various iterations. Routing in SPA in real hard. But this is coming together in a really nice way.

My goal is that all you need in your application is to define your routing config and to plug in the router of your choice. I'm experimenting with feather-route-matcher as a proof-of-concept that you really only need a simple router.

Actually, I still have some meiosis routing code for url-mapper lying around. I will rework this to into another drop-in example based on your new routing helpers and send it over. (probably next weekend or so). url-mapper is nice for someone who needs extra special powers for query strings. The feather matcher is 5 x lighter in code size though, so probably a better first choice example.

@smuemd as per your suggestions I now have (in routing-common-core and only the navigation example for now):

Routing for the component prop
a route is an array of route segments
findRoute changed to findRouteSegments
createRoute changed to createRouteSegments. In the example though I still assigned the result to Route because [Route.A(), Route.B()] is still nicer than [RouteSegment.A(), RouteSegment.B()]. Hopefully that slight inconsistency is acceptable.
Besides next(), I also added parentRoute, childRoute, and siblingRoute to Routing. Less things to import.
Also in Routing, I changed local and child to localSegment and childSegment for clarity.

I'm glad you mentioned url-mapper, I was also thinking of using it as another "plug in your router" example.


visualization.query functions concerning date incorrect

While looking for the bug causing #1043 I saw two functions concerning date fields which look suspicious:
https://github.com/UUDigitalHumanitieslab/I-analyzer/blob/e14bed8d6b0b13b3691a1bc59259df63fc2dab51/backend/visualization/query.py#L134
https://github.com/UUDigitalHumanitieslab/I-analyzer/blob/e14bed8d6b0b13b3691a1bc59259df63fc2dab51/backend/visualization/query.py#L148
This setup would break the functionality in ngram.get_total_time_interval to narrow down the time interval for corpora whose date fields are not called 'date'.

Yup, that's a mistake.
This should look for a range filter that specifies a format, e.g.:
{
    'range': {
        _: {
           'format': ...
           ...
        }
    }
}


preemptive Ransac

Hi,
  did you use preemptive ransac for this code? thanks a lot!

Yes, the RANSAC implementation is the same from Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images, which uses an adapted preemptive RANSAC (Algorithm 1).
This code is a direct implementation of that paper.


Add step by step instructions to install kubectx and kubens in linux

Includes:

Steps to install kubectx and kubens with an alias
Steps to add zsh completions

Sorry but this doesn’t look good.
I have never seen anyone create an alias for a one-off command.
It’s unmaintainable and unidiomatic.


different custom configurations for multiple feign clients support issue

Question:
For  handler one content-type(txt/plain) with two kind of response body (xml and json), we build two configurations like below:
public class WeChatPayServiceFeignClientConfig {
	@Bean(name = "weChatPayDecoder")
	public Decoder weChatPayDecoder() {
		return new WeChatPayDecoder();
	}
}

and
public class WeChatMiniAppUserServiceFeignClientConfig {
	@Bean(name = "miniAppUserDecoder")
	public Decoder miniAppUserDecoder() {
		return new MiniAppUserDecoder();
	}
}

It works well after build runable jar in windows, but the HystrixRuntimeException will be happen after I build it in linux and Mac, core error message like below:
feign.codec.DecodeException: Could not extract response: no suitable HttpMessageConverter found for response type [class com.hehe.washTruck.wechat.miniapp.model.WxMiniAppJscode2SessionResult] and content type [text/plain]
we have the seem JDK and mvn like below:
JDK:
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

MAVEN：
'''shell
Maven home: D:\michael\dev_app\maven\apache-maven-3.5.4\bin..
Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: D:\michael\dev_app\java\jdk1.8.0_181\jre
Default locale: zh_CN, platform encoding: GBK
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
'''
the spring-boot version is 1.5.10.RELEASE
and the spring-cloud version is Brixton.SR5
experiment:
The  issue also will be happen when I remove the configuration flag in @FeignClient in window

Please open an issue in the correct repo https://github.com/spring-cloud/spring-cloud-openfeign
You should use either Edgware.SR5 or Finchley.SR1.
Please learn how to format code on GitHub.


t-nettabrada/ddos

Required items, please complete
Change(s):

added analytic rules and master playbook to the Azure DDoS Protection Solution
Reason for Change(s):

enriching the solution
Version Updated:

2.0.0

Testing Completed:

Yes

Checked that the validations are passing and have addressed any issues that are present:

Yes

@netta11 : Please fix Validation errors

@netta11 : We had internal discussion with anki and he suggested that masterplaybooks will be go for seperate solution.
Now can you please exclude masterplaybooks from your package. Thanks!!

@v-spadarthi Please check the package and go ahead with other required things.
Thanks

I have tested CreateUI definition

Main Template also looking good


Suggestion: user defined aggregations for df.groupby

Any interest in user defined aggregations for DataFrame.groupby(...) and Series.groupby(...)? I'd be willing to implement it myself, but wanted gauge general interest before starting.
As an API, I envision offering the possibility of passing objects with attributes chunk, aggregate, combine, and split_every. Such objects could be given as arguments inside the aggregate spec.
Their semantics would follow DataFrame.reduction(..). For example:
first_value = dd.reduction(chunk=lambda s: s.iloc[0], aggregate=lambda s: s.iloc[0])

first_value_of_by_g = ddf.groupby(['g']).aggregate({'c': first_value})

The major issue I see here, that this would fix the current implementation of aggregate in terms of the steps chunk, aggregate, combine.
edit: come to think about it, the functions most definitely would need to be able to return dataframes to allow for keeping more than a single value of state, e.g., for mean.

The relevant internal API here is GroupBy._aca_agg.  It might be worth looking at how GroupBy.sum and friends are currently implemented.  Perhaps there is a way to reveal those functions nicely to users.

I had a look at the current .aggregate implementation - since I'm somewhat familiar :) - which also internally uses aca. Since I take it you are not generally opposed to the idea, I'll prototype an implementation and to have something to discuss on.

Great.  Thank you for taking this on.
On Tue, May 16, 2017 at 12:32 PM, Christopher Prohm <
<EMAIL_ADDRESS>wrote:

I had a look at the current .aggregate implementation - since I'm
somewhat familiar :) - which also internally uses aca. Since I take it
you are not generally opposed to the idea, I'll prototype an implementation
and to have something to discuss on.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/2340#issuecomment-301838959, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASszBVMe7ZHPGoBFgsvQB0jaqthdBA_ks5r6c-kgaJpZM4NcEBI
.

Closed by #2344


Order History page

Page functionality in place. Needs to be tested once we have the ability to place an order.

done


Create MySQL VM

The Image is created, but no way to run it on the school linux computers, so we decided to just use the remote hosted one that Jordan set up


Wildcard not matched, instead simply outputted in template

Hi,
I have a setup very similar to first example in the docs

File calledpreflight.js exists and that is not (and should be not) added in webpack entry.
It is copied to webpack configs output path using copy-webpack-plugin. On the way it receives hashing for filename itself which will change only when contents change.
I am using html-webpack-plugin to generate template, i this example auto injecting default one.
Accordingly to docs I specify minimatch ready string preflight.*.js which would match stuff in webpack's output path (in my case path: path.join(__dirname, 'public/assets'))

I tried both

**/*preflight.*.js
preflight.*.js

My test config is

// this copies file from 'src/preflight.js to public/assets/preflight.hash.js
config.plugins.push(new CopyWebpackPlugin([
  {
    from: 'src/preflight.js',
    to: `[name].[hash].[ext]`
  }
]));

config.plugins.push(new HtmlWebpackPlugin({
  title: `TEST`,
  filename: `${path.join(__dirname, 'public')}/index.html`,
  inject: true,
  hash: false,
  cache: true,
  showErrors: true,
  chunksSortMode: 'auto',
  excludeChunks: [],
  xhtml: false,
  alwaysWriteToDisk: true,
  minify: false
}));

config.plugins.push(new HtmlWebpackHarddiskPlugin());

config.plugins.push(new HtmlWebpackIncludeAssetsPlugin({
  assets: [
    'preflight.*.js'
  ],
  append: false,
  hash: false
}));

Template output (prod build, not devserver) is

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8">
    <title>TEST</title>
  <link href="//webpacktest-codesplitting.test/assets/site.3c550d0e6ef1b0cc4345.css" rel="stylesheet"></head>
  <body>
<script type="text/javascript" src="//webpacktest-codesplitting.test/assets/preflight.*.js"></script><script type="text/javascript" src="//webpacktest-codesplitting.test/assets/manifest.27fead361ac3388ca95d.js"></script>
<script type="text/javascript" src="//webpacktest-codesplitting.test/assets/vendor.d52f339f6cf1da9e710d.js"></script>
<script type="text/javascript" src="//webpacktest-codesplitting.test/assets/site.3c550d0e6ef1b0cc4345.js"></script>
</body>
</html>

Whatever I put into assets key is outputted as is within template. As you can see from sourcefile name I am testing code splitting 😄 . Disabling that does not change anything.
What am I missing? Did I overlook the docs and there is no actual minimatching against real files in filesystem that are copied over to output path?
Thanks!

software
version(s)

node
8.9.4

npm
5.6.0

Operating System
macOS 10.13.2

Webpack
3.10.0

copy-webpack-plugin
4.3.1

html-webpack-plugin
2.30.1

html-webpack-harddisk-plugin
0.1.0

html-webpack-include-assets-plugin
1.0.2

Hi,
I revisited this issue - my bad.
I had the assumption that if strings are supplied to assets, then matching will also be done. However it is done only when objects are passed to assets.
Thus to make it work I use FileManagerPlugin to copy files to webpack's output path. It does it onStart. In my example src/preflight/preflight.{js|css} is copied to public/assets/preflight.123.{js|css} (public/assets/ is webpack's output path).
And instead of trying to match those files in output path using
config.plugins.push(new HtmlWebpackIncludeAssetsPlugin({
  assets: [
    'preflight.*.js',
    'preflight.*.css'
  ]
}));

the way I did in original post https://github.com/jharris4/html-webpack-include-assets-plugin/issues/19#issue-291075864 and later comment https://github.com/jharris4/html-webpack-include-assets-plugin/issues/19#issuecomment-360158998
I have to do
config.plugins.push(new HtmlWebpackIncludeAssetsPlugin({
  assets: [
    {
      path: '',
      glob: 'preflight.*.js',
      globPath: outputPath
    },
    {
      path: '',
      glob: 'preflight.*.css',
      globPath: outputPath
    }
  ]
}));

outputPath is webpack's output directory.
It works.

And FYI. I can use it only for case where

Have files copied (not built / included in webpack runtime) to output directory
Have them referenced in built HTML using HtmlWebpackPlugin

The other need

Have files copied (not built / included in webpack runtime) to output directory
Have them inlined in built HTML using HtmlWebpackPlugin

will not work. script-ext-html-webpack-plugin by having
config.plugins.push(new ScriptExtHtmlWebpackPlugin({
  inline: [
    /preflight.*.js$/
  ]
}));

it will throw ERROR in   Error: ScriptExtHtmlWebpackPlugin: no asset with href 'preflight.123.js'
Which is funny because it knows that the file is called 'preflight.123.js' so something is matched.
script-ext-html-webpack-plugin is the plugin I use for inlining manifest/runtime in HTML when splitting code using CommonsChunkPlugin and it works.
Some bug or what https://github.com/numical/script-ext-html-webpack-plugin/issues/13
Yes, I've tried switching plugin order.
My current sollution if I want to

Have files copied (not built / included in webpack runtime) to output directory
Have them inlined in built HTML using HtmlWebpackPlugin

is to have custom keys in HtmlWebpackPlugin that have values of contents I want to inline
config.plugins.push(new HtmlWebpackPlugin({
  fsInlineContents: {
    'preflight.js': fs.readFileSync(path.join(__dirname, 'src/preflight/preflight.js'), 'utf8'),
    'preflight.css': fs.readFileSync(path.join(__dirname, 'src/preflight/preflight.css'), 'utf8')
  }
}));

and then in template I echo the contents out by doing
<script><%= htmlWebpackPlugin.options.fsInlineContents['preflight.js'] %></script>
<style><%= htmlWebpackPlugin.options.fsInlineContents['preflight.css'] %></style>

Yeah, it gets tricky when you're dealing with assets that are being copied to the webpack output path. You need to make sure those assets are copied before htmlWebpackPlugin starts processing things or else you start to get unexpected results...
Thanks for documenting your findings, and for the suggestions of using htmlWebpackPlugin.options.fsInlineContents or ScriptExtHtmlWebpackPlugin.
I'll take a look at those the next time I have assets that I want to copy and include.

note that fsInlineContents is my made up key name, it can be anything that does not clash with built in keys.


Add Docker Support

This is an enhancement or feature.

Summary
I've added docker support.

Context
With this commit, you no longer have to install ruby and run bundle install and install all dependencies if you want to run the image locally. You can just install docker and run:
docker-compose up

It will automatically download jekyll image and then install all required dependencies in an isolated environment and run the website from there.
I've already tested this and it is working as expected.

Seems like a good idea. However the benefit for a bundled Docker setup is too narrow as it's only useful for theme development, so I'm not considering this PR for now.


consolidated the bootstrap in its own file

This PR would make it more easy for modules to have their "own" applications (thinking of CLI apps).
This would be useful in for instance MistDoctrine (https://github.com/mstaessen/zf2-doctrine) or SpiffyDoctrine (https://github.com/SpiffyJr/SpiffyDoctrine). A submodule cannot rely on the locations of files. In this way, only bootstrap.php has be present and the bootstrap can set up the (console) application.

So we had a discussion on IRC about this PR and came to the conclusion that this might not be the best approach. I believe in the case of modules, this would still require some CLI entry point in the module which does something ugly like require_once(__DIR__ . '/../../../public/bootstrap.php');.
Really nothing in the module should be "reaching" into the application, because in theory, a module should not have any knowledge of where it is in relation to the application. For example, if I am in /usr/share/zf2/modules/SpiffyDoctrine/bin and I run ./doctrine orm:schema:create --dump-sql, then how would it locate the bootstrap.php to require? Maybe there are 5+ applications using this copy of SpiffyDoctrine. So ideally it would seem we need a CLI entry point at the application level that can then be pointed to a specific CLI handler in some module. That's still up in the air, as we haven't really figured out what's going on with the ZF2 CLI stuff yet.

Right, I did not think of cases where you would symlink from multiple places to the same module.
Then this would not work, I agree.
Can't we add an interface like AutloaderProvider to provide CLI commands in an entry point CLI. Suppose we would have a default CLI app in project_folder/bin. This application could iterate over all modules and add CLI commands provided by modules. This way, modules can provide CLI commands and modules don't need access to the bootstrap because it is provided by the base CLI application.

Right, that's pretty much what the consensus was after the discussion on IRC yesterday. Right now though, we really need someone to take on the CLI stuff so we can move forward with this. @robertbasic and @zucchi were working on it, but both of them have become busy and development on the CLI has stalled. There's an RFC if you or anyone else is interested in taking this on. I'll go ahead and close this PR for now, since there's no disagreement that an alternative solution is needed. :)


Alert opened with { cancelable: true } option isn't cancelable on iOS

Description
Alert on iOS isn't cancelable by tap outside
React Native version:
System:
OS: macOS 10.15.6
CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
Memory: 362.24 MB / 16.00 GB
Shell: 5.7.1 - /bin/zsh
Binaries:
Node: 14.15.4 - /usr/local/bin/node
Yarn: 1.22.10 - /usr/local/bin/yarn
npm: 6.14.10 - /usr/local/bin/npm
Watchman: 4.9.0 - /usr/local/bin/watchman
Managers:
CocoaPods: 1.10.1 - /usr/local/bin/pod
SDKs:
iOS SDK:
Platforms: iOS 14.3, DriverKit 20.2, macOS 11.1, tvOS 14.3, watchOS 7.2
Android SDK:
API Levels: 29, 30
Build Tools: 29.0.2, 30.0.3
System Images: android-29 | Google APIs Intel x86 Atom_64, android-29 | Google Play Intel x86 Atom, android-30 | Google APIs Intel x86 Atom
Android NDK: Not Found
IDEs:
Android Studio: 4.1 AI-201.87<IP_ADDRESS>53283
Xcode: 12.3/12C33 - /usr/bin/xcodebuild
Languages:
Java: 15.0.2 - /usr/bin/javac
npmPackages:
@react-native-community/cli: Not Found
react: 16.13.1 => 16.13.1
react-native: ~0.63.4 => 0.63.4
react-native-macos: Not Found
npmGlobalPackages:
react-native: Not Found
Steps To Reproduce
Provide a detailed list of steps that reproduce the issue.

Open Alert with { cancelable: true } on iOS
Tap outside

Expected Results
Alert should close
Snack, code example, screenshot, or link to a repository:
https://snack.expo.io/@ayubov/39663a

Hi @ayubov, currently cancelable is an Android only option, as you can see in docs:

https://reactnative.dev/docs/next/alert#android


AuthCodeExchangeError

Type: Feature Request
Not able to connect with the salesforce org and unable to set up proxy settings.
VS Code version: Code 1.84.2 (1a5daa3a0231a0fbba4f14db7ec463cf99d7768e, 2023-11-09T10:51:52.184Z)
OS version: Windows_NT x64 10.0.22621
Modes:

Where is this AuthCodeExchangeError shown? What steps are you taking that end up not working?

Where is this AuthCodeExchangeError shown? What steps are you taking that end up not working?

When I try to connect the Salesforce Org to VS Code using private LAN then I'm not able to connect salesforce org to VS Code.

This sounds like a problem with your company's setup. When I search the web for AuthCodeExchangeError, some of the top hits are from Salesforce forums, please check these.


Update to v4 of package format

When adding this to my Package.swift manifest, using swift-tools-version:4.2, when I update, I get the following warning:
warning: PackageDescription API v3 is deprecated and will be removed in the future; used by package(s): ColorizeSwift

The documentation for how to add this as a package also appears to use v3 format.
Can this all be updated to v4 please?

how yourPackage.swiftlooks like? Is this code working fine following version properly?

let package = Package(
    name: "YourApp",
    products: [
        .executable(name: "tool",  targets: ["tool"])
],
    dependencies: [
        .package(url: "https://github.com/mtynior/ColorizeSwift", from: "1.0.0")
    ],
    targets: [
        .target(name: "tool", dependencies: ["ColorizeSwift" ])
    ]
)

That looks right for updating how to add it as a package; the manifest of the package itself also apparently needs to be updated to v4.

Base on this, the library should be compatible with the minimal version of the v4 manifest.

Strange. It's still giving me the warning about the manifest still being in v3.

Maybe it is complaining about the code. The library has not been updated to Swift 4.x.

Hm. Possibly.

Right now I don't have time to investigate this issue, maybe after Christmas. If this is something you can solve on your own, I'm more than happy to merge the pull request.

I'll see what I can do

Base on this, the library should be compatible with the minimal version of the v4 manifest.

@mtynior I think this is just the requirement for the manifest to be valid, but in order to use the package as a dependency, other fields must be specified.
I did some tests and found that this format works for the Package.swift file (tested using swift build and using the package as a dependency in another package):
// swift-tools-version:4.2
import PackageDescription

let package = Package(
    name: "ColorizeSwift",
    products: [
        .library(name: "ColorizeSwift", targets: ["ColorizeSwift"]),
    ],
    targets: [
        .target(name: "ColorizeSwift", path: "Source"),
    ]
)

(Running Swift 4.2.1, macOS 10.14.2)


Python notebook show httpyac error

Hello,  @AnWeber
When I open python notebook *.ipynb file to edit, show the error message:

When I disable httpyac plugin, the message will be gone. Maybe this is a bug.

Can you give me a source code file to reproduce the error? I suspect that the Markdown parsing is to blame.

@AnWeber 👍

There is a fun workaround. Installing Httpbook should fix the error. Then a method from vscode-httpyac is overridden, which fixes the error. But with the next release it is directly in vscode-httpyac

@AnWeber 👍

fixed error with v5.1.0 (released today)


Missing ephemerides if no files on import time

Original comment by @Fedor-Lyanguzov on https://github.com/poliastro/poliastro/issues/126#issuecomment-230373112:

By the way, i've tried "Going to Mars with Python using poliastro" locally, it works, but not on first run. On first run code downloads kernel "de421", but do not make it the default kernel (ephem.default_kernel), because selecting default kernel is done on ephem import.

Steps to reproduce:
>>> import poliastro.ephem
No SPICE kernels found under ~/.poliastro.
Please download them manually or using

  poliastro download-spk [-d NAME]

to provide a default kernel, else pass a custom one as
an argument to `planet_ephem`.

>>> poliastro.ephem.download_kernel("de421")
Downloading de421.bsp from http://naif.jpl.nasa.gov/pub/naif/generic_kernels/spk/planets/, please wait...
Not Found
Downloading de421.bsp from http://naif.jpl.nasa.gov/pub/naif/generic_kernels/spk/planets/a_old_versions/, please wait...

>>> from astropy import time
>>> epoch = time.Time("2015-05-09 10:43")
>>> from poliastro import ephem
>>> r, v = ephem.planet_ephem(ephem.EARTH, epoch)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-df0835788b33> in <module>()
      2 epoch = time.Time("2015-05-09 10:43")
      3 from poliastro import ephem
----> 4 r, v = ephem.planet_ephem(ephem.EARTH, epoch)

/home/juanlu/.miniconda35/envs/py36test/lib/python3.6/site-packages/poliastro/ephem.py in planet_ephem(body, epoch, kernel)
    130
    131     """
--> 132     r, v = kernel[0, body].compute_and_differentiate(epoch.jd1, epoch.jd2)
    133     r = r * u.km
    134     v = v * u.km / u.day

TypeError: 'NoneType' object is not subscriptable


Swipe left and right through posts.

It would be very nice to swipe left and right to go forward and backward in the available posts, when reading an individual post on touch screen.

Hey @digitalextremist. Thanks for taking the time to open the issue and having a suggestion for Ghost! This could be a pretty awesome feature. If you wouldn't mind, could you put your suggestion over on http://ideas.ghost.org/? We like to keep GitHub for current bugs or issues being worked on. Thanks!

Will do! Sorry about that, I'm new to Ghost. Won't happen again.


Invalid JWT triggers HTML HTTP 500 error response with JWK verification

If JWK verification is used (JwkVerifyingJwtAccessTokenConverter) and an invalid token is sent along (i.e. a rubbish token that doesn't even decode) then the error response is HTML with a HTTP 500 error code.
This is due to line 121 within the JwkVerifyingJwtAccessTokenConverter:
L119 // Verify signature
L120 SignatureVerifier verifier = jwkDefinitionHolder.getSignatureVerifier();
L121 Jwt jwt = JwtHelper.decode(token);
L122 jwt.verifySignature(verifier);

A decoding exception is not handled here using an InvalidTokenException.

Ah, I was not 100% exact in my description.
The token is not complete rubbish, you can trigger the problem bei taking a valid token (may also be an expired one) and breaking either the payload or the signature part by removing one or more of the Base64 string characters. Then you'll run into the exception at the point in code I described and it'll generate the HTML response.
It might be not the biggest of all issues, but we found this out when checking security event logging using both valid tokens and broken ones.

@andreas-schilling It's still not clear to me on how to reproduce this issue.

you can trigger the problem by taking a valid token and breaking either the payload or the signature part by removing one or more of the Base64 string characters

What exactly do you mean as far as breaking either the payload or signature part from a valid token?
Are you able to provide a test that reproduces this issue or sample JWT?

I'll try to set up a small app with tests that shows what I mean. Might not be able to do so before next week.

@andreas-schilling I'm closing this due to lack of feedback re: sample


[BUG] Browser charts are separating charts that differ by v

Describe the bug
Populations that differ by one v used to be on 1 chart.
To Reproduce
URL: https://staging.datacommons.org/browser/dc/p/he31kkfcbf5w2
Screenshots
If applicable, add screenshots to help explain your problem.

It doesn't seem to always happen: e.g. this is the same stat var from the same dataset:

Closing this as the browser has switched to use the stat var hierarchy widget to show one chart per stat var.


Add ability to turn off rate limiting

I'm using my own MusicBrainz server, and it would be nice to turn off rate limiting. Maybe we could add a new field to the config called disableRateLimiting and then add some conditional logic to prevent it from running? or maybe we could allow users to set their own values passed into RateLimitThreshold. I think the first option makes more sense as it's simpler, and I can't think of reasons why users would want to slow down requests or speed them up (since they'd hit the limit).

Also, thank you for creating this library!

Good to hear you appreciate this library.
No problem to add an option disable the rate limiter.

Awesome. i'll do that soon

Could I get permission to push a branch?
git push --set-upstream origin rate-limit-disable
Alias tip: gp --set-upstream origin rate-limit-disable
remote: Permission to Borewit/musicbrainz-api.git denied to ben-obringer.
fatal: unable to access 'https://github.com/Borewit/musicbrainz-api.git/': The requested URL returned error: 403

Hi,
Thank you for implementing this feature.
You need to fork this repo and create a new branch in your fork. From there you can create a pull request.


Request for pre-trained model

Hi authors, I'm currently doing some research in twitter bot detection for a course project. I came across your BotRGCN project from your Twibot-22 page, and got quite interested in it. Could I ask if you would kindly share the pre-trained model for BotRGCN on twibot-22? I just finished my own training and wish to have a reference using the pre-trained model if possible~
Thanks very much for your great work!

@leopoldwhite can you help with this?

Hi authors, I'm currently doing some research in twitter bot detection for a course project. I came across your BotRGCN project from your Twibot-22 page, and got quite interested in it. Could I ask if you would kindly share the pre-trained model for BotRGCN on twibot-22? I just finished my own training and wish to have a reference using the pre-trained model if possible~
Thanks very much for your great work!

Thank you for your interest in our work. Here is the state_dict() of our BotRGCN model trained on Twibot-22.

Hi Authors, I could not understand "def tweets_preprocess" that it refers to a file called "tweets.npy" which I could not find in data files. I am trying to reproduce the results as part of my class in master degree and thank you for your help in advance.

Hi Authors, I could not understand "def tweets_preprocess" that it refers to a file called "tweets.npy" which I could not find in data files. I am trying to reproduce the results as part of my class in master degree and thank you for your help in advance.

Hi @keremyucel83, tweets.npy is generated after the first execution of the tweets_preprocess() and is used in the tweets_embedding(). If you set process=True in https://github.com/BunsenFeng/BotRGCN/blob/eabe7fdfc73019144ce3ea50edbd78dd24857031/main.py#L13
tweets.npy  should be generated and be saved to the ./Data directory.

Hi @leopoldwhite
It is totally my bad , in fact I set process as True but root was not set properly and output of npy was named as datatweets.npy. I would like to thank you so much for your quick help and answer.

Hi Authors,I have a question about the dimension of pre-training results. I see that the dimension used in your code is six dimensions, but in the process of modulation and the parameters of the model are set to five dimensions.Thank you for your patience and cooperation.
datasets.py- 251            num_prop=torch.cat((followers_count.reshape([229580,1]),friends_count.reshape([229580,1]),favourites_count.reshape([229580,1]),statuses_count.reshape([229580,1]),screen_name_length_days.reshape([229580,1]),active_days.reshape([229580,1])),1).to(self.device)

Hi Authors,I have a question about the dimension of pre-training results. I see that the dimension used in your code is six dimensions, but in the process of modulation and the parameters of the model are set to five dimensions.Thank you for your patience and cooperation.
datasets.py- 251 num_prop=torch.cat((followers_count.reshape([229580,1]),friends_count.reshape([229580,1]),favourites_count.reshape([229580,1]),statuses_count.reshape([229580,1]),screen_name_length_days.reshape([229580,1]),active_days.reshape([229580,1])),1).to(self.device)

Hi @naneral, the pre-processing results and pre-trained model weights  we provide are based on the Twibot22 format of Twibot20, where some of the numerical features and categorical feature are not reserved. More details here.

Thank you for you help!


fix: liveness probe

Fixes liveness probe to avoid getting killed prematurely.
RHICOMPL-3152

@vkrizan  oh , that could explain it :D


Support for OEM-built apps

Hello,
I'm an owner of a Visortech garagedooropener which has to be connected via a different application. The app-name is "ELESION" and it looks almost the same like the app "Tuya Smart". Even the QR-Code-button (for registering the Devices from the app to Tuya IoT Platform-Project) looks the same.
Tuya App

ELESION

When I try to scan the QR-Code it says that this is not the designated app.

I wonder if it can be possible to get it working with the ELESION app or if the only possibility is to use the other app with the devices (I'm not sure if the Visortech-device is working with the other app).

Hi @dominikjas , thanks for the feedbacks! We are planning to support Tuya OEM app login in Tuya HA integration (OEM Tuya HA Integration Service), which can probably satisfy your requirements. Could you help to provide the following details and email to<EMAIL_ADDRESS>for further discussion?

Your company brief introduction (Website, products, etc)
The product list you would like to support in the OEM Tuya HA integration, you check the Supported Device Category for reference.
The scenario of using Tuya HA integration
The customization requirements for Tuya HA integration (Login interface, integration features, etc)

Looking forward to your feedbacks!


Fix when clustering with dim specified

Was receiving error says import tables failed due to no existing module. Doing research I found that the tables module is an optional install via pandas. Adding the install to the requirements fixed this for me.

Hmm, when are you hitting this @zelein? IIRC I did have this in the requirements but it seemed to cause trouble for me in some cases. Perhaps I had borked my virtualenv or something like that.

Whoops, meant to document that. I ran into this when I tried run_experiment.py --plot


Privacy issue with point 4 ?

Regarding point 4 and the manager having access to his former team member onedrive, aren't you afraid of potential privacy issues ? Even when strictly related to work, I guess that some documents such as chat history could be considered as private.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 762932d1-8eef-ea51-c51d-a40ed88b8237
Version Independent ID: d0ef71ba-21b3-1352-85b5-63d7882782da
Content: OneDrive retention and deletion
Content Source: OneDrive/retention-and-deletion.md
Service: one-drive
GitHub Login: @kaarins
Microsoft Alias: kaarins

This will largely depend on your local privacy laws/company privacy policy.
However, this repo is for documentation bugs. I would recommend closing this particular issue :-)

This will largely depend on your local privacy laws/company privacy policy.
Yes agree. Maybe a warning should be added to this feature ?

However, this repo is for documentation bugs. I would recommend closing this particular issue :-)
Well, I didn't intended to report as a bug, I just used the "leave feedback" button on the page, and that's what happened.

What you could do is leave feedback on Uservoice - https://sharepoint.uservoice.com/forums/330318-sharepoint-administration.
The feedback button on the docs is specifically for feedback on the documentation itself rather than the product.
Thanks!

Thanks, so closing it.


tf 1.4 and tf1.5 and tf1.6

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):  No

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Mac OS

TensorFlow installed from (source or binary):
pip install from binary

TensorFlow version (use command below):
1.4 and 1.6

Python version:
2.7

Bazel version (if compiling from source):
N/A

GCC/Compiler version (if compiling from source):
N/A

CUDA/cuDNN version:
N/A

GPU model and memory:
N/A

Describe the problem
I had a model trained (with multiple days) using tf1.4, and I want to load it in tf1.6.  and the error occurs as below:
File "/Users/iyukun/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2427, in set_shapes_for_outputs
return _set_shapes_for_outputs(op)
File "/Users/iyukun/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2400, in _set_shapes_for_outputs
shapes = shape_func(op)
File "/Users/iyukun/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2330, in call_with_requiring
return call_cpp_shape_fn(op, require_shape_fn=True)
File "/Users/iyukun/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 627, in call_cpp_shape_fn
require_shape_fn)
File "/Users/iyukun/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 691, in _call_cpp_shape_fn_impl
raise ValueError(err.message)
ValueError: Dimensions must be equal, but are 900 and 600 for 'dynamic_seq2seq/decoder/decoder/while/BeamSearchDecoderStep/multi_rnn_cell/cell_1/basic_lstm_cell/MatMul' (op: 'MatMul') with input shapes: [?,900], [600,1200].
Looks like the MatMul op is changed from float16 to float32 in tf 1.5+?  If so,  is there a way to load the model in tf1.5+?

Nagging Assignee @tatatodd: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 64 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 79 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 95 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 110 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 125 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 140 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 155 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 170 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 185 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 200 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 215 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 230 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 245 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @tatatodd: It has been 260 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Hi @iyukuni !
It seems you are using older versions(1.x versions) of Tensorflow. We recommend that you upgrade  your code base to 2.6  versions as many features and bug fixes has been done in newer versions and create a new  issue still persists in newer versions. Please follow this thread for installation in Mac.Thanks!

Are you satisfied with the resolution of your issue?
Yes
No


OCIO UI/UX standards

As discussed in today's TSC meeting, the OCIO v2 stabilization period provides a good opportunity to discuss and agree upon standards and recommendations around UI/UX considerations while implementing OCIO v2 in host applications. Consistent UX will be really important particularly with many new UI-specific features.
This issue is intended to be an initial home for this discussion, including suggestions and mockups for how the color management experience can be improved through consistent OCIO v2 UX. #1117 could also be a good opportunity to test some of these ideas. If you have opinions on this topic, please share and present your ideas. Ultimately we would like to consolidate the outcome of this conversation into OCIO documentation with visual examples if possible.

Only one note here, the display linear EOTF transform to finish off the display colourimetry side would be invaluable for DCC compositing.


Restricting the action space for Fetch

I am trying to train a navigation policy using PPO on fetch. But during the execution of the policy the arms move as well. This is due to the control limits for the arms being passed in the action space "Box" object. How do I remove the arm control limits from this "Box" object so that PPO does not have to deal with moving the arms?

Were you using a .yaml config file? You can remove the controllers you don't want under "controller_config", which is where you decide the action space. For example, any InverseKinematicsController that is there.
On an irrelevant question, how do you visualize any trained model with igibson? I wrote some simple code to visualize a trained mode but it's not working well, which makes me question myself.


Partial scroll wheel zoom disabling

It was suggested that zooming by mouse wheel could work when scrollWheelZoom is set to false, but a modifier key (e.g. shift) is pressed. This way page scrolling with mouse wheel is not affected, but users still have a way to quickly zoom out.

Any update about this feature? It would be really cool to only start the zoom when the scroll starts on the map...

Any update? I really would like to use this.

Not a patch, sorry. However, here's a no underscore version of a debounce for a leaflet map assuming map = L.map. I'm sure this can be added nearly verbatim to leaflet.js with an option to toggle it.
// Only enable scrolling if the user hasn't scrolled the page in 1 second
document.addEventListener("scroll", function() {
        map.scrollWheelZoom.disable()
        var timeOut = null;
        if(timeOut != null){
                clearTimeout(timeOut);
        }
        timeOut = setTimeout("map.scrollWheelZoom.enable()", 1000);
});
}```

@Nycroth That's only on the map container - if the debounce applied to the whole document instead (and only zoomed if the event target was in the map container) it would solve the issue.

hi
is there some news on that issue?
the googlemap behavior on this issue is OK for me: if mousse wheel zoom is disabled, so ctrl + mousse wheel in enabled and users can zoom in or zoom out when mouse is over a map.

Reading through the various issues what is the consensus on the modifier key, is shift preferred over ctrl?
Keeping similarity with Google Maps might not be a bad idea, i.e. use scrollWheelZoom.disable() which can be overridden with ctrl+scroll.

Hi,
google maps display a backdrop with a text "Control + mousewheel to zoom".
Maybe it's possible to do the same with leaflet ?
Thanks.

Already implemented by the https://github.com/elmarquis/Leaflet.GestureHandling/ plugin. No plans so far to implement this in the core.


Added queue for studio processing in PS

Fixes problem with multiple batches in row.

Task linked: OP-2015 PS webpublishing fails on second batch


Alpha Version - User Guide (Feature: Find, Assign, Exit)

Update feature details to match new implementation in ResearchRoster.

Missed a few references to find command that needs to be updated.


PL start-installation

Start Installation updated with fixed links, better translation and current Nginx configuration.

Thanks!


Document mixins

Fixes #603

Codecov Report

Merging #1728 into master will increase coverage by 0.01%.
The diff coverage is 100%.

@@             Coverage Diff              @@
##             master    #1728      +/-   ##
============================================
+ Coverage     85.05%   85.06%   +0.01%
- Complexity     4494     4495       +1
============================================
  Files           431      431
  Lines         17786    17788       +2
  Branches       1872     1870       -2
============================================
+ Hits          15127    15131       +4
+ Misses         2001     2000       -1
+ Partials        658      657       -1

Impacted Files
Coverage Δ
Complexity Δ

...ansformer/ruby/RubyGapicSurfaceDocTransformer.java
95.29% <100%> (ø)
18 <0> (ø)
:arrow_down:

...ain/java/com/google/api/codegen/ProtoFileView.java
100% <100%> (+8.33%)
10 <10> (+1)
:arrow_up:

...ormer/nodejs/NodeJSGapicSurfaceDocTransformer.java
87.09% <100%> (ø)
6 <0> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 38316bd...01dcae7. Read the comment docs.

@geigerj PTAL


Derive ContentDecoder using HTTP Domain.

Closing this because it's not clear what the intent is and how it would actually work.


scrollEnabled false on Android

Bug description:

Can't disable scrolling (vertically) in WebView component. I have a WebView component with a simple HTML source within containing only text.
I'm using Expo.
To Reproduce:

<WebView
  scrollEnabled={false}
  source={{ html: `
    <div>
      Some very long text here
    </div>
  ` }}
/>

Expected behavior:

I want to disable scroll as I would with ScrollView for example, but prop scrollEnabled set to false is not working for me. I tried to disable scrolling inside HTML with style="height: 100%; overflow: hidden;" but this not working either.
Screenshots/Videos:

Environment:

OS: Android
OS version: 9.0 - API 28
react-native version: sdk-37
react-native-webview version: 10.3.2

I have the same issue

from the document https://github.com/react-native-webview/react-native-webview/blob/master/docs/Reference.md#scrollenabled scroll enabled props is only for ios and macos
it used to work on android as well

Any Idea when it use to work, I am working on drag and drop application and I am unable to stop scroll on webview in Android.
It is working perfectly is iOS though

Any Idea when it use to work, I am working on drag and drop application and I am unable to stop scroll on webview in Android.
It is working perfectly is iOS though

Same issue with an additional bug, I have many webviews on the screen which I'm not able to scroll currently, that's good but I still can see the scroll.

The issue in my case where I just wanted to disable the vertical scroll is solved using other prop named showsVerticalScrollIndicator which just hides the vertical indicator.

I am having the same issue.

Why was the problem closed? It still doesn't work. Needed for use inside ScrollView.
Props nestedScrollEnabled, according to the description, turns off the scrolling of the top ScrollView, but I just need the opposite to use refreshControl

2022 here. Still not working on Android.

2023 here. Still not working on Android.

2023-08. Still not working on Android.Should this issue be reopened?

2023-10 still the same issue persists

use the prop overScrollMode={'never'}, it worked for me

Still not working on 19-01-2024.

We need this for android please

any updates on this issue?

🥲

still the same issue using expo. sep 2024

Same isssue Oct 20, 2024.


Minor updates for Pybind + OSL Reference test

Merge in fix for debug python (https://github.com/pybind/pybind11/pull/2025)
Fix OSL reference generation unit test to have include files locally to allow for oslc for verification.

Looks good, thanks Bernard!


only command is open random question. How about open current file?

Not able to get this working. Only command is open random file. No option to open current file.

Hello, I just added the command you are suggesting. Thank you. If you still have any problems, you can ask.


[Bug]: Twilio help text wrongly indicates service SID instead of message SID

Is there an existing issue for this?

[X] I have searched the existing issues

Description
context: https://theappsmith.slack.com/archives/C040LHZN03V/p1688440111645399
In the Fetch message, Delete message, and Cancel message queries, the Message SID field is described as:
Specify the SID of the messaging Service used with the message.
But this field actually requires the SID of the message itself; not the message service. I tested this by:

Attempting a Fetch message query using the Message Service SID, which failed with a 400 error
scheduling a new message with a Schedule message query
fetching that new pending message with a List message query
using the SID from that message in the original Fetch message query, which was successful

the help text should instead read as: Specify the SID of the message.
Steps To Reproduce
given above
Public Sample App
No response
Environment
Production
Issue video log
No response
Version
Cloud

For Twilio, the configs are coming from Cloud Services database. Changing the MockPluginsState.ts has no effect.
https://github.com/appsmithorg/appsmith/blob/9118f3643277d0f97bff6b3102a314e29720a30e/app/client/test/factories/MockPluginsState.ts#L4116


chore(deps): upgrade dev dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-dev-dependencies-k8s-27-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 3910

Questions ?
Please refer to the Backport tool documentation


How to forbid the weekday view slide?

I'd like to change the background color of the topbar, but I didn't find a way to implement it. I have tried every solutions mentioned in the previous issues but it doesn't work./(ㄒoㄒ)/~~

I have solved the problem above, but I got a new problem: I want to make the weekday view not slide together with the day view, because if the background color of weekday view is changed, it looks strange when it slides o(╯□╰)o, so is there any solution to this problem? thanks so much.

The solution is to hide it and provide your own implementation then.


No Longer Communicates

Hello,
I have been using this Skill for a few months now with no issues and I thank you for that in advance.
However now it no longer seems to communicate?   The Echo understands me and it comes up in the Alexa app that it knows what I have said but now I get the following errors:
Skill Response was marked as failure
Request Identifier: amzn1.echo-api.request.b3a273c8-423d-402d-9792-a107979dfb4e The target Lambda application returned a failure response
I also get Unexpected Communication issue
Request Identifier: amzn1.echo-api.request.f3320bd8-740e-4e42-a832-17e660197c7d There was a problem communicating with the requested skill

I'm having the same issue... help

What version are you on?

17 Krypton... I was using the kodi Alexa skills last night and it was working fine... I came home today to watch but it's not working anymore.

@BarrieG14 I think I found the problem... check your kodi IP address 😁😁😁 for some reason mine changed... it's working for me now!!

The public IP address changed for yours or on your private network?

@m0ngr31 I'm on Kodi 15.2

My Kodi is ob a raspberry Pi btw

If you don't have a static IP address, you need to as the README tells you:
If you don't have a dedicated IP address, you'll need a dynamic DNS service to give you a static URL to use so you don't have to be constantly change this value.

So I ended up restarting from scratch but I cannot get it to create the Lambda function at all?
I get the following when I run python deploy-to-lambda.py
"Successfully installed fuzzywuzzy-0.15.0 gunicorn-19.7.1 pytz-2017.2 requests-2.
13.0 yaep-0.0.6
Starting new HTTPS connection (1): lambda.us-east-1.amazonaws.com
C:\Kodi\kodi-alexa-master>"
So I don't get the 'adding kodi-alexa-master and the 'successfully deployed kodi-alexa-master'?


Feature / database extract runs

Fixes #1217
Changes proposed in this pull request:

Add a function, extract_runs_into_db that takes a set of run_ids from one database file and inserts those runs into another database file.
Add a notebook explaining this function

The current limitations and behaviours are:

All runs to be inserted (in one function call) must come from the same experiment
The source database must be of the newest version. If it is not, calling the extraction function is a NOOP, and the user will be warned that an upgrade is needed. A boolean kwarg allows the user to have extract_runs_into_db upgrade the source DB.
The target DB will be created if it does not exist. If it does exist, ~it will be upgraded~ and is in an old version, the user will be warned similarly to how it works for the source DB.

@QCoDeS/core

@QCoDeS/core shall we merge once CI (minus codacy, who doesn't like fixtures) is happy?

@ThorvaldLarsen as discussed offline yesterday, the core team will get together and attack this. But just for completeness, I note here that I currently think changing the identifier you use is better. The dataset name seems like a good pick.


Options for "NaN" strings as NaN floats

Address Issue https://github.com/scikit-hep/awkward-1.0/issues/437

[x] stream JSON fragments from a JSON file
[x] stream JSON fragments from a string (same as above?)
[x] convert "NaN" and "Inf" strings as NaN and Inf floats
[x] convert "NaN" and "Inf" strings as other "predefined" strings
[x] allow user to re-defined "from_string" to "to_string" values (the UI?)

@jpivarski - that seems to be working out of the box. I'm creating the PR to test it on different architectures.

@jpivarski - the issue data sample is not a valid JSON. JSON uses only double quotes, not single quotes https://www.json.org/json-en.html
It would be possible to implement handling these, but I'm a bit hesitant. I'd rather encourage the user code producing a valid JSON. What do you think?

You're right: we shouldn't support single quotes. That's a Python print-out, not JSON.
This PR isn't so much about supporting @aswin1447's specific files as the general features of data like this:
{"x": 1.1, "y": []}
{"x": 2.2, "y": [1]}
{"x": 3.3, "y": [1, 2]}
{"x": 4.4, "y": [1, 2, 3]}
{"x": 5.5, "y": [1, 2, 3, 4]}
{"x": 6.6, "y": [1, 2, 3, 4, 5]}

as though it were in a single list with commas in between. This would not be by literally transforming the text: see
https://github.com/scikit-hep/awkward-1.0/blob/05ca0f3202e3aa2886bce1b7e811ebde52289a40/src/python/io.cpp#L20-L79
and
https://github.com/scikit-hep/awkward-1.0/blob/05ca0f3202e3aa2886bce1b7e811ebde52289a40/src/awkward1/operations/convert.py#L827-L833
It's already doing some funny business with ArrayBuilder's beginlist and endlist to support records; just a little more tweaking would be required to tell RapidJSON to keep parsing after a JSON document is done and handling the beginlist/endlist to return an array, rather than a record. It might not be necessary to rely on carriage returns (\n or \r\n) to separate JSON items in a stream, though this is a very common case. (In principle, it should be enough to see that the RapidJSON parser has come to the end of a valid JSON object but there's more text left; JSON provides its own delimiters.)
The other part of this is to optionally read and write specified strings as floating point nan, inf, and -inf. The nicest interface would let users specify strings that can be as these values, with no such interpretation being the default (the option can be None or a string). The place where this would be applied is:
https://github.com/scikit-hep/awkward-1.0/blob/master/src/libawkward/io/json.cpp
(This is also the only place where we can use RapidJSON directly—it's hidden from the interface/header files so that downstream users of the library won't also depend on RapidJSON. That's why some of the calling conventions are convoluted.)
This PR can also include issue #192, passing a Python file pointer down to the C++ (which also mentions this issue of taking a stream of JSON objects, by the way).

Sure, the issues are related. I was thinking of using a generator to stream the JSON fragments:
    def stream_read_json(fn):
        start_pos = 0
        with open(fn, 'r') as f:
            while True:
                try:
                    obj = json.load(f)
                    yield obj
                    return
                except json.JSONDecodeError as e:
                    f.seek(start_pos)
                    json_str = f.read(e.pos)
                    obj = json.loads(json_str)
                    start_pos += e.pos
                    yield obj

but I'm not sure about performance - it's way too slow or the test file is way too big :-)

but I'm not sure about performance - it's way too slow or the test file is way too big :-)

It would be much better to tweak the rules under which ArrayBuilder::beginlist() and ArrayBuilder::endlist() are called. The difference would be two function calls (in C++) versus modifying the text stream. Generally, these data files are big. (That's why we use RapidJSON's SAX parser, rather than creating a DOM—it can make the difference between having or not having enough memory is realistic use-cases.)

(*) failing integration tests are due to a C++17 string_vew. I'll deal with it alternatively :-)
@jpivarski - I run into a curious comparison error. The reported diff seems to be identical...
Are the numpy.inf and numpy.nan handled as I expect? What shall I use instead?  What shall I use instead?

Maybe it's because NaN is not equal to NaN (according to IEEE rules):
>>> float("nan") == float("nan")
False
>>> np.nan == np.nan
False

A suggestion on StackOverflow is to replace the NaN values with strings, which can be done after the conversion to Python lists.
>>> data1 = [[44.44,   45.45,   46.46,   47.47,   48.48],
...          [49.49,   50.5,    51.51,   52.52,   53.53,
...           54.54,   55.55,   56.56,   57.57,   58.58,
...           59.59,   60.6,    61.61,   62.62,   63.63],
...          [64.64],
...          [65.65, 66.66, float("nan"), float("nan"), 67.67, 68.68, 69.69]]
>>> data2 = [[44.44,   45.45,   46.46,   47.47,   48.48],
...          [49.49,   50.5,    51.51,   52.52,   53.53,
...           54.54,   55.55,   56.56,   57.57,   58.58,
...           59.59,   60.6,    61.61,   62.62,   63.63],
...          [64.64],
...          [65.65, 66.66, float("nan"), float("nan"), 67.67, 68.68, 69.69]]
>>> data1 == data2
False
>>>
>>> def fix(x):
...     if isinstance(x, dict):
...         return dict((k, fix(v)) for k, v in x.items())
...     elif isinstance(x, list):
...         return [fix(v) for v in x]
...     elif isinstance(x, tuple):
...         return tuple(fix(v) for v in x)
...     elif np.isnan(x):
...         return "nan"
...     else:
...         return x
...
>>> fix(data1) == fix(data2)
True

There's no such problem with infinity:
>>> float("inf") == float("inf")
True
>>> float("inf") == float("-inf")
False
>>> float("-inf") == float("-inf")
True
>>> np.inf == np.inf
True
>>> np.inf == -np.inf
False
>>> -np.inf == -np.inf
True

In the IEEE floating point definition, only one bit pattern corresponds to infinity, another bit pattern corresponds to minus infinity, but there's a large space of bit patterns that all correspond to not-a-number. Here are two of them:
>>> np.array([0, 0, 0, 0, 0, 0, 248, 127], dtype=np.uint8).view(np.float64)
array([nan])
>>> np.array([0, 0, 0, 0, 0, 0, 249, 127], dtype=np.uint8).view(np.float64)
array([nan])

Rather than requiring implementations of IEEE floating point arithmetic to complicate equality checks by checking to see if both sides are in or not in the "not-a-number" class, they're just defined as being not equal to each other. (After all, most implementations these days are in hardware, and you want to keep that as simple as possible.)

Oh, I'd rather these options not be a part of ArrayBuilderOptions, since they only apply to JSON (as they work around a JSON limitation). ArrayBuilder is used in a lot of contexts other than JSON. If there's some reason why it would be very difficult to do this, then we can consider this an internal detail, since it's not exposed to the Python interface to ArrayBuilder, but it's looking to me like a bad direction to grow.
The reason we have ArrayBuilderOptions is because ArrayBuilder makes a tree of indeterminate depth. At any moment, it could encounter new data that broadens the type of its array, so each node needs to have access to the configuration options, which includes such things as "how big should the initial allocation be?"
These JSON options don't need to be pushed down this way. The JSON-handling code is all in a SAX parser class that doesn't create nodes of a tree (it calls methods on ArrayBuilder, which internally builds those nodes). So the JSON options don't have to be "pushed down" in the same way. That's why they don't really belong in ArrayBuilderOptions.

done. It's the *Options that confused me :-)

Also, about the options themselves: users need to be able to specify the spelling of "nan", "inf", and "-inf", since there's no standard—different JSON writers generate different strings: "NaN", "Inf", "-Inf", "Infinity", "-Infinity", and "minusInfinity" are all things I've seen. A single dataset won't have multiple spellings, so these are three new options:

the string that gets mapped to std::numeric_limits<double>::quiet_NaN()
the string that gets mapped to std::numeric_limits<double>::infinity()
the string that gets mapped to -std::numeric_limits<double>::infinity()

reimplemented. There are a few parameters with default definitions that can be overwritten by a user. It's a long-ish list.  Please, comment.
convertNanAndInf=False # to float
replaceNanAndInf=False # with another string
fromNan='NaN'
fromInf='inf'
fromMinusInf='-inf'
toNan='NaN'
toInf='inf'
toMinusInf='-inf'

for example:
    array = awkward1.from_json('tests/samples/test-nan-inf.json', convertNanAndInf=True,
        fromInf='Infinity', fromNan='None')

any of which can be None (i.e. no string that gets mapped to a numeric limit).

looking into it :-) Thanks!

The to_json and from_json functions are separate and can get separate sets of options; no one needs all 6, they each need only 3. They can take the same names, too:
def to_json(
    array,
    destination=None,
    pretty=False,
    maxdecimals=None,
    nan_string=None,
    infinity_string=None,
    minus_infinity_string=None,
    buffersize=65536,
):
    ...

def from_json(
    source,
    nan_string=None,
    infinity_string=None,
    minus_infinity_string=None,
    highlevel=True,
    behavior=None,
    initial=1024,
    resize=1.5,
    buffersize=65536,
):
    ...

Giving them the same names highlights the symmetry. I've grouped them here with other JSON parameters (pretty and maxdecimals in to_json, before highlevel and behavior, which are usually last, but ArrayBuilder parameters, initial, resize, and buffersize are usually of even less interest).
There needs to be a distinction between "no conversion string" and "the conversion string that is 'None'", since the latter is a possible encoding of NaN.

Thanks, @jpivarski ! I think, I'm done with this PR.

@jpivarski - the tests pass. There is a workaround a string that is not a file name.
The following change would help to eliminate the need to distinguish between a file name and a string. Would that be satisfactory?
diff --git a/src/awkward1/operations/convert.py b/src/awkward1/operations/convert.py
index 6ffb080..11c135b 100644
--- a/src/awkward1/operations/convert.py
+++ b/src/awkward1/operations/convert.py
@@ -6,6 +6,8 @@ import numbers
 import json
 import collections
 import math
+import io
+import os
 import threading
 import distutils.version

@@ -842,7 +844,7 @@ def from_json(source,
     See also #ak.to_json.
     """
     layout = awkward1._ext.fromjson(
-        source,
+        source if not os.path.isfile(source) else open(source, "r").read(),
         nan_string=nan_string,
         infinity_string=infinity_string,
         minus_infinity_string=minus_infinity_string,

then there would be only need for ak::FromJsonString and FromJsonFile could be deleted.
Please, let me know if that's the direction you'd like me to take and if you'd like this be part of the PR. I've got it locally and all tests pass. Thanks!

Currently, have you put in any changes to let to_json and from_json accept a Python file object? I didn't see any such change in the GitHub diff, just the diff you quoted. (I'm only asking because I want to know if there's some code I'm supposed to look at.)
As for this proposed solution: it reads the whole JSON file as a string before doing any conversion. If a user is passing a filename or Python file object to the function, they're not expecting that, and very large files would then be a problem. (Note: a filled Awkward Array uses much less memory than the text of a JSON file.)
We do need FromJsonFile to do this incremental parsing. That's why we're using RapidJSON's SAX parser. If given a Python file object, you can get its file-handle as an integer:
>>> file = open("setup.py", "r")
>>> file.fileno()
3

(Don't check isinstance to see if it's a plain file—a lot of interfaces in Python have this fileno() method so they can be used interchangeably. Do hasattr(file, "fileno") to check for it. This also doesn't cover a common Python case of file-like objects that don't have a fileno but do have a read method. We can put that off for later, as it will require a FromJsonStringIncremental, which simply hasn't been written yet.)
The source of from_json has a weakness that if a string is both a filename and valid JSON, it's ambiguous how to parse it. I think we pick "filename" in these cases. Having a filename that is valid JSON is weird.

Currently, have you put in any changes to let to_json and from_json accept a Python file object? I didn't see any such change in the GitHub diff, just the diff you quoted. (I'm only asking because I want to know if there's some code I'm supposed to look at.)

No, I did not push any changes - all this is in my private area :-)

As for this proposed solution: it reads the whole JSON file as a string before doing any conversion. If a user is passing a filename or Python file object to the function, they're not expecting that, and very large files would then be a problem. (Note: a filled Awkward Array uses much less memory than the text of a JSON file.)

Yes, you are right.

We do need FromJsonFile to do this incremental parsing. That's why we're using RapidJSON's SAX parser. If given a Python file object, you can get its file-handle as an integer:
>>> file = open("setup.py", "r")
>>> file.fileno()
3

(Don't check isinstance to see if it's a plain file—a lot of interfaces in Python have this fileno() method so they can be used interchangeably. Do hasattr(file, "fileno") to check for it. This also doesn't cover a common Python case of file-like objects that don't have a fileno but do have a read method. We can put that off for later, as it will require a FromJsonStringIncremental, which simply hasn't been written yet.)

Ok, thanks.

The source of from_json has a weakness that if a string is both a filename and valid JSON, it's ambiguous how to parse it. I think we pick "filename" in these cases. Having a filename that is valid JSON is weird.

Please, let me know if I should continue working on this PR or is it ok asis to be merged? Thanks!

I just put in one correction, hiding nan_and_inf_as_float from users. (That's an optimization that should be implicit from the nan_string, infinity_string, or minus_infinity_string being None.)
I also noticed this:
>>> ak.from_json('["one", \n"two"] \n ["three"]')
<Array ['one', 'two', 'three'] type='3 * string'>

The output should be
[['one', 'two'], ['three']]

because each separate JSON document should be separate. Sorry that I wasn't clear when listing test cases.
Meanwhile,
>>> ak.from_json('{"x": 1, "y": 2} \n {"x": 10, "y": 20}')
<Array [[{x: 1, y: 2}], [{x: 10, y: 20}]] type='2 * var * {"x": int64, "y": int64}'>

is too deep: it should be
[{'x': 1, 'y': 2}, {'x': 10, 'y': 20}]

I'll be in a meeting now, so I'm not working on that now. I'll check in again tomorrow. Thanks!
As for passing in the FILE handle with fileno, that doesn't need to be in this PR.

I just put in one correction, hiding nan_and_inf_as_float from users. (That's an optimization that should be implicit from the nan_string, infinity_string, or minus_infinity_string being None.)
I also noticed this:
>>> ak.from_json('["one", \n"two"] \n ["three"]')
<Array ['one', 'two', 'three'] type='3 * string'>

The output should be
[['one', 'two'], ['three']]

because each separate JSON document should be separate. Sorry that I wasn't clear when listing test cases.

done.

Meanwhile,
>>> ak.from_json('{"x": 1, "y": 2} \n {"x": 10, "y": 20}')
<Array [[{x: 1, y: 2}], [{x: 10, y: 20}]] type='2 * var * {"x": int64, "y": int64}'>

is too deep: it should be
[{'x': 1, 'y': 2}, {'x': 10, 'y': 20}]

done.

I'll be in a meeting now, so I'm not working on that now. I'll check in again tomorrow. Thanks!
As for passing in the FILE handle with fileno, that doesn't need to be in this PR.

The last thing on my list is parsing a string that starts with a number:
        str = """ "1
        2

        3 " """
        array = awkward1.from_json(str)
>       assert awkward1.to_list(array) == [1, 2, 3]
E       assert [2, 3] == [1, 2, 3]
E         At index 0 diff: 2 != 1
E         Right contains one more item: 3
E         Full diff:
E         - [2, 3]
E         + [1, 2, 3]
E         ?  +++

@jpivarski - I think, I'll close it and make a new PR after the name migration.


NEW RULE IDEAS

NEW RULE IDEAS

[x] not for replication (Design) Added in rule SRD0065
[ ] Constraints: check for

[ ] udf usage (Performance)
[ ] nocheck (Design)

[ ] Suggest SCHEMABINDING for functions that do not touch tables (Performance)
[ ] Detect sql injection possibilities???

Notes:

Would like to only detect injection opportunies where sql is being concatenated in from a variable.
Would also like to build a sql injection scanner for .Net code

Moved to discussions.


Does the i3s layer support 3857 projection?

@xintongxia Do you recall if we support Web Mercator (EPSG: 3857) or just WGS 84 (EPSG:4326) in the new I3SLoader?

@ibgreen I3SLoader only supports WGS84

@The-End-Hero would be good to open an issue on https://github.com/uber-web/loaders.gl