Add benchmark for the memory use of creating a StellarGraph object

This (ab)uses pytest-benchmark's ability to add a custom timer, by creating a "timer" based on allocations. This timer gives the delta of the bytes allocated and freed between the two "now" instants (snapshot calls), and so can be used to track the size of an object created after the first one, and held alive until after the second one. For instance:
start = snapshot()
keep_it_alive = create_some_object()
end = snapshot()

# doesn't need to be deleted, but can be, as long as the variable outlives the call for `end`
del keep_it_alive

print(end - start, "bytes")

The allocations are measured using tracemalloc, available in Python 3.4 and later (and, since Python 3.6, has numpy integration: https://github.com/numpy/numpy/tree/master/tools/allocation_tracking).
The current results are:
$ pytest tests/ -k test_allocation
==================================================================================================================================================== test session starts =====================================================================================================================================================
platform darwin -- Python 3.6.9, pytest-3.9.3, py-1.8.0, pluggy-0.13.1
benchmark: 3.2.2 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /Users/wil9dw/projects/stellargraph/stellargraph, inifile:
plugins: benchmark-3.2.2, cov-2.8.1
collected 315 items / 309 deselected

tests/core/test_stellargraph.py ......                                                                                [100%]

------------------------------------------------------------------------------------------------------------ benchmark 'StellarGraph creation': 6 tests ------------------------------------------------------------------------------------------------------------
Name (time in s)                                                                Min                       Max                      Mean                StdDev                    Median                IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_allocation_benchmark_creation_from_networkx[None-0-0]               1,512.0000 (1.0)          2,080.0000 (1.0)          1,578.8000 (1.0)        121.9429 (1.08)         1,576.0000 (1.0)      64.0000 (inf)           1;1  0.0006 (1.0)          20           1
test_allocation_benchmark_creation_from_networkx[100-0-0]                1,576.0000 (1.04)         2,080.0000 (1.0)          1,601.2000 (1.01)       112.6978 (1.0)          1,576.0000 (1.0)       0.0000 (1.0)           1;1  0.0006 (0.99)         20           1
test_allocation_benchmark_creation_from_networkx[None-100-200]         143,752.0000 (95.07)      157,552.0000 (75.75)      144,578.0000 (91.57)    3,054.0825 (27.10)      143,896.0000 (91.30)     0.0000 (1.0)           1;3  0.0000 (0.01)         20           1
test_allocation_benchmark_creation_from_networkx[100-100-200]          232,336.0000 (153.66)     246,184.0000 (118.36)     233,034.8000 (147.60)   3,095.1325 (27.46)      232,336.0000 (147.42)    0.0000 (1.0)           1;2  0.0000 (0.01)         20           1
test_allocation_benchmark_creation_from_networkx[None-1000-5000]     3,170,936.0000 (>1000.0)  3,184,720.0000 (>1000.0)  3,172,150.0000 (>1000.0)  2,961.5994 (26.28)    3,171,512.0000 (>1000.0)   0.0000 (1.0)           1;4  0.0000 (0.00)         20           1
test_allocation_benchmark_creation_from_networkx[100-1000-5000]      4,006,040.0000 (>1000.0)  4,019,984.0000 (>1000.0)  4,007,377.2000 (>1000.0)  2,971.6428 (26.37)    4,006,744.0000 (>1000.0)   0.0000 (1.0)           1;4  0.0000 (0.00)         20           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

Why use pytest-benchmark?
Pros:

built-in outlier detection, summarisation
built-in saving and comparison (https://pytest-benchmark.readthedocs.io/en/latest/comparing.html)

Cons:

the units are seconds instead of bytes
requires a few moving parts, although the current implementation has checking/exceptions for the failure cases (not specifying the custom timer, and not returning an appropriate object)

https://github.com/ionelmc/pytest-benchmark/issues/28 is related.
See: #528

One thing to be clear here is: this is (trying to) measure the total size of new allocations that continue living after the benchmarked function exits: it is not measuring the total amount of allocations in that period, or the peak memory usage. This is because:

I didn't think of doing it another way, at the time
it is a better measure of the long-term impact on memory of creating the object in question (that is, if you create a StellarGraph object, how much memory does exactly that object take up, once you start doing other things)

My impression is that any temporary objects created during construction are typically going to be small, and most large values created are going to be long-lived. If it's likely to be useful, we could have separate benchmarks that use get_traced_memory to measure the peak memory use as well.
I experimented with the above and found the following very approximate/preliminary results:

features in nx
feature size
# nodes
# edges
measurement
bytes
increase (×)

False
1
0
0
long living
4504

False
1
0
0
peak
5072
1.13

False
1
100
0
long living
67768

False
1
100
0
peak
73512
1.08

False
1
100
200
long living
189504

False
1
100
200
peak
221688
1.17

False
100
0
0
long living
4504

False
100
0
0
peak
5072
1.13

False
100
100
0
long living
150968

False
100
100
0
peak
166192
1.10

False
100
100
200
long living
272128

False
100
100
200
peak
287352
1.06

False
None
0
0
long living
4504

False
None
0
0
peak
5072
1.13

False
None
100
0
long living
61856

False
None
100
0
peak
73512
1.19

False
None
100
200
long living
184960

False
None
100
200
peak
220632
1.19

True
1
0
0
long living
4504

True
1
0
0
peak
5072
1.13

True
1
100
0
long living
67648

True
1
100
0
peak
74864
1.11

True
1
100
200
long living
190816

True
1
100
200
peak
220696
1.16

True
100
0
0
long living
4504

True
100
0
0
peak
5072
1.13

True
100
100
0
long living
150848

True
100
100
0
peak
157936
1.05

True
100
100
200
long living
272352

True
100
100
200
peak
279440
1.03

True
None
0
0
long living
4504

True
None
0
0
peak
5072
1.13

True
None
100
0
long living
61856

True
None
100
0
peak
73512
1.19

True
None
100
200
long living
182176

True
None
100
200
peak
220344
1.21

That is, the peak memory usage of creating a StellarGraph is currently somewhere between 3% and 21% larger than the memory use of a long-lived StellarGraph.

Drop the 'if' from "Thus... if the benchmarks" (line 82) - git won't let me make this a suggestion.

Done (how peculiar that GitHub wasn't letting it happen 🤔 ).

I know you expanded this explanation for Andrew, but I find it very verbose. I'm not suggesting you change it again, but you could get away with just saying that short-term objects are found in gen 0 and long-term objects are found in gen 1 or 2, and thus gc.collect(0) will remove the short-term objects prior to memory measurement of the long-term objects.

Sure, but this doesn't explain why short-term objects are guaranteed to be in gen 0, which is only true with gc.disable(). I was trying to make it complete so that it makes sense even for people unfamiliar with garbage collection and how it works, as I imagine most engineers working on this library will be. 😄

And as for git not allowing suggestions, somehow there was a filter being applied (no idea why or how). Once I closed all filters, it works properly again.


hubble: Only patch ConfigMap and restart if required

When enabling Hubble, only patch the ConfigMap and restart the Cilium pods when actually required.

As a small hint, in case anyone wants to take this: the logic in (*hubble.K8sHubble).disableHubble here: https://github.com/cilium/cilium-cli/blob/7416ed66c8d019cc22c1913d9fb2ecabb9dd9f1f/hubble/hubble.go#L150-L170 could probably be reused to achieve this.

Indeed, this was fixed as part of #1137. Thanks for the report @walbertus!

Indeed, this was fixed as part of #1137. Thanks for the report @walbertus!


Add new math function

Hi, Paul!
In my project, there is a formula with a calculation of 'ln'. However, jexcel doesn't support it.
I'll be really appreciate If you add the support of advaced Mathematics. Thank you!

You can implement your own methods. As a reference you can use the last example in the following URL.
https://bossanova.uk/jexcel/v3/examples/spreadsheet-formulas


AO-15003: Show less configurations in README

This PR removes some unnecessary configuration descriptions from the README. This makes it easier to begin with.
See https://swicloud.atlassian.net/browse/AO-15003 for the details

Codecov Report

Merging #136 into master will decrease coverage by 0.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #136      +/-   ##
==========================================
- Coverage   81.96%   81.91%   -0.05%
==========================================
  Files          37       37
  Lines        4590     4590
==========================================
- Hits         3762     3760       -2
- Misses        704      705       +1
- Partials      124      125       +1

Impacted Files
Coverage Δ

v1/ao/internal/reporter/reporter_grpc.go
69.37% <0%> (-0.3%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 963bee1...6c23afb. Read the comment docs.


Adding bound interceptors for axios.

After looking around I thought your setup worked the best as an Axios middleware with Redux.  It was missing one thing I needed.  I needed the ability to add headers based on the Redux state.  The best way I could think of was to have an Axios interceptor that had access to the state.
In this PR it gives you the ability to define options like the following.
userOptions = {
  interceptors: {
    requestInterceptors = [...],
    responseInterceptors = [...]
  }
}

These interceptors would have the getState function added to their parameters.
Let me know your thoughts or if you have a better idea.

@nmaves Hi, thanks for contribution. Firstly I wanted to share idea about doing this on axios instance passed to middleware, but then I found out you cant use getState there.
The question is if it is possible to add midleware after store is created, maybe @gaearon can help with that. If it is possible, we can do something like this:
const client = axios.create();
const store = createStore(...); //pass everything expect middleware

client.interceptors.request.use(function () {/*...*/}); //bounding to client instance, not global axios

//and then add midlewares to store
store.enhance(
  //... middlewares
  axiosMiddleware(client)
)

Was there something about my code that you did not like?  I think the only other way would be in introduce your own interceptor that would have access to the state and allow you to modify the request object passed into the action before it gets executed.

Your code is fine. It's my personal preference to don't merge everything instantly. I don't like the second solution to much, because i don't want to reimplement interceptor logic in this middleware.
To the code, I will make few comments there. After we resolve them I will publish package as 0.4.1. Once again, thanks for you time to make this package better.

My bad that came across wrong.  I am still really new to the Redux/Axios space so I am all ears if my solution could be better.  I am not even sure how to test my PR locally.

The root of the issue I am trying to solve is that I need to sign each request with credentials that are stored in redux state.  Right now I have it working by using local storage but I would rather have the credentials stored in the state engine.

@nmaves I test my code locally with npm link. In the project you are using it you can type npm link ./path/to/redux-axios-middleware, which will create symlink from node_modules to your local folder with package

Good call, and thanks for the testing note.  I will try it out soon on my project.

@nmaves Please, fix that second comment and I'm ready to merge.
Now you have this
function bindInterceptors(client, getState, { requestInterceptors = [], responseInterceptors = [] } = {}) {
  requestInterceptors.forEach((interceptor) => {
    client.interceptors.request.use(interceptor.bind(null, getState));
  });
  responseInterceptors.forEach((interceptor) => {
    client.interceptors.response.use(interceptor.bind(null, getState));
  });
  interceptorsBound = true;
}

This ways the middleware options will look like this:
axiosMiddleware(client, {
  interceptors: {
    requestInterceptors: [] // some req interceptors,
    responseInterceptors: [] // some res interceptors,
  }
})

I believe this will be better:
axiosMiddleware(client, {
  interceptors: {
    request: [] // some req interceptors,
    response: [] // some res interceptors,
  }
})

Made that change.  I am slammed this week but I hope to add some documentation and an example of this in the next week or so.


Add Support and Documentation for /time Endpoint

https://m2x.att.com/developer/documentation/v2/time

@kristinpeterson could you please review the documentation changes in case I missed something? Thanks!

@mentos looks great! thanks for checking 🐑


Create and add opencv sample

OpenCV crate usage sample.

Load and store raw image file.
Image data is stored as Mat.
Color image, 8-bit grayscale, 16-bit grayscale, 32-bit float grayscale.
signed 16-bit grayscale stack (3D)

Done with #4


Table doesn't rerender when data becomes available in Applications and Sources

adding a key to the table which changes every time automations / sources change fixes the issue
this doesn't seem to be happening when DataTable is used directly like in Flux Runtime. I think some of the other sub sections use EventTable which makes use of DataTable directly so shouldn't have issues - I can't really test them as more sources seem to not be available when running Core against demo-01.

To test - comment out the <Loading ../> bit in Page and test behaviour with / without the key in the table

Closing in favour of #2191


added workflow as per issue #14

Fixes #14
@BamaCharanChhandogi please review and merge this PR
Also, please add labels and assign me this PR
Thank You!

LGTM, thanks.


Does it support specific highcharts meteogram?

I'm new in Vue. I'm not sure whether I can use vue2-highcharts with the meteogram as I cannot import the adjusted js file meteogram.

vue2-highcharts does not seem to support metegram


%timefield% and %context% error

Hey,
Having a little problem when i set the two fields, can i have more informations about this error ?

Hey there,
is your timefield has "@" sign before it? Sometimes timfields in kibana are in "@timefieldname" format

@Herb4t- does the graph work if you remove both %timefield% and %context% ?  What if you remove just %timefield%? If yes to both, than just like @emilmirzayev suggested, I think you gave the wrong field name for the timefield.  Double check your data structure.  Also, use browser's developer tools (usually F12 in chrome on Windows & Linux), make some change to the vega spec, and see what shows up in the networking tab - you should see the exact error that elasticsearch server returns.

Hi, thanks for your answer, i tested to remove both %timefield% and %context% and it work, but with one of this or both, it doesn't work, i checked my data field and all is ok, so i really don't understand how those two parameters works 🤔

@Herb4t- it seems Elasticsearch server is returning some unexpected error. Could you check the network tab of your browser's debugging tools - it should list all network requests that failed - and paste that error here. Make sure you open dev tools before you open your vega vis, or you can also make some minor modification and click play.

@nyurik
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/bilan_construction/_search] contains unrecognized parameters: [%context%], [%timefield%] -> did you mean [stored_fields]?"}],"type":"illegal_argument_exception","reason":"request [/bilan_construction/_search] contains unrecognized parameters: [%context%], [%timefield%] -> did you mean [stored_fields]?"},"status":400}
Is it ok for you ?

@Herb4t- this is very strange.  Both %context% and %timefield% should have been removed by the Vega plugin before being passed to the server, but that clearly is not happening.  To debug this, I think I would need a bit more info:

I assume the error message you pasted last comes when you used Vega plugin, not from the Kibana's "dev tools" tab. Correct?
Copy the _search request from the networking tab that generated that error - right click on the _search and click "copy / copy as curl"
What plugin and kibana version are you using?  (ideally, link to the specific plugin release file you installed, or tell me if you are using the built-in Vega version in Kibana 6.2.x)
When you create a new Vega graph, and you get the default vega-lite graph, does that work ok?
Could you send me your complete Vega-lite graph specification that generates an error?  Either copy/paste the text (not image), or attach the file to this ticket.  You can also email it to me if it should remain private<EMAIL_ADDRESS>Run these commands from the browser's development tools (console):

VEGA_DEBUG.VEGA_VERSION
VEGA_DEBUG.VEGA_LITE_VERSION

Thanks!

here is the curl of the error (from the browser dev tool)
curl "http://localhost:5601/elasticsearch/bilan_construction/_search?"%"25context"%"25=false&"%"25timefield"%"25="%"40timestamp" -H "Accept: application/json, text/plain, */*" --compressed -H "Accept-Language: en-US,en;q=0.5" -H "Connection: keep-alive" -H "content-type: application/json" -H "Host: localhost:5601" -H "kbn-version: 5.6.1" -H "Referer: http://localhost:5601/app/kibana" -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0" --data ""
i'm using Kibana 5.6.1, and my plugins :

i'm on old version of kibana to use the enhanced_tilemap plugin :/
using this veg version : https://github.com/nyurik/kibana-vega-vis/files/1688997/vega_vis-0.5.4--for-Kibana-5.6.1.zip

Aha, I think I know what's going on.  The very old Kibana version was using a different context param passing, that was later changed. You are using the newer one, but it might be that the older pattern is required.  See https://github.com/nyurik/kibana-vega-vis/releases/tag/v1.0.0 notes.  Try replacing both %context% and %timefield% with just    "%context_query%": "date_depot"

Ah my bad, ok, your solution work, i'll try to update to a newer version, thank you very much :)

Hi, I get the same error.I use kibana version 6.3.0
I made the same error I've done the operations you say.


Soft deleted entity ML Model still appears in the UI (reopened)

This issue was closed incorrectly, please see:
https://github.com/datahub-project/datahub/issues/6328
The meaning of soft delete is that it can be restored, but while it is in soft deleted state it is deleted as far as any user whether API or UI. The soft deleted entity should not appear in the UI. This is a real problem and should be fixed.
I am reopening it here.

Hey @jazzblue sorry if we closed that issue before you felt it was resolved.
The DataHub Core Team has decided that yes, soft deletion, means it can be restored, but we have implemented it so that soft deletion means the entity can appear in the UI. It will not be discoverable via Search or other mechanisms though.
If you would like something to not appear in the UI and you have no need for that metadata anymore, I would recommend you hard delete.
Hope that makes things clear. Thank you for engaging in the community!


[Storage] Add support for listing private endpoint connections

This change adds support for listing private endpoint connections through a GET collection API for Microsoft.Storage/storageAccounts/privateEndpointConnections.
Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[ ] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

/azp run automation - sdk

Can one of the admins verify this patch?

/azp run automation - sdk

/azp run automation - sdk

Need update this PR to use a common definition of private connection, https://github.com/Azure/azure-rest-api-specs/pull/8935


Dismiss all not working with headless toasts.

When I'm using headless toasts created with toast.custom(...) and trying to close all with toast.dismiss() without ID parameter, headless toasts are not closed. Works fine with default ones or if dismissing by ID.
https://codesandbox.io/s/sonner-headless-toasts-dismiss-rxtdsx
Is there a way to close all toasts including the headless ones?

Fixed in #218


fixes CI workflow configuration

The latest runs were broken for some reason so I fixed it.
Here's a summary of what this PR does:

upgrades LLVM to v14
migrates from the unmaintained action-rs/* actions to dtolnay/rust-toolchain, using Swatinem/rust-cache as a cache layer.
adds a cache layer for LLVM build
adds a weekly cron schedule for all workflows
fixes an issue in the docblocks

One thing I like to do is to add cron every week for building + doc. That way, we can anticipate that kind of error

Thank you!


Add new theme

I had some free time and pushed the fix myself, it should work fine. Thanks again, I'll merge the code now.


az storage share list for storage account without shares generating a stack trace

Describe the bug
Issuing command az storage share list --account-name myaccountname for a storage account without shares fails with HTTPSConnectionPool max retries exceeded... producing a lengthy python track trace.
To Reproduce
Create a new Resource Group:
az group create --name mytestresourcegroup --location westeurope
Create a new Storage Account:
az storage account create -n mytestsa -g mytestresourcegroup -l $AKS_PERS_LOCATION --sku Standard_ZRS
BUG is reproduced now by calling:
az storage share list --account-name mytestsa
However, the same command will work as expected if a file share is created:
export AZURE_STORAGE_CONNECTION_STRING=`az storage account show-connection-string -n mytestsa -g mytestresourcegroup -o tsv`
az storage share create -n mytestshare
az storage share list --account-name mytestsa

Expected behavior
The command should return an empty result.
Environment summary
I'm running on Ubuntu 18.04.1 LTS and azure-cli was installed by following the instruction found in https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-apt?view=azure-cli-latest.
Version information:
Python 3.6.5
azure-cli 2.0.54

Actually, after some testing, I was able to see this problem when the sku Standard_ZRS was used with kind Storage (not v1). @Koura specifying --kind StorageV2 should solve this problem.
@seguler this indicates that ZRS skus are only available for general purpose v2 accounts, but why does the service allow the creation of v1 accounts with ZRS sku? Clearer error messages from the service would make things more apparent.

Looks like the filer was able to resolve this issue. hope that no one minds if I close it here.


Use the charmed-openstack-exporter snap

Upstream can take some time until releases a new snap and this is blocking the workflow.
This change from using the golang-openstack-exporter (upstream) to use the charmed-openstack-exporter that is packaging the upstream code and release in the snapcraft store.
With this change the charm automatically uses the snap from the snapstore and removes the upstream snap and/or snap as a resource.
The resource is removed completely and won't be possible to use it as a resource

@samuelallan72 @chanchiwai-ray after talking with Americas, we decided to remove the possibility of using a custom snap as resource, so the code is reflecting this now. Moreover, I remembered that it's necessary to remove the upstream snap to not conflict with the new one. The snap.remove can be used even if the snap is not present so that is why it's outside the try except block

after talking with Americas, we decided to remove the possibility of using a custom snap as resource, so the code is reflecting this now. Moreover, I remembered that it's necessary to remove the upstream snap to not conflict with the new one. The snap.remove can be used even if the snap is not present so that is why it's outside the try except block

Just want to know the story, why suddenly decide not to use resource. The resource metadata can't be taken down from charmhub, and may not be a good user experience

I'm not sure removing support for the snap resource is a good idea, so leaving my review in 'request changes' until I understand the rationale.

Following up on this: we discussed internally, and the rationale for removing it was to simplify the charm (user experience and code logic).  We discussed airgapped environments, but it was put forward that snap store proxy would be used as the recommended method for handling this.

it looks pretty close now, thanks!  I just noted a couple of nits and a query about errors removing the old/upstream snap.

Following up on this: we discussed internally, and the rationale for removing it was to simplify the charm (user experience and code logic). We discussed airgapped environments, but it was put forward that snap store proxy would be used as the recommended method for handling this.

It does remove the convenience of testing with a custom built snap (eg. if we've working on a new feature or fix in openstack-exporter), but we can workaround that (manually ssh in to the unit and install it).

Something I'm not sure about is the future of the snap resource in charm metadata going forward. I've heard tell that we can't remove this now the charm is published on charmhub. Can we confirm that? If we can cleanly remove the resource from metadata, then I'm fine with going forward with removing it. However, if it's stuck in metadata forever, I'm in favour of retaining support, to avoid confusion in the user experience.

(non-blocking) I am +1 to keep the snap resource support.

With this change the charm automatically uses the snap from the snapstore and removes the upstream snap and/or snap as a resource.
The resource is removed completely and won't be possible to use it as a resource

The charm declared GA already, and this simply breaks the backward compatibility after that. Can't this part of the change be a separate review?

this simply breaks the backward compatibility after that.

@nobuto-m could you help us figure out exactly what breaks in the situation of removing the resource?  We asked the charm teams and it sounds like the resource will simple silently drop from the charm and the charm will switch from using the custom snap from resource to installing it from the snap store.

this simply breaks the backward compatibility after that.

@nobuto-m could you help us figure out exactly what breaks in the situation of removing the resource? We asked the charm teams and it sounds like the resource will simple silently drop from the charm and the charm will switch from using the custom snap from resource to installing it from the snap store.

One of the motivations to use a snap resource is for an air-gapped environment. I understand that the ideal solution is to use air-gapped mode of snap-store-proxy. However, when upgrading the charm for such an existing environment, does juju refresh just work? I don't think so. That's considered as a major breaking change in that context so separating out the change to drop the resource support would be the best thing for the time being.


Use Task.Run instead of Task.Factory.StartNew

Using TaskCreationOptions.LongRunning creates a dedicated thread, but it's released on the first await. Using Task.Run uses the thread pool instead.
Addresses #571

Thank you, @andrewlock!


Cosmetics: missing space in log message

There should be a space after "finished":
2019-12-07 19:46:12.605 [INF] NTFN: Historical spend dispatch finishedfor request outpoint=xxx:1, script=0 xxx (start=607101 end=607101) with details: <nil>

I am keen to pick this one up in order to learn how to do a PR

Fixed in #3813


page.should have_field fails

All my checks for have_field fail.
For example, when I run
          page.should have_field "Manufacturer", with: "Siemens"
          page.should have_field "Product name", with: "Lufthaken"
          page.should have_field "Unit", with: "Stk."
          page.should have_checked_field "Accepted"

I get the following error message:
  Failure/Error: page.should have_field "Manufacturer", with: "Siemens"
  NoMethodError:
   undefined method `fields' for Capybara::Session:Class

The matcher have_checked_field works as expected.

Are you using mongoid-rspec gem?
I had similar issue and the problem was in capybara and mongoid-rspec both having have_field matcher. I resolved ше by restricting usage of Mongoid::Matchers to model specs using configuration.include Mongoid::Matchers, :type => :model in rspec config.
If that doesn't help you can run rspec with -b option to see full backtrace. That can give some idea about what's wrong.


Deb/rpm package: support both CE and EE install one one machine.

Currently it is not possible because they are installed in the same folder.

Done.

Verified


cosmic-comp: 1.0.0-alpha.1 -> 1.0.0-alpha.2

Description of changes
Obligatory: #327063
Part of: #259641

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 344490 run on x86_64-linux 1

2 packages built:

cosmic-comp
cosmic-comp.debug


Wraith test is unable to capture shots, fallback image (invalid image) is stored

phantomjs 1.9.2 is installed on my machine and phantomjs -v command confirms it. Not sure why but when I run a wraith test to capture screenshots it throws following error:
No such file or directory - pwd
Using fallback image instead
And, it store fallback image (i.e. invalid image) instead of actual images. Please help!

Hi @darshan-dave, could you tell me:

your operating system (Linux, Windows..?)
the Wraith command you're using (latest, capture..?)
also please paste the contents of the YAML config file you're using

Operating System : Windows 7

Wraith command : 'history' and 'latest'

Content of config YAML file:
#Headless browser option
browser:
phantomjs: "phantomjs"

slimerjs: "slimerjs"
#If you want to have multiple snapping files, set the file name here
snap_file: "javascript/snap.js"
Type the name of the directory that shots will be stored in
directory: 'shots'
Add only 2 domains, key will act as a label
domains:
english: "http://www.live.bbc.co.uk/news"
russian: "http://www.live.bbc.co.uk/russian"
#Type screen widths below, here are a couple of examples
screen_widths:

320
600
768
1024
1280

#Type page URL paths below, here are a couple of examples
paths:
home: /
uk_index: /uk
#Amount of fuzz ImageMagick will use
fuzz: '20%'
#Set the number of days to keep the site spider file
spider_days:

10

#Choose how results are displayed, by default alphanumeric. Different screen widths are always grouped.
#alphanumeric - all paths (with, and without, a difference) are shown, sorted by path
#diffs_first - all paths (with, and without, a difference) are shown, sorted by difference size (largest first)
#diffs_only - only paths with a difference are shown, sorted by difference size (largest first)
mode: diffs_first
threshold: 5

Thanks @darshan-dave - I suspect this is down to Windows not understanding the pwd command. I've tagged this as a bug.

Thank you @ChrisBAshton
This works fine on one of my test Windows machine with same configuration, but not on another.

Thank you, @ChrisBAshton
The fix provided by you worked for me.
Thank you again.

Hi @darshan-dave - thanks for letting us know! :)


No output on LCD screen after uploading.

Hello. I purchased the Arducam RevC+ shield and OV5642 5MP camera module recently. I connected it to the Arduino Uno R3 but after compiling the code in the 1.5.5r-2 IDE, I got nothing on the LCD screen. I also noticed that the UTFT_SPI and itoa were not highlighted in orange like the other library file names in the #include directives in the early lines of the code, yet these files exist in the libraries folder of the compiler. I'm not sure what the problem is, especially after the code can successfully be compiled and uploaded to the arduino. I'd be most grateful for any help with this.

Here is the example file I'm using for the digital camera function, which I got from your repository.
ArduCAM_OV5642_Digital_Camera.txt

@WOL7, you need to copy both ArduCAM and UTFT_SPI libraries to the Arduino library directory.
And you have to modify the memory_saver.h file to uncomment the line to enable OV5642 camera module then compile the example sketch.
You can open the serial monitor to view the debug information printed from the UNO board to figure out where the problem is.

Thanks for the reply, @Lee. I had the first thing and then I uncommented the ov5642 line in memorysaver.h but my screen is still black. Is this because I don't have the lens mount? The serial monitor also displays "Arducam Start!" And then some other characters that don't make sense...

I realised I was using an OV2640 instead of a OV5642 and made he necessary changes. Thanks @Lee.


"Țepu" and "Țepu de Sus" are listed in the drop-down menu

Description:
"Țepu" and "Țepu de Sus" are listed in the drop-down menu alongside its corresponding county and commune, according to the law
Precondition:
The website is up an running.

Step 1
Type "Țepu" in the search bar.

Expected results
The location is listed in the drop-down menu as "Sat Țepu, GALAȚI (Țepu)" and "Comuna Țepu, GALAȚI".

Step 2
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 3
Type "Țepu de Sus" in the search bar.

Expected results
The location is listed in the drop-down menu as "Sat Țepu de Sus, GALAȚI (Țepu)".


Add Py3 - How to add column descriptions

Baseline has requested help a few times in getting column descriptions added to their bigquery table. This notebook should help in acting as a reference.

@wnojopra what would you like to do with this PR? (e.g., bring the notebook style up to date or close it?)

Closing. I can submit a new PR for an updated notebook.


Remote CentOS 7 "Run XAgent with GUI" cmd has no log

Issue Description / 问题描述
Please provide a detailed description of the error or issue you encountered. / 请详细描述您遇到的错误或问题。
I execute the command "Run XAgent with GUI" as described in README in a CentOS 7 server. (All dependency is installed)
docker exec XAgent-Server systemctl stop nginx
But there is no log as the screenshot shows.
Steps to Reproduce / 复现步骤
Please provide the specific steps to reproduce the error. / 请提供复现错误的具体步骤。
Run docker exec XAgent-Server systemctl stop nginx, and I get nothing:

Run curl in the remote server seems fine:

When assessing the URL through a local browser, not work:

Expected Behavior / 预期行为
Describe the behavior you expected to see. / 请描述您期望的正确行为。
Environment / 环境信息

Operating System / 操作系统：CentOS 7
Python Version / Python 版本：3.11
Other Relevant Information / 其他相关信息：

Error Screenshots or Logs / 错误截图或日志
If possible, please provide relevant screenshots or logs of the error. / 如果可能，请提供相关的错误截图或日志文件。
ps -ef | grep nginx

lsof -i:5173

``
Additional Notes / 其他备注
If you have any additional information or notes, please add them here. / 如果有其他补充信息，请在此处添加。

if you are using remote server, please make sure you add port forward for both 5173 and 8090 ports

Thanks!! 👍

Sorry to interrupt again! @luyaxi
I added one forward rule from 80 -> 5173, and then I am able to browse the index.html and log in.
firewall-cmd --zone=public --add-forward-port=port=80:proto=tcp:toport=5173 --permanent

But after submitting the question, the web hangs.

I guess I should forward port=8090 also. But I don't know how to forward. Should I change the nginx.conf? And then execute firewall-cmd --zone=public --add-forward-port=port=8090:proto=tcp:toport=ANY_OTHER_PORT --permanent?

You should use port 8090:
firewall-cmd --zone=public --add-forward-port=port=8090:proto=tcp:toport=8090 --permanent


Add @stdlib/string/base/dotcase package

Resolves #848.
Description

What is the purpose of this pull request?

This pull request adds a @stdlib/string/base/dotcase package.
Related Issues

Does this pull request have any related issues?

This pull request:

resolves #848

Questions

Any questions for reviewers of this pull request?

No.
Other

Any other information relevant to this pull request? This may include screenshots, references, and/or implementation notes.

This package is implemented in a similar fashion as other packages for changing string cases.
Checklist

Please ensure the following tasks are completed before submitting this pull request.

[x] Read, understood, and followed the contributing guidelines.

@stdlib-js/reviewers

Merging in as all tests and checks have passed.

Thanks for merging this PR and all your hard work!


Simulated rover drives forward when turning in place at slow speeds

Hey guys,
Since it's especially important this year to have the simulated and physical rovers behave similarly, I'd like to raise an issue we've seen with the simulated rovers during our tests.
Currently, if the simulated rover attempts to turn in place at ~0.8 rad/sec or slower, instead of simply turning in place, the rover creeps forward while turning. Sometimes the creep is large enough that the rover starts a 90 degree turn from a position completely outside of the home ring, and finishes it on top of the outer ring of April Tags.
Unfortunately, the Base Code's default turn speeds are fast enough to mask this discrepancy between the simulated and physical rover, both when teleoping through the GUI or letting the rover drive autonomously.
Turning in place using the keyboard teleop sends a Twist to the skidsteer plugin with angular z = +/- 4.5 rad/sec, while SBridge publishes Twists in autonomous mode that max out at ~ +/- 3.2 rad/sec and settle at ~ +/- 1 rad/sec when the rover is performing an in-place turn. So, I don't expect you to have observed this in your tests, but you can still replicate the issue without modifying any source code by publishing to the skidsteer topic directly:
rostopic pub /achilles/skidsteer geometry_msgs/Twist "linear:
  x: 0.0
  y: 0.0
  z: 0.0
angular:
  x: 0.0
  y: 0.0
  z: 0.8"

You could also use rosrun rqt_robot_steering rqt_robot_steering. The interface lets you type the topic to publish on, set min and max values for the Twist, and has convenient keybindings (hover over the buttons for the tooltips). Set the max angular vel to +-0.8 rad/sec and turn the rover back and forth from it's starting position. Eventually it will have moved all the way onto the home ring.
The creeping gets worse with even smaller values. It's also present regardless of how fast the simulation is running.
I hoped that simply forcing the simulation to run at real time, while possibly increasing the number of ODE solver iterations from 50 to 100+ would fix the issue (gz physics -i 100 at runtime). Unfortunately, 100 iterations doesn't really remove the creep, especially at speeds slower than 0.8 rad/sec, and pushing the setting much higher with a step size of 0.001 makes it pretty hard to run at a 1x real time factor while simulating multiple rovers.
I did, however, find two ODE parameters that fix this problem. The Constraint Force Mixing parameter (CFM) and Error Reduction parameter (ERP) are sometimes used to model spongy or springy joints, but they can also be used together to reduce numerical errors in the simulation. See http://www.ode.org/joints.pdf
A CFM of 0 (the global default), puts a hard constraint on a joint, while a positive value allows the joint to be pushed on to violate its now soft contraint. You can visualize this parameter in action if you open Gazebo Client and set the global CFM in physics > constraints (try 0.2 for example, it's kind of fun! Probably cap the update rate at 1000 to slow everything down). The whole rover moves as though all the joints are connected by springs, and you can see the wheels pull away from the base during a turn.
The ERP determines how quickly any joint error will be reduced. An ERP of 1 will, in theory, eliminate any joint error in one time step. The global default ERP is 0.2, and recommended values are 0.1 to 0.8 according to the linked documentation. Again, to see what the ERP does to the rover's model, try setting it to an entertaining 0.05, while leaving the CFM at 0.2. Now the wheels can pull completely away from the base, and almost pull it along the ground, but you can see they do still return slowly to their proper position.
I think using a CFM of 0.01 and an ERP of 0.8 dramatically improves the rover's ability to turn in place without driving forward.  Additionally, you can set these parameters on individual joints. I set them on each of the four wheel joints, while leaving the default global values, and the turning performance is equivalent, but setting the parameters on just the wheel joints virtually eliminates any of the residual bounciness that using the global parameters can result in.
Using a larger ERP for the wheel joints also gives you the added benefit of preventing the errors from intermittently growing large enough so the rover moves forward on its own. Sometimes I'll forget the simulation is running, and come back to see that the rover has somehow driven forward into the home ring when it should have been stopped.
I built the base code and recorded a video of two rovers turning back and forth at 0.8 rad/sec in the simulation. Achilles (black) has the current model, and Aeneas (yellow) has the CFM and ERP parameters set for each wheel joint.
https://youtu.be/_CpjMkTUQls
You can see in the video, Achilles moves all the way onto the home ring, while Aeneas doesn't move forward at all.
I pushed a branch to my fork with the update to Aeneas if you want to check it out and start the simulation with one original model and one updated model for comparison.
https://github.com/darrenchurchill/SwarmBaseCode-ROS/tree/sim-cfm-erp-wheels
Is this a change you guys would consider?

@darrenchurchill Thank you for your awesome effort to make the simulation more realistic! I think this is a change that we would consider; however, I need @gmfricke and others to weigh in on this before we move ahead.
Sorry for the slow response, it has been a busy time and there aren't as many of us working on the swarmathon as there used to be.

No problem, Will, I understand. Thanks for getting back to me. Let me know what you guys think, and whether you'd like a full pull request with all the models updated.


Fix 'Use of scanf() operation that permits buffer overflows (CWE-120)' issue in Math\MillerRabbin_PrimalityTest\MillerRabbin.cpp

CodeFactor found an issue: Use of scanf() operation that permits buffer overflows (CWE-120)
It's currently on:
Math\MillerRabbin_PrimalityTest\MillerRabbin.cpp:9
Commit 2540cfe0630d456587b44543133cd5aac7998e50

I'll remove the redundant code from the file if you assign it to me.


Add SKIP_PUSH_GCS env so we can run without a push to GCS

Not everyone has permissions on that container ;-)

Sorry for the delay on this one. LGTM.

This appears to have broken Jenkins:
./hack/jenkins/build.sh: line 50: KUBE_SKIP_PUSH_GCS: unbound variable
Build step 'Execute shell' marked build as failure


Trivial: Remove setLogLevel for subsystems that no longer exist in libp2p_helper

This removes the following error messages at Mina startup:
libp2p_helper: failed to set log level debug for mplex: error: No such logger
libp2p_helper: failed to set log level debug for stream-upgrader: error: No such logger
libp2p_helper: failed to set log level debug for discovery: error: No such logger
libp2p_helper: failed to set log level info for addrutil: error: No such logger

!ci-build-me


Spread syntax introduced in 0.3.19 breaks old browser support

Version 0.3.19 appears to be the first to use the spread syntax (...), and I suspect this is what's causing incompatibility between this version and older browsers.
https://github.com/shakiba/planck.js/commit/32eddf9a64d881658e05975762537b3b87f637af#diff-83c7ea65fda1a6f61c704f578d789d95R7334-R7335

Thanks, fixed!


Updated german translation (951 - 954)

Update the german translations for 951 to 954.

 Wait till i "removed" the uneeded ones

This request has conflicts, could you resubmit this request?

With the help in IRC (by jhass), i now added the translation for 955. (And thus no more conflicts)


Autoform render field of type Map<String, Integer> cannot enable the submit button

Describe the bug
I have the following model:
public record PackageRecord(     String id,     Long version,     String name,     Map<String, Integer> features ) { }
This is how I render the input fields for features:

This is how it looks like on the browser:

The problem is: when I change the value of the input, I am not able to enable the submit button. Am I doing this correctly? What is the proper way to render text fields for Map?
Expected-behavior
No response
Reproduction

System Info
OS: Mac OS Sonoma 14.6.1
Hilla: 24.4.6
Browser: Firefox 129.0.2 (64bit)

From the looks of it, seems that the issue is that any changes from the user aren't updating the form binder. The item value is mutated, but this change is not being picked up.
Please try using field with the form binder API to get proper two-way bindings.

Thank you @platosha. That makes sense.
Any pointers on how I can make use of field to get the bindings?
I am not yet able to find a way to do that.


Make it easy to run a simple testnet

This PR introduces several commits that make it easy (easier?) to run a single test network:

First, make the genesis config-build deterministic, this is a common request and just makes it easier to work with the tooling. It also makes it easier to support resumption of an existing single test node network.
Second, add a slight bit of libra-swarm code to libra-node to support a single validator network
Next up, was to tweak some other components to allow for using config defaults and specifying the config path for this test layer... this let's us have well defined behavior within docker
Finally, we introduce a docker container that bundles this all together... of course the current image it points to won't work until it gets built with the first three commits, but a dev can build the docker container locally in the mean time.

I'll try to figure out how to make a compose that unifies starting a faucet in addition... but no promises.

release binaries dependency change summary:
target packages:
  M libra-genesis-tool 0.1.0 (initial, path 'config/management/genesis')
    * added features: testing
    * (unchanged features: [none])

release binaries dependency change summary:
target packages:
  M libra-genesis-tool 0.1.0 (initial, path 'config/management/genesis')
    * added features: testing
    * (unchanged features: [none])

Thank you ❤
Haven't read the code. Any instructions on how I can run this locally?

Wow! Thank you!

The libra-node takes in a --test and prints out all the yummies that you need to connect to it
If you want to use the docker-compose, I'll add a little more comment into that file, but right now we would need to build it from source.
I'll add a readme.md to the compose directory, but effectively you need docker and docker compose installed (the latter is installed by default on mac and windows). Open up your favorite cli, cd into the appropriate compose directory, and run docker-compose up

release binaries dependency change summary:
target packages:
  M libra-genesis-tool 0.1.0 (initial, path 'config/management/genesis')
    * added features: testing
    * (unchanged features: [none])

release binaries dependency change summary:
target packages:
  M libra-genesis-tool 0.1.0 (initial, path 'config/management/genesis')
    * added features: testing
    * (unchanged features: [none])

I guess ConfigBuilder never dies :)

This is "our" config-builder based upon genesis. I have one more PR coming that will make it easy to add Faucet.
At which point!!! I am deleting the hell out of config-builder <3 (config/config-builder)
Thanks for the timely review!

/land
I hope this goes through!!!!

release binaries dependency change summary:
target packages:
  M libra-genesis-tool 0.1.0 (initial, path 'config/management/genesis')
    * added features: testing
    * (unchanged features: [none])

release binaries dependency change summary:
target packages:
  M libra-genesis-tool 0.1.0 (initial, path 'config/management/genesis')
    * added features: testing
    * (unchanged features: [none])

:lock: Merge Conflict

/land

release binaries dependency change summary:
target packages:
  M libra-genesis-tool 0.1.0 (initial, path 'config/management/genesis')
    * added features: testing
    * (unchanged features: [none])

/priority high


Piping results with Streams2: TypeError: dest.on is not a function

How does piping results with streams work? I get error:

TypeError: dest.on is not a function

My goal is to execute several queries:
SET FOREIGN_KEY_CHECKS = 0;
TRUNCATE TABLE user_account;
TRUNCATE TABLE user_profile;
TRUNCATE TABLE user_account_status;
SET FOREIGN_KEY_CHECKS = 1;

Here is my operation:
        pool.query('SET FOREIGN_KEY_CHECKS = 0').stream({highWaterMark: 5})
            .pipe('TRUNCATE TABLE user_account', function(err, res, flds) {
                console.error('[err] :', err);
                console.log('[res] :', res);
                console.log('[flds] :', flds);
                done();
            })

And error thrown:
  Message:
    TypeError: dest.on is not a function
  Stack:
    TypeError: dest.on is not a function
        at Readable.pipe (C:\Users\user\sbox\node\app\node_modules\mysql\node_modules\readable-stream\lib\_stream_readable.js:527:8)
        at Object.<anonymous> (C:\Users\user\sbox\node\app\spec\mySpec.js:69:14)

How to do it?

Hi @wzup, it looks like there is a general misunderstanding on what streams2 is in Node.js, as your .pipe() call does not make any sense. You can lean more about streams2 probably in other sources. For how to use .pipe(), you can also refer to the Node.js documentation on this method: https://nodejs.org/api/stream.html#stream_readable_pipe_destination_options


Use /kresus/app/ as Cloud9 workspace by default

It was previously hard-coded to /home/user/kresus/app/ on janitor.technology, but this was lost when switching to Cloud9 SDK.

@bnjbvr Please have a look when you can. 😄

LGTM, thanks for the patch! Will rebase and merge manually. Cheers!

Woohoo, thank you for merging on framagit! 🍾


login process sends plain text passwords to the server

Regarding to current spec - on login process user password of Matrix account is sent from client to server into json as plaintext. So any homeserver owner can easily get user password via simple dump json of login action!
This dramatically decrease security of user accounts! Especially after recently enforcing E2EE and cross-signing on Riot clients - now users need to create two passwords for Matrix account (one - for login, second - for SSS encryption), and most of them will use one password for both things, because generate (and remember!) two unique passwords is too hard task for regular modern users, that used to use WhatsApp, Telegram, Viber, which works even without using passwords, and have E2EE (for private chats only, but users don't read this nuance)!
So, if Matrix homeserver admin will dump login passwords of all users, most of them will suitable for decrypt his Secure Secret Storage and admin got access to his e2ee keys!
For solve this security problem - my suggestion is implement new login type, when user password is hashed on client side, and sent to server as hashed string. For enhance protection, server (or even client) can generate salt, that will be used on client side to generate unique hash of password each time. For not broke behavior on old clients, we can keep old plaintext "m.login.password" login type, and provide new "m.login.hash" mode for more secured login type.
What do you think about described security problem and proposed solution? Maybe others can provide better solutions for this problem?

Also, as I understand, we can use same hashing method on user register process, for not send plaintext password to server even on account registering process!

Client-side key derivation ("expensive hash function") from the password at first sight seems safe. What to do with key derivation parameters & salt probably also needs consideration. Password changes would also need to change the server-side password at the same time.

something I missed in the wall of text on first reading: the specific problem here is that the password is common to SSSS and login, so an HS admin can snoop submitted passwords and use them to decrypt SSSS.
@uhoreg (or someone else more familiar with SSSS): is this actually correct, or is there more to it?

The main problem is that account password can be easily leaked via any homeserver admin, and provided way to make it more secure.
Second problem is that this password often match to SSSS password, so leaking account password helps with unlocking SSSS too for most of homeserver users, and leak all E2EE keys.

The main problem is that account password can be easily leaked via any homeserver admin, and provided way to make it more secure.

right, but this isn't a problem unless that password is used elsewhere. If the HS owner wants to act as a user, they have a myriad of ways to do so without having to capture the password.

I think @richvdh's summary matches my understanding with a slight tweak to the wording:

the specific problem here is that the password can be common to SSSS and login, so an HS admin can snoop submitted passwords and try to use them to decrypt SSSS.

right, but this isn't a problem unless that password is used elsewhere. If the HS owner wants to act as a user, they have a myriad of ways to do so without having to capture the password.

But why we don't store plaintext passwords in database, but use hash and even salt? :) Because user password is secret, that must be not known to service owner, because most of users use same passwords in other services, and this homeserver admin can hack all other user accounts, if will know plaintext password. But if admin have only hash of password, he can't do this.

Ignoring the lack of backwards compatibility between clients here.
The biggest issue with this suggestion is that it would require an "escape hatch" where the server would need a method to signal to the client that it NEEDS the plaintext password to submit to a password provider such as LDAP.
This sort of mechanism could easily be exploited by a malicious server just the same as the plaintext password being sniffed in the first place.
Equally a server could throw you into a fake SSO flow which does the same thing.

I think it's probably worth separating the discussion relating to SSSS and the wish to not send plain text passwords to the HS. I think providing a way to ensure the HS can't decrypt SSSS is valuable.

I suggest creating new login type, something like type: ["m.login.password", "m.login.hash", "m.login.token" ]
This will not break old clients, but new clients will can show for users achive "Your password is end-to-end encrypted" on login form.

I think providing a way to ensure the HS can't decrypt SSSS is valuable.

We don't send the SSSS passphrase to the server if that's what you're thinking

This will not break old clients, but new clients will can show for users achive "Your password is client-side encrypted" on login form, if server support this.

It would have to be the inverse, and show a massive warning if the server claims it doesn't support it.
It still wouldn't work around the fake SSO to capture credentials.

It still wouldn't work around the fake SSO to capture credentials.

Yes, but SSO not leak plain-text password to homeserver :)

It would if it is a Fake (phishing style) SSO which the hs hosts itself and takes credentials from.

@MurzNN:

But why we don't store plaintext passwords in database, but use hash and even salt? :)

it guards against an attacker who gets access to a copy of the database somehow. That's a very different bar to an admin who has the ability to capture live traffic and write to the database. "Encryption at rest", etc.
I really feel like you're obsessing about the wrong problem here, but if you really wanted to solve the problem you are obsessing about: clearly a simple hash is ineffective (otherwise the hash just becomes a password); you'd need a challenge/response system. No doubt there is prior art here, but I don't know what it is.
As @Cadair says: maybe we should separate "user password can be used to decrypt SSSS" from "plain text passwords are sent to the HS". The former (if true) is a valid oncern to me; the latter is a very, very minor concern compared with the difficulty of addressing it.

I think providing a way to ensure the HS can't decrypt SSSS is valuable.

We don't send the SSSS passphrase to the server if that's what you're thinking

But isn't the SSSS passphrase the same as the login password, which we do send to the server?

My current issue is only about "plain text passwords are sent to the HS", and mentioned SSSS password is only example of potential problem with this issue. Same problem if homeserver admin got user plaintext password and try to use it with his email account, github account, etc.

But isn't the SSSS passphrase the same as the login password, which we do send to the server?

No, SSSS passphrase is the "Recovery Passphrase" riot asks for, it can be set to the same but it is explicitly stated it should not be in the prompts.

@t3chguy right. in which case, this whole issue seems like a huge storm in a tiny teacup.

No, SSSS passphrase is the "Recovery Passphrase" riot asks for, it can be set to the same but it is explicitly stated it should not be in the prompts.

My thoughts here is can we make it so it can't be set to the same as doing so is baad. :grin:

in which case, this whole issue seems like a huge storm in a tiny teacup.

Well this issue would in theory, best case, allow the Recovery Passphrase to go away completely.
Take one user input, $input;
User submits password=hash($input)
User uses $input as the SSSS Recovery passphrase

My thoughts here is can we make it so it can't be set to the same as doing so is baad.

Not without sending the recovery passphrase to the server to test if it is the same which defeats the point of it not being the same and the server not knowing it.

Not without sending the recovery passphrase to the server to test if it is the same which defeats the point of it not being the same and the server not knowing it.

I was thinking the server could attempt to use the account password to decrypt it and if it succeeds it would fail?

But the server doesn't know the account password unless you submit it again,
plus even then the server could lie to have you upload your real data and then decrypt it.

The only way it'd be possible is if clients stored hash2($password) [other hash func]
and compare it to that. Though it would require all existing sessions to re-auth :(

This issue is largely a dup of https://github.com/matrix-org/matrix-doc/issues/1699  We may not use the method suggested in the doc that is linked in the issue, but I think that issue sufficiently captures the intent of allowing users to only use one password without the server finding out the password.  I'll put a comment in that issue that outlines some considerations.


:mode="'single" not working

as it should be the form of the inline component, because even though I put :mode = "'single'" I still get the format of 'range

Could you link to a reproduction? I just tried the mode="'single'" and inline="true", and for me it's working fine.


I will add name and photo of maintainer in the end of readme file of your repo

like the one mentioned below

its not needed


Account for fb

@danwalkowski for developers who use the latest Facebook SDK and want to allow it to return NO for application:didFinishLaunchingWithOptions:. Tested by both myself and the Gogobot team, this works.

We need partners to toggle on a new flag, [branch accountForFacebookSDKPreventingAppLaunch];, if they use the Facebook SDK

@danwalkowski please review

this is good to go. just need a read to make sure I didn't botch anything

tested by a partner who verified it works

looks good :+1:


Positional Parameters autocompletion vs Sub-Commands

Hi Remko,
I am enjoying picocli very much.  It is quite fast and simple to work with.
I had a few issues with positional parameters, which are already documented.
My workaround, which you may find interesting, was to programmatically generate sub-commands instead.
This is working great for autocompletion and so on.
Perhaps you could internally remodel positional parameters as a special case of sub-commands.
            CommandLine cmd = new CommandLine(commands);

            CommandLine regionCommandLine = new CommandLine(new RegionQueryCommand());

            for(String region: RegionQueryCommand.Regions.regions()) {
               regionCommandLine.addSubcommand(region,
                       new RegionSubCommand(region));
            }

            cmd.addSubcommand("region", regionCommandLine);

I used the @Parent annotation on RegionSubCommand the to pass the "parameter" to the RegionQueryCommand, then invoke the parent's run().
Thanks again for the great project!

I am not sure that I want to bring this idea into the library, but it may still be interesting for others.
If you were to write an article about this idea, I can link to it from the picocli documentation. Does that sound interesting to you?

@shannonbay Thanks again for raising this.


fix(website): copyright update

part of #1836

Let's merge. We don't need to spend time nitpicking.

We don't need to spend time nitpicking.

🤣 Exactly.


can't  subscribe  to only one segmented ticker plant

I have 2 segmented tps (stp1,stp2) and one segmentedchainedtickerplant (sctp)
I would like to subscribe to  stp1 from sctp ( i used sctp.tickerplantname)
I have the following  configuration:
localhost,{KDBBASEPORT}+1,segmentedtickerplant,stp1.....
localhost,{KDBBASEPORT}+2,segmentedtickerplant,stp2.....
localhost,{KDBBASEPORT}+25,segmentedchainedtickerplant,sctp .....    -.sctp.tickerplantname stp1

connect to segmentedchainedtickerplant -> .z.W and  i see open handles for stp1 and stp2 instead of stp1 only

Hi @Igor-Mochalin
Thank you for raising this issue.
This behaviour shouldn't cause any problems. The SCTP connects to any process of type segmentedtickerplant by default so both sets of handles should appear in .z.W. However, the SCTP only subscribes to the name that you have given. So with your config, the subscription should be correct.
You can double check the subscription by checking that .stpps.subrequestall in stp2 does NOT contain any handles to the SCTP (and vice versa for stp1). Please let us know if this is not that case.
Kindest regards
Support team

Thanks for quick reply!

The SCTP connects to any process of type segmentedtickerplant by default so both sets of handles should appear in .z.W

Hm ..  it's strange behavior , any reason for that ?

You can double check the subscription by checking that .stpps.subrequestall in stp2 does NOT contain any handles to the SCTP (and vice versa for stp1

Yes,  looks ok! Thanks a lot!

I dont believe there is a specific reason for this other than the fact that the SCTP uses the .servers.CONNECTIONS list to control process connections, which is standard in TorQ.
Glad to hear its working. If there are any other issues, please let us know.
Kindest regards
Support team

Ok, thanks  for  explanation!  One  more  issue: ( or maybe  by design )
I can't use  something  like this
localhost,{KDBBASEPORT}+25,segmentedchainedtickerplant,sctp .....    -.sctp.tickerplantname stp1 stp2
i.e .
i would  like  to subscribe to 2 tps  from one  chain  tp ...
current  behavior:
subscription  works  fine  for stp1
and  .stpps.subrequestall  is empty  for  stp2

At the minute, the SCTP doesn't support multiple subscriptions, however for a quick solution if you head to code/segmentedtickerplant/sctp.q and replace the subscribe function in there with the following:
subscribe:{[]
  s:.sub.getsubscriptionhandles[`;tickerplantname;()!()];
  if[count s;
      subproc:flip s;
      `.sctp.tph set subproc`w;
      // get tickerplant date - default to today's date
      .lg.o[`subscribe;"subscribing to ", .Q.s1 subproc`procname];
      r:flip .sub.subscribe[subscribeto;subscribesyms;schema;replay;] each flip subproc;
      if[`d in key r;.u.d::r[`d]];
      if[(`icounts in key r) & (loggingmode<>`create); // dict r contains icounts & not using own logfile
        subtabs:$[subscribeto~`;key first r`icounts;subscribeto],();
        .u.jcounts::.u.icounts::$[0=count r`icounts;()!();subtabs!enlist [sum r`icounts]subtabs];
      ]
    ];
  }

This should allow multiple subscriptions with the config that you have. Note that some other features of the SCTP may not function as expected, though hopefully this is still of some use. I will pass on the feedback that this should be implemented properly.
Kindest regards
Support team

Hi @Igor-Mochalin ,
As an addendum to what @etennoQ has already said, the SCTP has been designed to be a replica of the main STP and for the relationship to be 1:1, so while it may be possible to get it to subscribe to multiple STPs, some of the more advanced functionality, like recovering from upstream logs, may not work as intended.
The good news is that if you just need a process which subscribes to multiple STPs and passes on updates, it is possible to throw something together in just a few lines of code:
// Example process which subscribes to multiple STPs and publishes the updates it receives

// Load in common code (including subscription code) and connect to all STPs
.proc.loadcommoncode:1b;
.servers.CONNECTIONS:`segmentedtickerplant;

// Set up connection management and subscribe to all STPs
init:{
  .servers.startup[];
  subs:.sub.getsubscriptionhandles[`segmentedtickerplant;`;()!()];
  subinfo::.sub.subscribe[`;`;1b;0b;] each subs;
 };

// Publish any updates it receives
upd:.stpps.pub;

// Run init function on startup
.proc.addinitlist(`init;`);

This process subscribes to all STPs and publishes any updates it receives to its subscribers, and is fairly simple to put together!
I hope this helps,
Kind Regards
Support Team

yes,  thanks  a lot!


Send forest-name parameter through on queries when it's set on QueryManager

Data movement needs the ability to run forest-specific queries.  We've added the capability to the REST layer.  Now the Java client API layer needs to pass along the forest-name parameter to queries when the forest is set on QueryManager.

We'll no longer be setting it on DatabaseClient (see #418) but we will still need to test that when we set it on QueryManager that it works.  So I'll change the title and summary of this issue.

see #418

Yeah, this is somewhat a dup of #418.


doc: update README, make it easy to read and follow

Rearrange chapter
Add tutorial on deploying API server
Add Emoji

Screenshots

LGTM! Thanks for your contribution! You can merge it when ready!

Thanks, I have nothing to update, let's merge it.

@DoodleBears btw can you make a bit modifications as above?

@DoodleBears btw can you make a bit modifications as above?

What modifications

@DoodleBears btw can you make a bit modifications as above?

What modifications, since the content wrote by author have their intention, I did not revise the content that much

@DoodleBears See code review

Sorry, I did not see the comment in code review?

Can you see it here? https://github.com/imoneoi/openchat/pull/70/files/0bccc1438e99a26afd8059aba121bf8babde8d25

@DoodleBears Merged, great thanks! I've made a little modifications.
BTW I tested this command should work. Will it work in your env?
python -m ochat.serving.openai_api_server --model openchat/openchat_3.5

Yes. it works on my env (WSL)


Add attributes for the colors

There is no way currently to set the colors. It should default to the project attributes for primary color and be editable with XML attributes.

I found that you can override some of the colors in your project using the same name but it doesnt work for all of them. Is better to have an easy way to do it.

@sp00ne, @vscotti: see https://github.com/savvisingh/DateRangePicker/issues/15#issuecomment-419448429.


Setup middleware

What do you think about adding middleware feature for the view controllers setup?
I mean something similar to the proxy but with the intend to use for view controller configuration before destined map. So instead of adding setup to all specific map's we could use desired url that will match specific routes that could be configured the same way without code duplication. Code below is from the example but without presentationSetup closure that was used in routingexample://present... routes.
E.g. :
public func registerRoutes() {
    // Here is the new feature
    router.setupProxy("routingexample://present/*") { (vc, parameters, data) in
        vc.navigationItem.leftBarButtonItem = UIBarButtonItem(barButtonSystemItem: .Cancel,
                                                              target: vc,
                                                              action: #selector(vc.cancel))
    }

    router.map("routingexample://push/login",
               source: .Storyboard(storyboard: "Main", identifier: "LoginViewController", bundle: nil),
               style: .Push(animated: true))

    router.map("routingexample://present/login",
               source: .Storyboard(storyboard: "Main", identifier: "LoginViewController", bundle: nil),
               style: .InNavigationController(.Present(animated: true)))

    router.map("routingexample://push/privilegedinfo",
               source: .Storyboard(storyboard: "Main", identifier: "PrivilegedInfoViewController", bundle: nil),
               style: .Push(animated: true))

    router.map("routingexample://present/privilegedinfo",
               source: .Storyboard(storyboard: "Main", identifier: "PrivilegedInfoViewController", bundle: nil),
               style: .InNavigationController(.Present(animated: true)))

    router.map("routingexample://push/settings",
               source: .Storyboard(storyboard: "Main", identifier: "SettingsViewController", bundle: nil),
               style: .Push(animated: true))

    router.map("routingexample://present/settings",
               source: .Storyboard(storyboard: "Main", identifier: "SettingsViewController", bundle: nil),
               style: .InNavigationController(.Present(animated: true)))

    router.proxy("routingexample://*", tags: ["Views"]) { route, _, _, next in
        if shouldPresentViewControllers {
            let route = route.stringByReplacingOccurrencesOfString("push", withString: "present")
            next(route, nil, nil)
        } else {
            next(nil, nil, nil)
        }
    }

    router.proxy("/*/privilegedinfo", tags: ["Views"]) { route, parameters, data, next in
        if authenticated {
            next(nil, nil, nil)
        } else {
            next("routingexample://present/login?callback=\(route)", parameters, data)
        }
    }

    router.proxy("/*", tags: ["Views"]) { route, parameters, data, next in
        print("opened: route (\(route)) with parameters (\(parameters)) & data (\(data))")
        next(nil, nil, nil)
    }
}

I don't know for now how it could be implemented. Because I wonder whether proxy implementation could be extended to achieve this feature.

Without diving into the code  I think we may be able to accomplish something like, with generics, or a new proxy closure type:
    router.proxy("routingexample://present/*") { (vc, route, parameters, data) in
        vc.navigationItem.leftBarButtonItem = UIBarButtonItem(barButtonSystemItem: .Cancel,
                                                              target: vc,
                                                              action: #selector(vc.cancel))
    }

We'd have to ensure the proxy only gets called in the event of a route that leads to a vc setup. Might take some more thought but I can see the advantages of it.

I looked into the code. What makes it a little bit tricky is that view controllers are initialised in the extended Routing map method which is very convenient. So adding new HandlerType maybe is not the best way to because we could loose current elastic implementation.


Fix broken markdown link in code search documentation

Why:
Closes https://github.com/github/docs/issues/8387
What's being changed:
Fixes a markdown link syntax error
Check off the following:

[ ] I have reviewed my changes in staging (look for the latest deployment event in your pull request's timeline, then click View deployment).
[ ] For content changes, I have completed the self-review checklist.

Writer impact (This section is for GitHub staff members only):

[ ] This pull request impacts the contribution experience

[ ] I have added the 'writer impact' label
[ ] I have added a description and/or a video demo of the changes below (e.g. a "before and after video")

@apexskier Thanks so much for opening a PR with this fix! I'll update the branch and get this merged once tests are passing 🍏


python3Packages.numba: don't default to obsolete shellhooks

Description of changes
autoAddDriverRunpath now exists at the pkgs scope, so it should always be available.
Also, cudaPackages.autoAddDriverRunpathHook and cudaPackages.autoAddOpenGLRunpathHook have been deprecated
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Addresses https://github.com/NixOS/nixpkgs/issues/141803#issuecomment-2065475119


Is there a way to apply "random" filter in our querry feed?

for example for "all unread" feed we use:
"query:Unread Articles:unread = \"yes\""
But can we write expression for random sorting also?

This is not possible because filters' job is to decide what items to display. They can't affect the order of items.
Instead, you should sort the list when it is displayed. In case of a query feed, that'd be an article list. Take a look at keys g (sort operation) and G (rev-sort), and also the setting article-sort-order.
Does that answer your question, @therajat08?

yeah i think g will do. thanks


Access to HostControl

I'm using Topshelf and Quartz and using the application both as a console program (via task scheduler) and as a Windows service.
My jobs are set up as Quartz jobs.
When the application is run as a console program, the job is triggered via StartNow(). I want the application to exit immediately after the job is finished, instead of having to press Ctrl+C. I know that regular Topshelft services can be stopped with HostControl.Stop(), but I can't find a way to access the HostControl from Quartz jobs and can't find a way to set the host in the jobdata either.
I've dug through the Topshelf and Topshelf.Quartz code and as far as I can tell, there is currently no way. Do you know of a simple code enhancement that would enable this?

Any help?

I'm kind of resurrecting a dead thread, but I was looking around for something similar and google brought me hear. What I have done is save a static reference to the HostControl during startup and then use that in my quartz job. There's probably something a little more sophisticated that could be done with a DI container to register the HostControl instance, but I haven't needed that yet. I'd be interested to know what you did.


Blocking tasks (sync or async) leads to ui frozen

Describe the bug
Stacked UI elements are not updated when a task is running, which blocks the main thread. This is the case for both async and sync code. For example, I want to create a spinner that pops up conditionally when I press a button, and place that spinner in a stacked configuration. Instead of showing up, the stacked element greys out. I can see the other element where the spinner is created, that it pops up. For async this is not the case.
This occurs on other elements as well, for instance markdown elements.
Environment
{
"marimo": "0.4.10",
"OS": "Windows",
"OS Version": "11",
"Processor": "Intel64 Family 6 Model 126 Stepping 5, GenuineIntel",
"Python Version": "3.12.2",
"Binaries": {
"Browser": "123.0.6312.123",
"Node": "v14.16.0"
},
"Requirements": {
"click": "8.1.7",
"importlib-resources": "missing",
"jedi": "0.19.1",
"markdown": "3.6",
"pymdown-extensions": "10.7.1",
"pygments": "2.17.2",
"tomlkit": "0.12.4",
"uvicorn": "0.29.0",
"starlette": "0.37.2",
"websocket": "missing",
"typing-extensions": "4.9.0",
"black": "24.3.0"
}
}
Code to reproduce
import marimo

__generated_with = "0.4.10"
app = marimo.App()

@app.cell
def __():
    import marimo as mo
    import asyncio
    from time import sleep
    return asyncio, mo, sleep

@app.cell
def __(mo):
    get_do_something_async, set_do_something_async = mo.state(False)
    get_do_something_sync, set_do_something_sync = mo.state(False)
    return (
        get_do_something_async,
        get_do_something_sync,
        set_do_something_async,
        set_do_something_sync,
    )

@app.cell
def __(mo, set_do_something_async):
    button_do_async = mo.ui.button(label="Do something async", on_click=lambda v: set_do_something_async(True))
    return button_do_async,

@app.cell
def __(mo, set_do_something_sync):
    button_do_sync = mo.ui.button(label="Do something sync", on_click=lambda v: set_do_something_sync(True))
    return button_do_sync,

@app.cell
def __(get_do_something_async, mo):
    ui_do_async_spinner = mo.status.spinner(title="loading") if get_do_something_async() else mo.md("Waiting for async button press")
    ui_do_async_spinner
    return ui_do_async_spinner,

@app.cell
def __(get_do_something_sync, mo):
    ui_do_sync_spinner = mo.status.spinner(title="loading") if get_do_something_sync() else mo.md("Waiting for sync button press")
    ui_do_sync_spinner
    return ui_do_sync_spinner,

@app.cell
def __(get_do_something_sync, mo):
    ui_do_sync_pure_markdown = mo.md("""Running""") if get_do_something_sync() else mo.md("Waiting for sync button press")
    ui_do_sync_pure_markdown
    return ui_do_sync_pure_markdown,

@app.cell
async def __(asyncio, get_do_something_async, set_do_something_async):
    _trigger_async = get_do_something_async()

    async def do_something_async():
        await asyncio.sleep(5)
        print("Done")

    if _trigger_async:
        await do_something_async()
        set_do_something_async(False)
    return do_something_async,

@app.cell
def __(get_do_something_sync, set_do_something_sync, sleep):
    _trigger_sync = get_do_something_sync()

    def do_something_sync():
        sleep(5)
        print("Done")

    if _trigger_sync:
        do_something_sync()
        set_do_something_sync(False)
    return do_something_sync,

@app.cell
def __(button_do_sync, mo, ui_do_sync_spinner):
    mo.vstack([button_do_sync, ui_do_sync_spinner])
    return

@app.cell
def __(button_do_async, mo, ui_do_async_spinner):
    mo.vstack([button_do_async, ui_do_async_spinner])
    return

@app.cell
def __(button_do_sync, mo, ui_do_sync_pure_markdown):
    mo.vstack([button_do_sync, ui_do_sync_pure_markdown])
    return

if __name__ == "__main__":
    app.run()

I'm not 100% sure i follow the use case - i'll try to digest it more. But in the meantime, have you see the mo.output.replace/mo.output.append/mo.output.clear? The allow you to update the cell output (replace/append/clear) before the cell finished.
e.g.
mo.output.replace(mo.status.spinner(title="loading"))

// do things

mo.ouput.replace(mo.md("First result: ", result)

// do more things

result

https://docs.marimo.io/api/outputs.html#cell-outputs

It's related to the example given in https://github.com/marimo-team/marimo/issues/1271,
This works when the action (clicking a button), is inexpensive. However, in my use case I am calling a calculation engine, which takes seconds to return the results. Instead of showing the spinner, it doesn't show up and once the output returns it is gone. I'll take a look at the mo.output, maybe that's what I need..

@mrdobalina2k is this good to close out? not sure if you are still experiencing issues


fix(tex): modify xpath to skip empty row

Solves #918
Also, refactor some fstrings and write some comments

Thanks @grossir


Add BFS Traversal in graph using C++

Please assign me this issue

here you go @biswajit150803 assigned

Make sure to add this into the separate C++ folder


Update index.js

You may want to discard this as it's a bit of a silly PR but it works the same and is a bit smaller. Supports all the way back to Electron 0.32.0.

Thanks for the PR, man.
Now I had a reason to update this and add ESLint as well.
Merged!


API for selecting arrays by block

It might be useful to provide an API to select Dask arrays by block.  This would be useful for operations like selecting all internal blocks, selecting every other block, etc..  Presumably this would use numpy-like slicing on an attribute and return a dask array back:
>>> x = da.arange(20, chunks=(5,))
>>> x.blocks[:2]
dask.array<getitem, shape=(10,), dtype=int64, chunksize=(5,)>

>>> x.blocks[:2]
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

>>> x.blocks[::2].compute()
array([0, 1, 2, 3, 4, 10, 11, 12, 13, 14])

I think that this was previously mentioned by @shoyer, but I can't find the issue.  cc also @jakirkham and @stuartarchibald

Have you seen issue ( https://github.com/dask/dask/issues/3274 )?

So presumably these are similar in that x.blocks[i, j,k] is semantically equivalent to x.to_chunks()[i, j, k]?

This would be very useful, especially if you have some function being mapped that operates using globally aware indexing (potentially from https://github.com/dask/dask/pull/3686) but will only be applicable on the e.g. first column of blocks. Also when streaming it allows the dropping of undersized blocks from a domain with view of waiting for more data to combine with them.

So presumably these are similar in that x.blocks[i, j,k] is semantically equivalent to x.to_chunks()[i, j, k]?

Basically. Was thinking of a nested list initially. Though we have had some success with NumPy object arrays that contain Dask Array chunks.

This would be very useful, especially if you have some function being mapped that operates using globally aware indexing (potentially from https://github.com/dask/dask/pull/3686)...

It could be useful there. We have often used da.index combined with the original array to get at this need. Though this gives you actual index and not block index. Here's a simple example. Though there are more complex ones as well.

Generally this functionality would be a nice alternative for use cases that are simply outside of the typical map_blocks API. For instance, performing operations that reduce each chunk to a scalar, storing individual chunks to different files, operations where the shape might not fit into what map_blocks expects, and more general operations that act differently depending on which chunk is considered.

Yes, I agree that this would be useful.
On Fri, Jun 29, 2018 at 8:10 AM jakirkham<EMAIL_ADDRESS>wrote:

So presumably these are similar in that x.blocks[i, j,k] is semantically
equivalent to x.to_chunks()[i, j, k]?
Basically. Was thinking of a nested list initially. Though we have had
some success with NumPy object arrays that contain Dask Array chunks.
This would be very useful, especially if you have some function being
mapped that operates using globally aware indexing (potentially from #3686
https://github.com/dask/dask/pull/3686)...
It could be useful there. We have often used da.index combined with the
original array to get at this need. Though this gives you actual index and
not block index. Here's a simple example
https://github.com/dask/dask/blob/0.18.1/dask/array/routines.py#L1093-L1096.
Though there are more complex ones as well.
Generally this functionality would be a nice alternative for use cases
that are simply outside of the typical map_blocks API. For instance,
performing operations that reduce each chunk to a scalar, storing
individual chunks to different files, operations where the shape might not
fit into what map_blocks expects, and more general operations that act
differently depending on which chunk is considered.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/3684#issuecomment-401384047, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABKS1m-b00uUIcm7u2qcp2Zrfz9bwpIXks5uBkNbgaJpZM4U7d8m
.

This is implemented in #3689

If any thoughts come to mind, @d-v-b, would be great to hear them. ;)

This looks like it would be quite handy insofar as it obviates the need for kludgy uses ofmap_blocks.  (Especially if iterator support is added)

IDK if we will do iterator support initially, but this is pretty close and works.


How to add background sound in an existing sound and output length will be the smallest?

for direct ffmpeg command I found is something like this.
ffmpeg -i background.mp3 -i main.mp3 -filter_complex "[1:a]asplit=2[sc][mix];[0:a][sc]sidechaincompress=threshold=0.015:ratio=2:level_sc=0.8:release=500:attack=1[compr];[compr][mix]amix" final.mp3
Is it possible to do it using ffmpeg-python?

I have the same question!


[WIP] Support for generating TypeName of an arbitrary ASTNode

WIP: Do not merge yet.
This PR aims to add the capability to generate a TypeName node for any Expression ASTNode. This is primarily done by parsing the typeString property associated with each node, but requires first writing a handy new function resolveAny that can resolve any definition (variable, function, contract, struct, enum, etc..) in the context of any ASTNode. This is useful for building UserDefinedTypeName nodes that need to refer to an exact definition. It will be also useful to remove some duplicated hacky code from scribble. Plan:

[ ] Add resolveAny() function that can resolve any definition by name in the context of an ASTNode with associated tests
[ ] Add new InternalTypeName abstract class, with a subtree of classes describing some internal types that cannot be declared, but can appear as types of intermediate expressions:

                                   ______`InternalTypeName`______
                                 /             |                  \
               `LiteralTypeName`   |      `TupleTypeName`          `TypeTypeName`
                                               |
                                    `BuiltinTypeName`

[ ] migrate the typeString parser from scribble and add the getExprType() function leveraging the typestring parser.
[ ] add associated tests for getExprType()

Codecov Report

Merging #32 (aab09e7) into master (709043c) will decrease coverage by 0.26%.
The diff coverage is 82.87%.

@@            Coverage Diff             @@
##           master      #32      +/-   ##
==========================================
- Coverage   92.82%   92.56%   -0.27%
==========================================
  Files         208      210       +2
  Lines        4184     4330     +146
  Branches      596      653      +57
==========================================
+ Hits         3884     4008     +124
- Misses        186      199      +13
- Partials      114      123       +9

Impacted Files
Coverage Δ

src/ast/definitions.ts
82.01% <82.01%> (ø)

src/ast/index.ts
100.00% <100.00%> (ø)

src/misc/index.ts
100.00% <100.00%> (ø)

src/misc/utils.ts
100.00% <100.00%> (ø)

src/ast/ast_node.ts
91.37% <0.00%> (+0.57%)
:arrow_up:

src/ast/ast_reader.ts
94.38% <0.00%> (+1.12%)
:arrow_up:

src/ast/modern/modifier_invocation_processor.ts
100.00% <0.00%> (+12.50%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 709043c...aab09e7. Read the comment docs.

This is superseded by one of https://github.com/ConsenSys/solc-typed-ast/pull/34 or https://github.com/ConsenSys/solc-typed-ast/pull/35


Running a container with CSI NFS plugin: operation not permitted.

Summary
I've installed rexray CSI NFS plugin for use with Docker.
Once installed and volume created, I cannot run a container with mapping to NFS volune:
[ajn@manager1 ~]$docker run -it --name alpine2 --hostname alpine2 -v rexray-csi-nfs:/mnt alpine sh

docker: Error response from daemon: chown /var/lib/docker/plugins/7b415fee603f793c01c73e7412be87e550e4f73c7778d76d45f3f748e24dfb9c/rootfs/var/lib/rexray/docker/volumes/rexray-csi-nfs: operation not permitted.
See 'docker run --help'.
[ajn@manager1 ~]$

Bug Reports
Version
rexray is not in any binary directory
[ajn@manager1 ~]$ which rexray

/usr/bin/which: no rexray in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/ajn/.local/bin:/home/ajn/bin)

[ajn@manager1 ~]$  sudo /var/lib/docker/plugins/7b415fee603f793c01c73e7412be87e550e4f73c7778d76d45f3f748e24dfb9c/rootfs/usr/bin/rexray version

REX-Ray
-------
Binary: /var/lib/docker/plugins/7b415fee603f793c01c73e7412be87e550e4f73c7778d76d45f3f748e24dfb9c/rootfs/usr/bin/rexray
Flavor: client+agent+controller
SemVer: 0.11.1
OsArch: Linux-x86_64
Commit:<PHONE_NUMBER>ac4426b04941b4002fb33b9efd972e
Formed: Tue, 19 Dec 2017 23:26:34 CET
[ajn@manager1 ~]$

[ajn@manager1 ~]$ docker version

Client:
Version:       17.12.0-ce
API version:   1.35
Go version:    go1.9.2
Git commit:    c97c6d6
Built: Wed Dec 27 20:10:14 2017
OS/Arch:       linux/amd64
Server:
Engine:
Version:      17.12.0-ce
API version:  1.35 (minimum version 1.12)
Go version:   go1.9.2
Git commit:   c97c6d6
Built:        Wed Dec 27 20:12:46 2017
OS/Arch:      linux/amd64
Experimental: false

[ajn@manager1 ~]$ cat /etc/centos-release

CentOS Linux release 7.4.1708 (Core)

[ajn@manager1 ~]$ sudo uname -a

Linux manager1.ajnouri.com 3.10.0-693.17.1.el7.x86_64 #1 SMP Thu Jan 25 20:13:58 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Expected Behavior
Creating a container or a service with volume mapped to NFS, should run without error.
Actual Behavior
[ajn@manager1 ~]$docker run -it --name alpine2 --hostname alpine2 -v rexray-csi-nfs:/mnt alpine sh

docker: Error response from daemon: chown /var/lib/docker/plugins/7b415fee603f793c01c73e7412be87e550e4f73c7778d76d45f3f748e24dfb9c/rootfs/var/lib/rexray/docker/volumes/rexray-csi-nfs: operation not permitted.

Steps To Reproduce
1- Install the plugin
[ajn@manager1 ~]$ docker plugin install rexray/csi-nfs

Plugin "rexray/csi-nfs" is requesting the following privileges:

network: [host]
mount: [/dev]
allow-all-devices: [true]
capabilities: [CAP_SYS_ADMIN]
Do you grant the above permissions? [y/N] y
latest: Pulling from rexray/csi-nfs
9b1e956461c7: Download complete
Digest: sha256:68073617ada9f88334df5a200c7131ee9f3c3e47811ce86075fb5b375897ed08
Status: Downloaded newer image for rexray/csi-nfs:latest
Installed plugin rexray/csi-nfs

2- Create the volume
[ajn@manager1 ~]$ docker volume create -d rexray/csi-nfs -o host=<IP_ADDRESS> -o export=/var/nfs/general rexray-csi-nfs

rexray-csi-nfs

3- run a container with volume mapped to container dir.
[ajn@manager1 ~]$docker run -it --name alpine2 --hostname alpine2 -v rexray-csi-nfs:/mnt alpine sh

docker: Error response from daemon: chown /var/lib/docker/plugins/7b415fee603f793c01c73e7412be87e550e4f73c7778d76d45f3f748e24dfb9c/rootfs/var/lib/rexray/docker/volumes/rexray-csi-nfs: operation not permitted.
See 'docker run --help'.
[ajn@manager1 ~]$

The first step should always be enabling debug logging.

There is no such file as /etc/rexray/config.yml

Configuration Files

There is no such file as /etc/rexray/config.yml

Logs
It is very important when filing an issue to include associated logs. There are
two different logs about which to be concerned: the service log (if REX-Ray is
running as a service) and the client log.
Service Log

There is no such file as/var/log/rexray/rexray.log

Client Log

Thanks for the feeback @AJNOURI. Can you try creating a volume without dashes in the name? I am wondering if there is a problem in the volume name translation.


training about my dataset

Thanks for your open source.  When i training my dataset, I have some problem.
The error is KeyError: 'bg_object_segmentation', then  I Comment out the line about the bg_object_segmentation.
After done the above said, the error is  "AttributeError: Cannot find field 'gt_bo_masks' in the given Instances!"
How can i fix it ? Should I add the "bg_object_segmentation" in json file?

you could refer to the provided script here for producing bilayer annotations, which contain the the field 'gt_bo_masks'.

Thanks,I have found the process.sh and run it .
But it also have the error about KeyError: 'bg_object_segmentation'

And when I use original pascal voc 2012 to training. It have the same error about :
AttributeError: 'Instances' object has no attribute 'pred_boxes'  &&  AttributeError: 'Instances' object has no attribute 'gt_boxes'


bincapz flags rust binary for var/tmp reference

ref: https://github.com/wolfi-dev/os/pull/16138
Source diff: https://github.com/astral-sh/ruff/compare/v0.3.4...v0.3.5
The string itself looks like something from a test 🤔
var/tmp/dev/shmstrptime_replaceexamplessee

What's interesting is that one of the embedded strings is strptime which appears to be a Rust datetime function that figures prominently in the diff.
I wonder how this ended up in the final binary, but flagging this for later.

It turns out that ruff too tries to detect programs that hardcode this directory - which is why bincapz flags it:
https://github.com/astral-sh/ruff/blob/42d52ebbec275389ee0458a3bf373268278bd51a/crates/ruff_linter/src/rules/flake8_bandit/settings.rs#L8

Leaving as a true positive, even if rust is merging everything into a long character mess.