Zurb Foundation 5 Admin Template

Hi, as promised, I have finished a basic zurb foundation 5 template.
I have left many of the classes that were used, since I never use BS, when I wasn't sure if they were BS classes or utility ones I just left them. This can be polished.

I have opinionatedly decided to use a fixed top menu bar
I have ised svg iconfonts (foundation ones) and haven't set a png fallback because of how Foundation handles it, it's not just a class, it's a diferent tag. Since Foundation 5 is supposed to aim at the latest browsers I thought this seemed coherent. Still we can work a way to check a variable with modernizer and chage it.
I have added an app.css file with little style changes I had to force for everything to look nice.
I have tried to test the template with all the examples, but I wasn't able to do so with some of them.
It has some problems on mobile, but they are the sam BS3 is having, it would need to reformat how the filters and their forms are being produced.
I need to find a way to control the width of the columns on the tables without hacking them.
Forms and Tables don't need clases to be Foundation styled.

I hope this helps the community, I can keep on collaboarting with this, I will use it on a real project at our office as soon as it's merged, so I will be able to test it properly and add suggestions.
Have a nice wek!

:+1: Nice! waiting for this, I will do some tests

:+1:

Can you also add support for modals?
Also looks like this is going to break column_editable_list (http://flask-admin.readthedocs.org/en/latest/api/mod_model/#flask_admin.model.BaseModelView.column_editable_list): https://github.com/vitalets/x-editable/issues/10

Hi Paul, will check that as soon as I can. To help me do it faster, could you point to me an example where modals are boing used with Bootstrap?

Here are some examples:
Modal macros in lib.html to create buttons and the modal window:
https://github.com/flask-admin/flask-admin/blob/master/flask_admin/templates/bootstrap3/admin/lib.html#L100
Adding modal window to list.html:
https://github.com/flask-admin/flask-admin/blob/master/flask_admin/templates/bootstrap3/admin/model/list.html#L175
Check if modal is enabled and change presentation in edit.html and create.html:
https://github.com/flask-admin/flask-admin/blob/master/flask_admin/templates/bootstrap3/admin/model/edit.html#L18
https://github.com/flask-admin/flask-admin/blob/master/flask_admin/templates/bootstrap3/admin/model/create.html#L18
Adding buttons to list.html: https://github.com/flask-admin/flask-admin/blob/master/flask_admin/templates/bootstrap3/admin/model/list.html#L22
I think you're going to do most of it with javascript since Foundation doesn't load stuff from data-remote by default like Bootstrap does: http://foundation.zurb.com/forum/posts/1399-adding-reveal-links-via-javascriptajax

Thanks Paul,
will check this after the weekend. Have to run some tests. Do I make another PR once this is solved?

@ondoheer, no need for another PR. We can keep this open as long as you need.

Hello Petrus,
got that. I have run into some problems.

I haven't managed to get Flask-admin  1.2.0 to show me modals for editing nor creating when using BS2 or BS3, I tried modifying the examples and then a personal project of mine. It's like it's not reading the properties and the template is just showing me the regular links (no modal=true on the URI).

The modals are being created on my theme, but the content comes out empty, I wanted to know how you guys are handling it.

I'm still working on getting x-editable to work with plain jQuery or jQuery-UI

@ondoheer The modal window feature is brand new, so it's not available on the latest pypi build. To try it out, you'll have to install the latest version straight from the github repo. You can do this with pip:
pip install git+https://github.com/flask-admin/flask-admin.git@master

Then, if you pip freeze you should see:
Flask-Admin==1.2.1-dev

Hope that helps.

Sorry, I didin't realize this master wasn't synced with PyPi, that explains a few "bugs" I encountered! Thanks for that.

Hello guys,
just found some more time to work on the project today, I have stomped against some problems:

I'm using x-editable jqueryui version, but it seems to run into Bug 687 . I updated the js as stated in PR 724 but that just managed to silence the JS error.

I tried to use pure jquery x-editable but that just caused more problems. It seems the problem has to do with setting x-editable to popup mode. I tried to find where to change that to inline mode in the flask.admin code, but didin't find it. Before just overriding anytrhing I wanted to consult with you where I should do this and how.

I have updated the modal templates, they now work foundation style, but they don't extend the templates with models/modal/{create, edit}.html as they should. Haven't found why yet. Any help would be appreciated.

I have tested the modals with layout from BS2, they appear and render the form, but they aren't working when trying to send the data. I have been working over BS2 templates because I always read  the project managers don't feel BS3 is ready to overtake BS as the default theme.

Any help getting this to work would be awesome, I'm, kind of swamped at the office and would like to help this get through as best as I can.

Any updates?

Hello Miguel,
for personal reasons I couldn't finish this, we were doing it so it became
one of the comany tools.
On Sun, Aug 6, 2017 at 6:51 AM, Miguel Piedrafita<EMAIL_ADDRESS>wrote:

Any updates?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/flask-admin/flask-admin/pull/937#issuecomment-320502161,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADgsFBvJZIXn4DIPfIowHUp5tAeGDmO2ks5sVajNgaJpZM4FUn0m
.


Add Remove Old Artifacts

https://github.com/c-hive/gha-remove-artifacts

CI failing due to unrelated issue

@sdras Anything I can do to move this along?

Nice addition, thanks!


Package Deployer Error

When I run the PackageDeployer.exe file and I click Continue on the initial screen, I get the following error on the next screen...  No import packages were found.  Exiting the application.
How can I resolve this?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e56ea0df-2e9a-989e-92a8-375ac6b00da4
Version Independent ID: de0efc1d-0813-029f-223b-9a7b4af55b6f
Content: Prerequisites for Connected Field Service for IoT Central
Content Source: ce/field-service/cfs-iot-central-setup.md
GitHub Login: @krbjoran
Microsoft Alias: krbjoran

Hi @shaileshwath , can you try to tick the "Unblock" button before you unzip the file? I think this will solve the issue. Thanks!


Fixed typo in it's

because you probably didn't mean it is

You may disregard these recommendations if you used the Edit on GitHub button from dvc.org to improve a doc in place.

❗ Please read the guidelines in the Contributing to the Documentation list if you make any substantial changes to the documentation or JS engine.
🐛 Please make sure to mention Fix #issue (if applicable) in the description of the PR. This causes GitHub to close it automatically when the PR is merged.
Please choose to allow us to edit your branch when creating the PR.
Thank you for the contribution - we'll try to review it as soon as possible. 🙏

thanks @himanshugarg !


Update no-ssr to v0.2.1

Upgrade no-ssr to v0.2.1, which makes it able to conditionally render no-ssr components.
Example:
<no-ssr>
    <some-component v-if="showComponent"></some-component>
</no-ssr>

Codecov Report

Merging #2181 into dev will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##              dev    #2181   +/-   ##
=======================================
  Coverage   98.04%   98.04%
=======================================
  Files          15       15
  Lines         970      970
=======================================
  Hits          951      951
  Misses         19       19

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9bf73b0...aab684f. Read the comment docs.

Awesome! Thank you @danielpost


deduplicate variadic buffers in MutableArrayData::extend for ByteView arrays

Which issue does this PR close?

Closes #.
Rationale for this change
MutableArrayData adds all variadic buffers from input arrays together,  potentially duplicating the same buffers in the output array.

What changes are included in this PR?
extend now checks if the same buffer is added from some other input array and changes the views to be appended to point to the new deduplicated buffer indices

Are there any user-facing changes?

@tustvold happy to add tests if this approach is more inline with what you had in mind in https://github.com/apache/arrow-rs/pull/6779

@alamb @tustvold I did add a string view case for the interleave benchmark and ran on main, this PR (interleave-deduplicated), and https://github.com/apache/arrow-rs/pull/6779 (interleave-specific-impl)
❯ critcmp interleave-main interleave-deduplicated interleave-specific-impl
group                                                                                interleave-deduplicated                interleave-main                        interleave-specific-impl
-----                                                                                -----------------------                ---------------                        ------------------------
interleave string_view(0.5, 50, true) 100 [0..100, 100..230, 450..1000]              2.33      2.3±0.02µs        ? ?/sec    1.82  1767.6±25.30ns        ? ?/sec    1.00    968.7±4.97ns        ? ?/sec
interleave string_view(0.5, 50, true) 1024 [0..100, 100..230, 450..1000, 0..1000]    1.80     13.9±0.17µs        ? ?/sec    1.35     10.4±0.11µs        ? ?/sec    1.00      7.7±0.11µs        ? ?/sec
interleave string_view(0.5, 50, true) 1024 [0..100, 100..230, 450..1000]             1.80     13.3±0.13µs        ? ?/sec    1.39     10.3±0.10µs        ? ?/sec    1.00      7.4±0.09µs        ? ?/sec
interleave string_view(0.5, 50, true) 400 [0..100, 100..230, 450..1000]              1.93      5.8±0.05µs        ? ?/sec    1.49      4.5±0.05µs        ? ?/sec    1.00      3.0±0.02µs        ? ?/sec

I believe the penalty introduced by this PR would be mitigated for interleave's case if we also merge #6779, for other cases it feels like the read / transfer over the wire improvements might outweigh the cost. Happy to hear your thoughts

Thank you @onursatici  -- I hope to find time to review this PR this weekend or early next week

@alamb no worries and thank you for having a look. I added some tests now checking the deduplication and remapping behaviour, let me know whenever you have time if this looks good, happy holidays!


join room when listen all public + using bot name

if you use: botname help and the bot is not in the room it fails on server side to send the reply.
if you use @botusername help rocket.chat adds the user to the room prior to sending the message through and hubot recieving.
https://github.com/RocketChat/Rocket.Chat/blob/develop/packages/rocketchat-lib/server/methods/sendMessage.js#L40
It checks to see if the user has a subscription (ie in the room) if they aren't it fails.
So if you use botname instead of @botusername we need to have the bot join the room so it will have a subscription.  Of course this will only work for public rooms, private will still have to be added to the room to see or respond

Please assign this to me


Fix the handling of unsigned values on MySQL

We were sending the right bytes, but we weren't properly setting the
is_unsigned flag on the bind parameters, resulting in it always being
sent as signed. This results in a client error about overflow when
deserializing large values, and either a server error or incorrect
results when serializing them.
The fact that the tests for this all used -1 should have been a huge
red flag, but it slipped through.
Unfortunately, there's no way for us to reasonably fix this without a
breaking change or crazy hacks (we could do something like write an
additional byte at the end if it's unsigned, but that's probably not
worth it to avoid a breaking change). For that reason, we will need to
make the next version of Diesel 2.0. We shouldn't go crazy with this
change, but this would also allow us to make #1457 happen as well.
The other alternative would be to introduce a Mysql2 backend, and
deprecate the current one.
Fixes #1745

@sgrif given that this previously caused an error, my view is that it should be introduced as a point release. If you consider the number of applications which are inadvertently broken today and would be fixed by this change vs programs which would be broken by introducing this change, the former is likely to be quite large, and the latter is likely to be zero.
That is, unless it is possible today to round-trip a value through the database without an error, and get a different result from what you'd get with this change applied?

@Diggsey The problem is that it's not just a change in behavior, it could cause code which was previously compiling to no longer compile. While I do think that it's incredibly unlikely that anyone is exhaustively matching this enum (there's no reason to do so unless you're implementing a connection), it's documented and part of our public API. Specifically this enum is structured to allow exhaustively matching, so I think it would be against the spirit of our commitment to semver if we can avoid it. (We can avoid it, see #1749)

Makes sense 👍


[dr-autosync] after running in sync_recovery for 10min, hit 9005 error

Bug Report

What version of TiKV are you using?

[2023/08/17 03:00:49.417 +08:00] [INFO] [lib.rs:90] ["Release Version:   6.5.3"]
[2023/08/17 03:00:49.417 +08:00] [INFO] [lib.rs:90] ["Edition:           Community"]
[2023/08/17 03:00:49.417 +08:00] [INFO] [lib.rs:90] ["Git Commit Hash:   cb0dab3e4e0f3081966e871057272a40aa7ffc2d"]
[2023/08/17 03:00:49.417 +08:00] [INFO] [lib.rs:90] ["Git Commit Branch: heads/refs/tags/v6.5.3-flashback"]
[2023/08/17 03:00:49.417 +08:00] [INFO] [lib.rs:90] ["UTC Build Time:    Unknown (env var does not exist when building)"]
[2023/08/17 03:00:49.417 +08:00] [INFO] [lib.rs:90] ["Rust Version:      rustc 1.67.0-nightly (96ddd32c4 2022-11-14)"]
[2023/08/17 03:00:49.417 +08:00] [INFO] [lib.rs:90] ["Enable Features:   pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure"]
[2023/08/17 03:00:49.417 +08:00] [INFO] [lib.rs:90] ["Profile:           dist_release"]
What operating system and CPU are you using?

Steps to reproduce

What did you expect?
No 9005 error reported.
What did happened?
[2023/08/17 05:46:31.446 +08:00] [INFO] [utils.go:60] ["current status is "] [curStatus=sync_recover]
....
[2023/08/17 05:47:11.429 +08:00] [INFO] [workload.go:771] ["[2023-08-16 21:47:11] execute run failed, err exec SELECT c_discount, c_last, c_credit, w_tax FROM customer, warehouse WHERE w_id = ? AND c_w_id = w_id AND c_d_id = ? AND c_id = ?(wID=827,dID=1,cID=1750) failed Error 9004: Resolve lock timeout\r\n"]
[2023/08/17 05:47:15.080 +08:00] [INFO] [workload.go:771] ["[2023-08-16 21:47:15] execute run failed, err exec UPDATE warehouse SET w_ytd = w_ytd + ?  FOR UPDATE failed Error 9005: Region is unavailable\r\n"]
[2023/08/17 05:47:15.091 +08:00] [INFO] [workload.go:790] ["workload run with errors."]
...
[2023/08/17 05:47:31.460 +08:00] [INFO] [utils.go:60] ["current status is "] [curStatus=sync_recover]

tidb1-0/tiup.deploy.tidb-4000.log/log/tidb.log:[2023/08/17 05:47:10.919 +08:00] [WARN] [session.go:2242] ["run statement failed"] [conn=3046803895171941323] [schemaVersion=64] [error="[tikv:9005]Region is unavailable"] [session="{\n  \"currDBName\": \"tpcc\",\n  \"id\":<PHONE_NUMBER>171941323,\n  \"preparedStmtCount\": 63,\n  \"status\": 3,\n  \"strictMode\": true,\n  \"txn\": \"443605946167722038\",\n  \"user\": {\n    \"Username\": \"root\",\n    \"Hostname\": \"<IP_ADDRESS>\",\n    \"CurrentUser\": false,\n    \"AuthUsername\": \"root\",\n    \"AuthHostname\": \"%\"\n  }\n}"]
tidb1-0/tiup.deploy.tidb-4000.log/log/tidb.log:[2023/08/17 05:47:10.919 +08:00] [INFO] [conn.go:1181] ["command dispatched failed"] [conn=3046803895171941323] [connInfo="id:3046803895171941323, addr:<IP_ADDRESS>:55966 status:11, collation:utf8mb4_general_ci, user:root"] [command=Execute] [status="inTxn:1, autocommit:1"] [sql="SELECT s_i_id, s_quantity, s_data, s_dist_01, s_dist_02, s_dist_03, s_dist_04, s_dist_05, s_dist_06, s_dist_07, s_dist_08, s_dist_09, s_dist_10 FROM stock WHERE (s_w_id, s_i_id) IN ((?,?),(?,?),(?,?),(?,?),(?,?),(?,?),(?,?)) FOR UPDATE [arguments: (893, 4142, 893, 72494, 893, 90084, 893, 23585, 893, 18465, 893, 6014, 893, 57382)]"] [txn_mode=PESSIMISTIC] [timestamp=443605946167722038] [err="[tikv:9005]Region is unavailable\nSELECT s_i_id, s_quantity, s_data, s_dist_01, s_dist_02, s_dist_03, s_dist_04, s_dist_05, s_dist_06, s_dist_07, s_dist_08, s_dist_09, s_dist_10 FROM stock WHERE (s_w_id, s_i_id) IN ((?,?),(?,?),(?,?),(?,?),(?,?),(?,?),(?,?)) FOR UPDATE [arguments: (893, 4142, 893, 72494, 893, 90084, 893, 23585, 893, 18465, 893, 6014, 893, 57382)]\ngithub.com/pingcap/tidb/server.(*clientConn).executePreparedStmtAndWriteResult\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn_stmt.go:288\ngithub.com/pingcap/tidb/server.(*clientConn).executePlanCacheStmt\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn_stmt.go:225\ngithub.com/pingcap/tidb/server.(*clientConn).handleStmtExecute\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn_stmt.go:217\ngithub.com/pingcap/tidb/server.(*clientConn).dispatch\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1430\ngithub.com/pingcap/tidb/server.(*clientConn).Run\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1152\ngithub.com/pingcap/tidb/server.(*Server).onConn\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/server.go:648\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"]

/assign @nolouch
/severity major
Mark it major or can't reproduce till now.

same with https://github.com/tikv/tikv/issues/15366, so close it.


[0.18.1] Optimize FileWriteOutBytes to avoid high system cpu usage (#9722)

Backports the following commits to 0.18.1:

Optimize FileWriteOutBytes to avoid high system cpu usage (#9722)

Only conflict in this backport was processing/src/test/java/org/apache/druid/segment/writeout/FileWriteOutBytesTest.java which did not exist in 0.18.1


"ForEach-Object -Parallel" problem with "6>$null"

Q:"ForEach-Object -Parallel" released, the first thing you think of is?
A:ping test
Too many people will test "ForEach-Object -Parallel" with Test-Connection
Q:How to hide the output? use Test-Connection
A:
you must use "6>$null"
eg:
Test-Connection  -Count 1 -TargetName bing.com -Quiet
Pinging bing.com [<IP_ADDRESS>] with 32 bytes of data:
Reply from <IP_ADDRESS>: bytes=32 time=43ms TTL=128
Ping complete.
True
Test-Connection  -Count 1 -TargetName bing.com -Quiet 6>$null
True
Q:Combination "ForEach-Object -Parallel" and "6>$null" problem?
A:
yes!so, "ForEach-Object -Parallel" need update.
hi @PaulHigin  ,this is my feedback,have a look,thanks.
Steps to reproduce
Get-Content /powershell/71.txt |ForEach-Object -Parallel {if (Test-Connection -Count 1 -TargetName $_ -Quiet  6>$null) {$_} }
Expected behavior
Get-Content /powershell/71.txt |ForEach-Object  {if (Test-Connection -Count 1 -TargetName $_ -Quiet  6>$null) {$_} }

<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>                                                                                <IP_ADDRESS>                                           <IP_ADDRESS>

Actual behavior
Pinging <IP_ADDRESS> [<IP_ADDRESS>] with 32 bytes of data:
Pinging <IP_ADDRESS> [<IP_ADDRESS>] with 32 bytes of data:
Pinging <IP_ADDRESS> [<IP_ADDRESS>] with 32 bytes of data:
Pinging <IP_ADDRESS> [<IP_ADDRESS>] with 32 bytes of data:
Pinging <IP_ADDRESS> [<IP_ADDRESS>] with 32 bytes of data:
Reply from <IP_ADDRESS>: bytes=32 time=246ms TTL=128
Ping complete.
<IP_ADDRESS>
Reply from <IP_ADDRESS>: bytes=32 time=259ms TTL=128
Ping complete.
<IP_ADDRESS>
Reply from <IP_ADDRESS>: bytes=32 time=241ms TTL=128
Ping complete.
<IP_ADDRESS>
Reply from <IP_ADDRESS>: bytes=32 time=271ms TTL=128
Ping complete.
<IP_ADDRESS>

Environment data
centos76 + ps v7.0 preview3
reporter
By the chinese powershell study & research group. QQ group No. is 183173532

You can also accomplish hiding the output of TestConnecction with -InformationAction 'Ignore', but I agree that the stream redirect should probably work.

@awakun it work,thanks.
Maybe we can't let Test-Connection return that information, at -Quiet

Yeah. I have a WIP PR that accomplishes this. I need to bring it back up and fix whatever's going awry with the failing test so we can get that merged and sorted for 7.0

@chuanjiao10   Thanks for report.  To suppress the information data stream try doing this:
Get-Content /powershell/71.txt |ForEach-Object -Parallel {if (Test-Connection -Count 1 -TargetName $_ -Quiet) {$_} } 6>$null

remove Resolution-Answered  label
this fixed via  https://github.com/PowerShell/PowerShell/pull/10478


any way to profile memory usage in dlio?

Is there any way to track memory usage in a DLIO benchmark process?
I ask because I have a test node with 256 GB of memory.   When running unet3d with 9375 files, I see the MPI process grow from 5.5 GB to 11 GB relatively quickly.   Each of the 4 spawn reader processes is 1.5 GB.  That works out to about 18 GB per MPI process.   I can run 12 MPI processes with DAOS and get full accelerator efficiency, but I can't scale up farther because the machine runs out of memory, and processes are too big to swap, I have to reboot the node to get it working again.
My thought was to try to track down what causes the jumps in memory size in the MPI process, maybe there is a way to reduce the amount of RSS space consumed, but if this has been done before, I'd rather learn from the experts.  :-)

Can u disable persistent workers within the data loader to false. I was seeing such memory increases with the persistent workers as pytorch keeps spawning more workers while keeping previous workers alive

So the memory usage is caused by the pin_memory=True parameter in the DataLoader() instantiation in torch_data_loader.py.  If changed to False, memory usage drops from 11+GB to 2.7 GB.  Individual steps are a tad slower, from 1.36 seconds up to 1.40 seconds.  My config has 4 reader_threads, there are 2 samplers per thread, so pytorch is adding 1 GB of memory to the process for each reader_thread/sampler combo.
Any idea what is going on when memory is pinned, what is different about the behavior versus when not pinned?   Why is 1 GB being pinned per reader_thread/sampler combo when the tensors are only a few Kbytes in size?   I'd like to understand this better.

My understanding the persistent workers + memory pinning was screwing things up.
With the new changes I recently did, it might have been fixed.

Nope, I just upgraded to today's main branch and reran with top (shift M) running in a separate window.  Parent process RES value is 11.2 Gbytes, SHR is 9.5 Gbytes, VIRT is 22.0 Gbytes.
PID     USER     PR  NI  VIRT    RES    SHR    S %CPU  %MEM  TIME+     COMMAND
135981  root     20  0   22.0g   11.2g  9.5g   S 8.0   4.5   0:38.18   python3
136124  root     20  0   12.0g   3.2g   1.0g   R 56.1  1.3   1:20.03   python3
136187  root     20  0   11.6g   2.8g   597736 R 14.0  1.1   1:22.17   python3
135998  root     20  0   11.5g   2.7g   597644 S 8.3   1.1   1:21.67   python3
136061  root     20  0   11.5g   2.7g   597644 S 43.5  1.1   1:23.41   python3

The memory of the parent process starts to grow immediately as it walks through the 4 child processes, 2 samplers each, sending the "start sending tensors" message on the sockets.   By the time all 8 messages have been sent, the parent process is already near the 11 GB size.   This is in the very first epoch, so I don't think persistent workers has any effect here, the problem occurs with the first set of workers.
I will mention that my test node does happen to have a GPU in it.   The torch code uses both pin_memory=True plus it searches for an attached GPU, and if both are true, then pin_memory is activated.   I mention that in case your test machines do NOT have GPUs in them, I don't think pin_memory would have an effect in that case even if pin_memory=True.

The pin_memory call is being made in pin_memory() in python3.9/site-packages/torch/utils/data/_utils/pin_memory.py at the first parameter check, in which device is None:
    if isinstance(data, torch.Tensor):
        return data.pin_memory(device)

I can't step into the data.pin_memory() code because it's C++, but I can see memory allocation occurring there.  At the moment I haven't figured out how to dig deeper, see the actual memory allocations, why they happen.  Of course I expect some allocation, incoming samples are copied to a Queue() in pinned memory, but I don't see why 8 GB would be allocated given that each step contains only 4 samples.   The main process grows to 11 Gbytes for RES quickly and then stays steady after that for all subsequent epochs, so there is not a continuous memory leak.

I changed the number of reader_threads from 4 to 8.   In this case the parent process RES size jumped from 11 GB to 19.2 GB, which is about another 8 GB, so the expansion size of the main process appears to go up by 1 GB for each reader_thread/sampler combo.  Does each reader_thread/sampler combo get a separate Queue() of pinned memory, sized at 1 GB each?

@hariharan-devarajan not sure, is a name-ping required for a person to see additions to a ticket, other than the creator?
Anyway, I would be interested in your thoughts on the above.  I am running the unet3d benchmark, with num_files_train set to 9376, multiprocessing_context is fork.

@krehm This seems odd. Couple of things could be happening. But do u think this could be a garbage collection issue? if so, can u do del of the numpy array data once it's returned to free memory reference for garbage collection?
For my testing, I generally use a GPU-based machine itself, but maybe I didn't come across this due to the number of files and we have large memories on our clusters.
Maybe u can do a Valgrind run with more workers on one process and see if u spot any memory leaks. NOTE: I have never tried it with a Python app, let alone a Pytorch app.

Another quick thing to check is if your samples are smaller(say 4096) would the memory consumption be proportional ?

I am running a single DLIO accelerator instance.  I reduced the number of files from 9375 to 168, it made no difference.  By step 5 in the first epoch, the main process is already 11.1 GB in size.  The child processes are 2.7 GB, which is what the parent was just before it started talking to the child processes.  Throughout the rest of the benchmark, the main process does not grow further in size, so it is not a simple continuous memory leak.
I thought it might have something to do with the pytorch getCachingHostAllocator used with cuda, just a wild guess.  I tried setting PYTORCH_NO_CUDA_MEMORY_CACHING=1 but it made no difference.

Another hunch. What is the sample size? The default sample size is 200MB, right? Also, is there any prefetching cache setup? If so, what is its value?
A couple of things to try again.

Make smaller samples. It will reduce memory, hopefully.
Set the prefetch cache to 0 to force no memory caching for the data loader.

Is the 1GB also corresponds to read threads u use? What is the relation between number of read threads and memory.
I know prefetch factor is not per process but per read thread

If I double the number of reader threads, the growth doubles.   Or if I double the prefetch_factor, the growth doubles.   So the growth corresponds to the number of samples in the pin_memory queue at any point in time.

As far as I know there is no way to set the prefetch cache size in bytes. I
will try and look the documentation.
On Wed, Feb 28, 2024, 11:45 AM krehm @.***> wrote:

If I double the number of reader threads, the growth doubles. Or if I
double the prefetch_factor, the growth doubles. So the growth corresponds
to the number of samples in the pin_memory queue at any point in time.
—
Reply to this email directly, view it on GitHub
https://github.com/argonne-lcf/dlio_benchmark/issues/141#issuecomment-1969724564,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFB2NRAI3UYPF5Z5YM2CYGLYV6CMLAVCNFSM6AAAAABCHPWVLSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRZG4ZDINJWGQ
.
You are receiving this because you were mentioned.Message ID:
@.***>


tensorflow 2.0

tensorflow2.0
Traceback (most recent call last):
File "F:/workspace/python/tf-fit/keras_plus/optimizer/test.py", line 26, in
model.fit(x_train, y, epochs=5)
File "E:\Anaconda3_5_0_0\envs\DeepLearning\lib\site-packages\tensorflow\python\keras\engine\training.py", line 873, in fit
steps_name='steps_per_epoch')
File "E:\Anaconda3_5_0_0\envs\DeepLearning\lib\site-packages\tensorflow\python\keras\engine\training_arrays.py", line 151, in model_iteration
f = _make_execution_function(model, mode)
File "E:\Anaconda3_5_0_0\envs\DeepLearning\lib\site-packages\tensorflow\python\keras\engine\training_arrays.py", line 521, in _make_execution_function
return model._make_execution_function(mode)
File "E:\Anaconda3_5_0_0\envs\DeepLearning\lib\site-packages\tensorflow\python\keras\engine\training.py", line 2231, in _make_execution_function
self._make_fit_function()
File "E:\Anaconda3_5_0_0\envs\DeepLearning\lib\site-packages\tensorflow\python\keras\engine\training.py", line 2177, in _make_fit_function
'_fit_function', [self.total_loss] + metrics_tensors)
File "E:\Anaconda3_5_0_0\envs\DeepLearning\lib\site-packages\tensorflow\python\keras\engine\training.py", line 2149, in _make_train_function_helper
params=self._collected_trainable_weights, loss=self.total_loss)
File "F:\workspace\python\tf-fit\keras_plus\optimizer\lamb.py", line 65, in get_updates
self.weights = [self.iterations] + ms + vs
File "E:\Anaconda3_5_0_0\envs\DeepLearning\lib\site-packages\tensorflow\python\keras\optimizer_v2\optimizer_v2.py", line 507, in setattr
super(OptimizerV2, self).setattr(name, value)
AttributeError: can't set attribute

Keras 2.2.4 does not yet support Tensorflow 2.0.


[Grid] Item sizing in column-container shrinks items width.

[x] This is not a v0.x issue.
[x] I have searched the issues of this repository and believe that this is not a duplicate.

Expected Behavior
Using Grid to define column layout, I expect items with sizing props (like xs={4}) to have dynamic height but the width should be constant.
Current Behavior
Currently, using props like mentioned above, provides css property of max-width for the item (together with flex-basis) while it shouldn't. If ever, for column variant, it should be max-height, I guess.
Steps to Reproduce
Here is the sandbox with  primitive example, as a presentation of the case.
Link: https://codesandbox.io/s/q9pzw7o15w
BTW: What is the correct way to make Grid column container to fill whole available space? I don't like using height: 100% explicitly, like in the example.

Tech
Version

Material-UI
v1.3.1

React
v16.4.1

@Filson14 From what I understand, you are hitting a limitation of the CSS flexbox model and our abstraction on top of it. The xs, sm, md, lg, md properties focus on the width, not the height.

So this is really an expected behaviour? As you can see in the example, setting xs for items in column container results in managed height (because of flex-basis) AND width (because of your max-width). If those properties should focus on the width, why they have influcence on the components height (which in my opinion should be the point of focus in column variant)?
Thanks for quick response tho, @oliviertassinari :wink:

@oliviertassinari I think there are two aspects to this issue: there are some things the Grid is doing when direction=column that it shouldn't (bug), and there are some things that Grid doesn't try to do when direction=column as thoroughly as for direction=row. I think @Filson14 is primarily concerned about the bug aspect of this.
The issue is in generateGrid currently in lines 53-57 of Grid.js:
    styles[key] = {
      flexBasis: width,
      flexGrow: 0,
      maxWidth: width,
    };

Then these keys get referenced in lines 231-235 like:
      [classes[`grid-xs-${String(xs)}`]]: xs !== false,

The problem is that these style keys do not take the flex-direction into account, so whether the direction is "row" or "column", the maxWidth is being set as if the direction is "row". In order to address this bug, generateGrid needs to make the direction part of the style key and only include the maxWidth property when the direction is "row".
As long as you are in agreement that this general approach makes sense, I'll take care of a pull request for this sometime in the next week or two. For now, I would not attempt to set maxHeight instead of maxWidth for "column" direction, I would just avoid the erroneous width setting.

@ryancogswell I'm not sure to fully understand the problem. What's the use case for X different than false in this configuration?
<Grid container direction="column">
  <Grid item xs={X}>

This is more painful to fix than I realized at first. The complication is that direction is specified on the container (i.e. parent) Grid but the maxWidth is being applied on the items within it.

We can work around the problem using CSS selectors.

@oliviertassinari Your latest comment combined with your initial comment on this issue cleared up some things for me. I think this is primarily a documentation issue.
@Filson14's sandbox contains the following:
<Grid container direction="column" xs justify="space-between">
            <Grid item style={green} xs={8}>
              A
            </Grid>
            <Grid item style={blue} xs={4}>
              B
            </Grid>
</Grid>

I was focusing on the fact that the xs={8} and xs={4} have surprising effects in that they are changing the width of the two Grid items even though they are in a container with direction="column". I was trying to come up with a solution to avoid this, but what I think I understand you saying is that it just doesn't make sense to do this. breakpoint={1-12} just isn't supported with direction="column". The main problem then is just that the documentation doesn't make this very clear. It would be helpful to lay this out explicitly in the Limitations section and probably something in the API documentation as well.

I'll do a pull request sometime in the next week or so with a stab at the documentation changes.

@ryancogswell Your analysis of the situation and plan of action sounds perfect to me.

@oliviertassinari When making changes to the docs, I'm not clear on the process for dealing with the translations. Should I just make the change to, for example, grid.md, or should I be duplicating the English change to the translation files as well, or does something in the translation process automatically detect when translation files are older than the English and need to be updated? I didn't see anything yet in the Contributing guide covering this. For now, I'll move forward on the assumption that I should only change the original, English files.

@ryancogswell You should ignore the -x.md markdown files. Crowdin syncs the different versions. Your assumption is correct.

I just stumled upon this issue while searching for a reason of this unexpected behaviour. I don't truly understand why breakpoint={1-12} does not make sense with direction="column" for particular uses. I work with a fixed height container and this could be really usefull to use that functionality.

We can work around the problem using CSS selectors.

@oliviertassinari I think this could be a nice workaroud.
I don't know if it's the correct way to handle this, but what worked for me is to apply to the container the following class:
	verticalContainer: {
		'& .MuiGrid-item': {
			maxWidth: 'none',
		},
	},

The change looks safe 🤔.

Note that we have a follow-up issue in #18225.


Enable a partial import

Currently, an import is all or nothing. Would be nice to allow for a partial import, could come in handy for Getty vocabs. Probably especially useful for importing only The children of a certain collection?
Perhaps by passing in a query as normally passed to the find method of a provider? And by default falling back to all?

@BartSaelen @dieuska Thoughts?


Is dependencies possible?

Are we able to connect items with other items? like shown on the screenshot below:

Hi, any updates on this? I also would like to know if its possible to connect items with others.
Thanks in advance,
Guilherme.

I don't think this is natively supported in the library. But we used another library with  this one to show some kinda of dependency. I don't know if this helps.

Thanks, I'll check it out!

@menasiephrem thanks for your comment. I'm new at react. Can you write a short example how you can implement it? I have both libraries imported but I don't know how to combine both. Would appreciate an answer. Thanks!


WebSocket connection to 'ws://IPDEVSERVER:8080/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED, ok on localhost

Hello ALL !
My project is ok on my local but i try to push my project on my dev server ...
this is my config.yml file
gos_web_socket:
server:
port: 8080        #The port the socket server will listen on
host: <IP_ADDRESS>   #The host ip to bind to
router:
resources:
- @ApplicationFrontFrontBundle/Resources/config/pubsub/routing.yml
So i launch my gos server like this
root@dev:/var/www/dev.xxxx.com/#php app/console gos:websocket:server
[2017-07-19 14:24:59] websocket.INFO: Starting web socket
[2017-07-19 14:24:59] websocket.INFO: Launching Ratchet on <IP_ADDRESS>:8080 PID: 24474
Server seems to be ok but when i try to connect like this
var ws = WS.connect("ws://IPDEVSERVER:8080"); // i try also with hostname dev.** but same error
ws.on("socket/connect", function(session){
});
I have error in my console
WebSocket connection to 'ws://IPDEVSERVER:8080/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED
I have check my firewall but there is nothing ( by journalctl -xe )
thx for your help !!!
philou

Try binding to <IP_ADDRESS>

thx BunnyHolder ! perfect !

Hello all, I am having the same issue.
I use server & pusher wamp IPs as <IP_ADDRESS> (or <IP_ADDRESS>) and for the js part at WS.connect(); I use the <IP_ADDRESS> (or <IP_ADDRESS>) (I have tested to use all 4 combos) However while the server starts with no problem client console gives me this error:
WebSocket connection to 'ws://X.X.X.X:8080/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED
At my local machine I have user everywhere <IP_ADDRESS> and it works fine, but when deploying I get this error (I use an AWS EC2 server) and I have opened port 8080.
Thanks for your help!

thx bro !

Try binding to <IP_ADDRESS>

@BunnyHolder Can you kindly explain what do you mean by binding to <IP_ADDRESS> ?
Kind regards,

Try binding to <IP_ADDRESS>

I love u bro
u saved me :)))))))

Hello!
I got the same error : I fix it by allowing the COOKIES in my web browser(Chrome for me).
Let me know if it work for you

Try binding to <IP_ADDRESS>

YEAH!!!!! It worked!! :)

@dataf3l @rasool707  What does meen by binding with <IP_ADDRESS>

还是不行
springcloud
WebSocket connection to 'ws://<IP_ADDRESS>:9666/ws' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED

@bilalswiftsolutions https://github.com/thoughtbot/til/blob/master/docker/binding-to-0-0-0-0-in-rails.md

还是不行
springcloud
WebSocket connection to 'ws://<IP_ADDRESS>:9666/ws' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED

我也一样

还是不行
springcloud
WebSocket connection to 'ws://<IP_ADDRESS>:9666/ws' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED

我也一样

删除掉重新来，我就解决了


GPU Expressions

Extends expression dependencies.  Build GPU-compatible representations for suitable zoom-based property expressions,  make them available to shaders via UBOs, and evaluate them in those shaders.
Currently only applies to non-custom line layers.

@louwers I have pre-commit set up to run on commits, and running with --all-files changes a bunch of files that I haven't interacted with, e.g., bin/offline.cpp.   It would be better to to apply all those changes to main first, rather than including them here, I think.

@TimSylvester Sure. I'll make a PR.

Updated benchmark, pretty much the same.
CPU:
Average frame encoding time: 2.78 ms
Average frame rendering time: 6.55 ms

Average frame encoding time: 2.77 ms
Average frame rendering time: 6.47 ms

Average frame encoding time: 2.79 ms
Average frame rendering time: 6.57 ms

GPU:
Average frame encoding time: 2.76 ms
Average frame rendering time: 6.70 ms

Average frame encoding time: 2.79 ms
Average frame rendering time: 6.77 ms

Average frame encoding time: 2.76 ms
Average frame rendering time: 6.76 ms

So, a small hit in render time with no real benefit to encoding time.  The CPU work we're avoiding is not very costly, and we can't avoid updating and binding the evaluated property UBO unless every single property could be done on the GPU.


[Tab] fixed textColorPrimary style

@midan888 It's not a mistake, don't worry. I plan on refactoring the way style modifiers are written in the very near future. Hopefully, it will make more sense. I agree that the name is confusing.

@oliviertassinari Ah, ok, but the thing is in my project when you set value to Tab prop textColor="primary" it takes secondary value not the primary one. Can you check that?

When you set value to Tab prop textColor="primary" it takes secondary value not the primary one. Can you check that?

@midan888 It's expected. The textColor property is about changing the color of the selected state.


e2e: TestLeaderElection: delete the lease and increase the retry period

In some random cases none of the deschedulers acquires a lease. Making the test fail due to already existing lease.

/lgtm
/approve


New extension: Upload

Upload file to the network like catbox, file.io etc..
examples
(Catbox.moe)

(file.io)

and whats this supposed to do

upload files

http cant do that

examples
(Catbox.moe) ### (file.io)

You can combine your extension with other useful features (for example, fetching stream, reading status code, requesting with callback, etc.) so we needn't create another pull requests for similar features.

http cant do that

Okay well now there's a use since you added some more stuff. Pretty cool

i also made a similar extension abt 1y ago, i'd be happy for this one to replace nine
https://github.com/TurboWarp/extensions/pull/685

progress: [status code] added

progress: [Raw respond] [api call] added

You can combine your extension with other useful features (for example, fetching stream, reading status code, requesting with callback, etc.) so we needn't create another extension for similar features.
Update:

[ ]  fetching stream
[x] reading status code
[x] requesting with callback

Any reviews?

add new apis and types(IDK why i write this 6 hours...😒😒)

[x] base64
[x] MIME type
[x] Pastebin
[x] 0x0.st
[ ] (Deleted cus too bad) ddownload.com


how IF data is generated using RINEX file

hello HI. I am a research student , I have downloaded the brdc file and it is so much long .
but the IF data generated is small .
can you please explain how the IF data is generated using the RINEX nav. file?

The duration of the IF data is specified by the "-d" option, not the length of the BRDC file.

can you please explain how IF data is generated using nav. file

What does the White selected portion means
please give a explation.
thankyou.

PRN, azimuth, elevation, range, and ionospheric delay of the visible satellites.

how the user location is attached to the generated prns


Simplify DependencyResolver

Summary

Function compact looks like simple arr.filter(Boolean) and used only once.
Moved check paths.size to start of function

Test plan

Codecov Report

Merging #6044 into master will decrease coverage by 0.02%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #6044      +/-   ##
==========================================
- Coverage   64.16%   64.14%   -0.03%
==========================================
  Files         217      217
  Lines        8328     8322       -6
  Branches        4        4
==========================================
- Hits         5344     5338       -6
  Misses       2983     2983
  Partials        1        1

Impacted Files
Coverage Δ

packages/jest-resolve-dependencies/src/index.js
97.61% <100%> (-0.3%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0a528cc...a110f25. Read the comment docs.


feat: enable tss auth on spl whitelisting

Currently, whitelisting has only authority check, or equivalent to admin role in evm contracts. Zetaclient is signing with tss, so whitelist should be allowed for both authority and tss, this PR modifies that.
Testing currently with this draft PR where tss signature is used for whitelisting: https://github.com/zeta-chain/node/pull/2984, and added unit tests here to verify both tss and authority work.

Codecov Report
Attention: Patch coverage is 0% with 71 lines in your changes missing coverage. Please review.

Project coverage is 6.71%. Comparing base (6367073) to head (bf7fb1a).

Files with missing lines
Patch %
Lines

programs/protocol-contracts-solana/src/lib.rs
0.00%
71 Missing :warning:

Additional details and impacted files
@@           Coverage Diff            @@
##            main     #60      +/-   ##
========================================
- Coverage   8.17%   6.71%   -1.46%
========================================
  Files          1       1
  Lines        318     387      +69
========================================
  Hits          26      26
- Misses       292     361      +69

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Spans are continued in Handler Interceptors

without this change an explicit new span is created on the server side. Its name is equal to the method name of the controller. It introduces some nice advantages in terms of readability of trace.
with this change we're continuing a previous span on the server side. We're attaching the tags and logs to that span with information about controller class and controller name. Also events related to start and finish of the controller are there.
fixes #471 #469 #427

Here is an example of the new trace. BTW @adriancole the controller start and finish - they were to be events not tags right?

LGTM

@marcingrzejszczak why does a controller start and finish twice?

AFAIR it's going to error processing to another controller (cause a 404 is thrown or sth).

AFAIR it's going to error processing to another controller (cause a 404 is
thrown or sth).

so this means the controller tag is the "primary controller" then?

Yeah exactly. It's the first controller that started processing the request.

well since it can hop controllers, is it possible to rename the annotation
to the controller name start/stop? that might make it more clear what's
going on.

ps I think there's a lot of detail we are trying to cram into a single
span.. personally, if there are multiple controllers being called, I'd
make them local spans (and not copy any irrelevant tags to them such
as http tags)

Current coverage is 74.60% (diff: 86.84%)

Merging #474 into 1.0.x will increase coverage by 0.32%

@@              1.0.x       #474   diff @@
==========================================
  Files           130        130
  Lines          2963       2981    +18
  Methods           0          0
  Messages          0          0
  Branches        415        415
==========================================
+ Hits           2201       2224    +23
- Misses          577        579     +2
+ Partials        185        178     -7

Powered by Codecov. Last update 0f4427b...41bafdf


[material-ui] Blog template has broken image links

Steps to reproduce
Link to live example: (required)
Steps:

Go to https://mui.com/material-ui/getting-started/templates/blog/
Image is not displayed

Current behavior
Images are missing in the blog template
Expected behavior
No response
Context
No response
Your environment

npx @mui/envinfo
  Don't forget to mention which browser you used.
  Output from `npx @mui/envinfo` goes here.

Search keywords: material-ui

Closing this one, since it's a duplicate of https://github.com/mui/material-ui/issues/42955, and we already have PRs linked there


FIPS integration

Make it easy to build crypto provider that only uses FIPS approved algorithms, and make https://docs.rs/rustls/latest/rustls/crypto/struct.CryptoProvider.html#method.fips report FIPS status.

Codecov Report
Attention: Patch coverage is 61.90476% with 24 lines in your changes missing coverage. Please review.

Project coverage is 89.52%. Comparing base (de8d555) to head (54f09ca).

Files with missing lines
Patch %
Lines

src/hkdf.rs
0.00%
3 Missing :warning:

src/hmac.rs
0.00%
3 Missing :warning:

src/lib.rs
0.00%
3 Missing :warning:

src/prf.rs
0.00%
3 Missing :warning:

src/quic.rs
0.00%
3 Missing :warning:

src/signer/mod.rs
0.00%
3 Missing :warning:

src/tls12.rs
0.00%
3 Missing :warning:

src/tls13.rs
0.00%
3 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main       #6      +/-   ##
==========================================
- Coverage   90.64%   89.52%   -1.12%
==========================================
  Files          16       17       +1
  Lines        1742     1805      +63
==========================================
+ Hits         1579     1616      +37
- Misses        163      189      +26

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


New package: textable v0.1.0

Registering package: textable
Repository: https://github.com/MUsmanZahid/textable.jl
Created by: @MUsmanZahid
Version: v0.1.0
Commit: 325a5a84c3315c67082457438bcedf17c09fde11
Reviewed by: @MUsmanZahid
Reference: https://github.com/MUsmanZahid/textable.jl/issues/1

Sorry I'm kinda new to creating packages. The build step is failing because it cannot find a 'Package.toml' file however I have checked other people's approved packages and they do not seem to have the file either. Am I missing something?

This is a bug in automerge that was fixed in this PR: https://github.com/JuliaRegistries/RegistryCI.jl/pull/115
That being said, the convention in the Julia community is to start package names with an uppercase letter.
[noblock]

Thanks! I guess I'll change the name and submit a new PR :)

GIven that we have LaTeXStrings.jl, maybe TeXTables would be a good name?

That sounds good! I was also considering XTable since the package aims to implement similar functionality as the 'xtable' R package but I don't know if that would cause problems with the original xtable authors.

I doubt it, but you can always ask them if you can get in contact with them.


[BUG]xgb train exception in py 3.9.7

Describe the bug
raise exception when train model use xgb
my code like this
(ray) [ray@ml-test ~]$ cat test_mars_xgb.py
import ray
ray.init(address="ray://<IP_ADDRESS>:10001")

import mars
import mars.tensor as mt
import mars.dataframe as md
session = mars.new_ray_session(worker_num=2, worker_mem=2 * 1024 ** 3)

from sklearn.datasets import load_boston
boston = load_boston()

data = md.DataFrame(boston.data, columns=boston.feature_names)

print("data.head().execute()")
print(data.head().execute())

print("data.describe().execute()")
print(data.describe().execute())

from mars.learn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(data, boston.target, train_size=0.7, random_state=0)

print("after split X_train: %s" % X_train)

from mars.learn.contrib import xgboost as xgb

train_dmatrix = xgb.MarsDMatrix(data=X_train, label=y_train)
test_dmatrix = xgb.MarsDMatrix(data=X_test, label=y_test)

print("train_dmatrix: %s" % train_dmatrix)

#params = {'objective': 'reg:squarederror','colsample_bytree': 0.3,'learning_rate': 0.1, 'max_depth': 5, 'alpha': 10, 'n_estimators': 10}

#booster = xgb.train(dtrain=train_dmatrix, params=params)

#xg_reg = xgb.XGBRegressor(objective='reg:squarederror', colsample_bytree=0.3, learning_rate=0.1, max_depth=5, alpha=10, n_estimators=10)
xg_reg = xgb.XGBRegressor()

print("xg_reg.fit %s" % xg_reg)

model = xg_reg.fit(X_train, y_train, session=session)

#xgb.predict(booster, X_test)

print("results.predict")

test_r = model.predict(X_test)

print("output:test_r:%s" % type(test_r))
print(test_r)

To Reproduce
To help us reproducing this bug, please provide information below:

Your Python version：3.9.7
The version of Mars you use：0.9.0rc1
Versions of crucial packages, such as numpy, scipy and pandas

Ray：1.11.0
Numpy：1.22.3
Pandas：1.4.1
Scipy：1.8.0

Full stack of the error.

(ray) [ray@ml-test ~]$ python test_mars_xgb.py
2022-03-24 10:59:42,970 INFO ray.py:432 -- Start cluster with config {'services': ['cluster', 'session', 'storage', 'meta', 'lifecycle', 'scheduling', 'subtask', 'task', 'mutable'], 'cluster': {'backend': 'ray', 'node_timeout': 120, 'node_check_interval': 1, 'ray': {'supervisor': {'standalone': False, 'sub_pool_num': 0}}}, 'session': {'custom_log_dir': None}, 'storage': {'default_config': {'transfer_block_size': '5 * 1024 ** 2'}, 'plasma': {'store_memory': '20%'}, 'backends': ['ray']}, 'meta': {'store': 'dict'}, 'task': {'default_config': {'optimize_tileable_graph': True, 'optimize_chunk_graph': True, 'fuse_enabled': True, 'initial_same_color_num': None, 'as_broadcaster_successor_num': None}}, 'scheduling': {'autoscale': {'enabled': False, 'min_workers': 1, 'max_workers': 100, 'scheduler_backlog_timeout': 20, 'worker_idle_timeout': 40}, 'speculation': {'enabled': False, 'dry': False, 'interval': 5, 'threshold': '75%', 'min_task_runtime': 3, 'multiplier': 1.5, 'max_concurrent_run': 3}, 'subtask_cancel_timeout': 5, 'subtask_max_retries': 3, 'subtask_max_reschedules': 2}, 'metrics': {'backend': 'ray', 'port': 0}}
2022-03-24 10:59:42,970 INFO api.py:53 -- Finished initialize the metrics with backend ray
2022-03-24 10:59:42,970 INFO driver.py:34 -- Setup cluster with {'ray://ray-cluster-1648090782/0': {'CPU': 2}, 'ray://ray-cluster-1648090782/1': {'CPU': 2}}
2022-03-24 10:59:42,970 INFO driver.py:40 -- Creating placement group ray-cluster-1648090782 with bundles [{'CPU': 2}, {'CPU': 2}].
2022-03-24 10:59:43,852 INFO driver.py:55 -- Create placement group success.
2022-03-24 10:59:45,128 INFO backend.py:82 -- Submit create actor pool ClientActorHandle(44dff4e8c2ea47cdd02bb84609000000) took 1.2752630710601807 seconds.
2022-03-24 10:59:46,268 INFO backend.py:82 -- Submit create actor pool ClientActorHandle(9ee3d50e43948f0f784697b809000000) took 1.116509199142456 seconds.
2022-03-24 10:59:48,475 INFO backend.py:82 -- Submit create actor pool ClientActorHandle(01f40453e2be6ed5ff7204d409000000) took 2.1755218505859375 seconds.
2022-03-24 10:59:48,501 INFO backend.py:89 -- Start actor pool ClientActorHandle(44dff4e8c2ea47cdd02bb84609000000) took 3.352660894393921 seconds.
2022-03-24 10:59:48,501 INFO backend.py:89 -- Start actor pool ClientActorHandle(9ee3d50e43948f0f784697b809000000) took 2.2049944400787354 seconds.
2022-03-24 10:59:48,501 INFO ray.py:526 -- Create supervisor on node ray://ray-cluster-1648090782/0/0 succeeds.
2022-03-24 10:59:50,148 INFO ray.py:536 -- Start services on supervisor ray://ray-cluster-1648090782/0/0 succeeds.
2022-03-24 10:59:50,494 INFO backend.py:89 -- Start actor pool ClientActorHandle(01f40453e2be6ed5ff7204d409000000) took 1.9973196983337402 seconds.
2022-03-24 10:59:50,494 INFO ray.py:541 -- Create 2 workers succeeds.
2022-03-24 10:59:50,722 INFO ray.py:545 -- Start services on 2 workers succeeds.
(RaySubPool pid=15700, ip=<IP_ADDRESS>) 2022-03-24 10:59:50,720        ERROR serialization.py:311 -- __init__() missing 1 required positional argument: 'pid'
(RaySubPool pid=15700, ip=<IP_ADDRESS>) Traceback (most recent call last):
(RaySubPool pid=15700, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/serialization.py", line 309, in deserialize_objects
(RaySubPool pid=15700, ip=<IP_ADDRESS>)     obj = self._deserialize_object(data, metadata, object_ref)
(RaySubPool pid=15700, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/ray/communication.py", line 90, in _deserialize_object
(RaySubPool pid=15700, ip=<IP_ADDRESS>)     value = _ray_deserialize_object(self, data, metadata, object_ref)
(RaySubPool pid=15700, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/serialization.py", line 215, in _deserialize_object
(RaySubPool pid=15700, ip=<IP_ADDRESS>)     return self._deserialize_msgpack_data(data, metadata_fields)
(RaySubPool pid=15700, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/serialization.py", line 174, in _deserialize_msgpack_data
(RaySubPool pid=15700, ip=<IP_ADDRESS>)     python_objects = self._deserialize_pickle5_data(pickle5_data)
(RaySubPool pid=15700, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/serialization.py", line 164, in _deserialize_pickle5_data
(RaySubPool pid=15700, ip=<IP_ADDRESS>)     obj = pickle.loads(in_band)
(RaySubPool pid=15700, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/lib/tblib/pickling_support.py", line 29, in unpickle_exception
(RaySubPool pid=15700, ip=<IP_ADDRESS>)     inst = func(*args)
(RaySubPool pid=15700, ip=<IP_ADDRESS>) TypeError: __init__() missing 1 required positional argument: 'pid'
2022-03-24 10:59:50,770 WARNING ray.py:556 -- Web service started at http://<IP_ADDRESS>:50749
(RaySubPool pid=3583) 2022-03-24 10:59:50,725   ERROR serialization.py:311 -- __init__() missing 1 required positional argument: 'pid'
(RaySubPool pid=3583) Traceback (most recent call last):
(RaySubPool pid=3583)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/serialization.py", line 309, in deserialize_objects
(RaySubPool pid=3583)     obj = self._deserialize_object(data, metadata, object_ref)
(RaySubPool pid=3583)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/ray/communication.py", line 90, in _deserialize_object
(RaySubPool pid=3583)     value = _ray_deserialize_object(self, data, metadata, object_ref)
(RaySubPool pid=3583)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/serialization.py", line 215, in _deserialize_object
(RaySubPool pid=3583)     return self._deserialize_msgpack_data(data, metadata_fields)
(RaySubPool pid=3583)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/serialization.py", line 174, in _deserialize_msgpack_data
(RaySubPool pid=3583)     python_objects = self._deserialize_pickle5_data(pickle5_data)
(RaySubPool pid=3583)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/serialization.py", line 164, in _deserialize_pickle5_data
(RaySubPool pid=3583)     obj = pickle.loads(in_band)
(RaySubPool pid=3583)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/lib/tblib/pickling_support.py", line 29, in unpickle_exception
(RaySubPool pid=3583)     inst = func(*args)
(RaySubPool pid=3583) TypeError: __init__() missing 1 required positional argument: 'pid'
/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function load_boston is deprecated; `load_boston` is deprecated in 1.0 and will be removed in 1.2.

    The Boston housing prices dataset has an ethical problem. You can refer to
    the documentation of this function for further details.

    The scikit-learn maintainers therefore strongly discourage the use of this
    dataset unless the purpose of the code is to study and educate about
    ethical issues in data science and machine learning.

    In this special case, you can fetch the dataset from the original
    source::

        import pandas as pd
        import numpy as np

        data_url = "http://lib.stat.cmu.edu/datasets/boston"
        raw_df = pd.read_csv(data_url, sep="\s+", skiprows=22, header=None)
        data = np.hstack([raw_df.values[::2, :], raw_df.values[1::2, :2]])
        target = raw_df.values[1::2, 2]

    Alternative datasets include the California housing dataset (i.e.
    :func:`~sklearn.datasets.fetch_california_housing`) and the Ames housing
    dataset. You can load the datasets as follows::

        from sklearn.datasets import fetch_california_housing
        housing = fetch_california_housing()

    for the California housing dataset and::

        from sklearn.datasets import fetch_openml
        housing = fetch_openml(name="house_prices", as_frame=True)

    for the Ames housing dataset.

  warnings.warn(msg, category=FutureWarning)
data.head().execute()
2022-03-24 10:59:51,023 INFO session.py:979 -- Time consuming to generate a tileable graph is 0.0007078647613525391s with address ray://ray-cluster-1648090782/0/0, session id zLE6ibnXqYxfFNUiCEndgZaF
      CRIM    ZN  INDUS  CHAS    NOX     RM   AGE     DIS  RAD    TAX  PTRATIO       B  LSTAT
0  0.00632  18.0   2.31   0.0  0.538  6.575  65.2  4.0900  1.0  296.0     15.3  396.90   4.98
1  0.02731   0.0   7.07   0.0  0.469  6.421  78.9  4.9671  2.0  242.0     17.8  396.90   9.14
2  0.02729   0.0   7.07   0.0  0.469  7.185  61.1  4.9671  2.0  242.0     17.8  392.83   4.03
3  0.03237   0.0   2.18   0.0  0.458  6.998  45.8  6.0622  3.0  222.0     18.7  394.63   2.94
4  0.06905   0.0   2.18   0.0  0.458  7.147  54.2  6.0622  3.0  222.0     18.7  396.90   5.33
data.describe().execute()
2022-03-24 10:59:51,504 INFO session.py:979 -- Time consuming to generate a tileable graph is 0.0005688667297363281s with address ray://ray-cluster-1648090782/0/0, session id zLE6ibnXqYxfFNUiCEndgZaF
             CRIM          ZN       INDUS        CHAS         NOX          RM         AGE         DIS         RAD         TAX     PTRATIO           B       LSTAT
count  506.000000  506.000000  506.000000  506.000000  506.000000  506.000000  506.000000  506.000000  506.000000  506.000000  506.000000  506.000000  506.000000
mean     3.613524   11.363636   11.136779    0.069170    0.554695    6.284634   68.574901    3.795043    9.549407  408.237154   18.455534  356.674032   12.653063
std      8.601545   23.322453    6.860353    0.253994    0.115878    0.702617   28.148861    2.105710    8.707259  168.537116    2.164946   91.294864    7.141062
min      0.006320    0.000000    0.460000    0.000000    0.385000    3.561000    2.900000    1.129600    1.000000  187.000000   12.600000    0.320000    1.730000
25%      0.082045    0.000000    5.190000    0.000000    0.449000    5.885500   45.025000    2.100175    4.000000  279.000000   17.400000  375.377500    6.950000
50%      0.256510    0.000000    9.690000    0.000000    0.538000    6.208500   77.500000    3.207450    5.000000  330.000000   19.050000  391.440000   11.360000
75%      3.677083   12.500000   18.100000    0.000000    0.624000    6.623500   94.075000    5.188425   24.000000  666.000000   20.200000  396.225000   16.955000
max     88.976200  100.000000   27.740000    1.000000    0.871000    8.780000  100.000000   12.126500   24.000000  711.000000   22.000000  396.900000   37.970000
2022-03-24 10:59:51,992 INFO session.py:979 -- Time consuming to generate a tileable graph is 0.0019736289978027344s with address ray://ray-cluster-1648090782/0/0, session id zLE6ibnXqYxfFNUiCEndgZaF
after split X_train:          CRIM    ZN  INDUS  CHAS    NOX     RM   AGE     DIS   RAD    TAX  PTRATIO       B  LSTAT
191   0.06911  45.0   3.44   0.0  0.437  6.739  30.8  6.4798   5.0  398.0     15.2  389.71   4.69
380  88.97620   0.0  18.10   0.0  0.671  6.968  91.9  1.4165  24.0  666.0     20.2  396.90  17.21
337   0.03041   0.0   5.19   0.0  0.515  5.895  59.6  5.6150   5.0  224.0     20.2  394.81  10.56
266   0.78570  20.0   3.97   0.0  0.647  7.014  84.6  2.1329   5.0  264.0     13.0  384.07  14.79
221   0.40771   0.0   6.20   1.0  0.507  6.164  91.3  3.0480   8.0  307.0     17.4  395.24  21.46
..        ...   ...    ...   ...    ...    ...   ...     ...   ...    ...      ...     ...    ...
275   0.09604  40.0   6.41   0.0  0.447  6.854  42.8  4.2673   4.0  254.0     17.6  396.90   2.98
217   0.07013   0.0  13.89   0.0  0.550  6.642  85.1  3.4211   5.0  276.0     16.4  392.78   9.69
369   5.66998   0.0  18.10   1.0  0.631  6.683  96.8  1.3567  24.0  666.0     20.2  375.33   3.73
95    0.12204   0.0   2.89   0.0  0.445  6.625  57.8  3.4952   2.0  276.0     18.0  357.98   6.65
277   0.06127  40.0   6.41   1.0  0.447  6.826  27.6  4.8628   4.0  254.0     17.6  393.45   4.16

[354 rows x 13 columns]
train_dmatrix: DataFrame(op=ToDMatrix)
/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/xgboost/compat.py:36: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
  from pandas import MultiIndex, Int64Index
xg_reg.fit XGBRegressor()
2022-03-24 10:59:53,085 INFO session.py:979 -- Time consuming to generate a tileable graph is 0.0010030269622802734s with address ray://ray-cluster-1648090782/0/0, session id zLE6ibnXqYxfFNUiCEndgZaF
(RaySubPool pid=15805, ip=<IP_ADDRESS>) Exception in thread Thread-42:
(RaySubPool pid=15805, ip=<IP_ADDRESS>) Traceback (most recent call last):
(RaySubPool pid=15805, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/threading.py", line 973, in _bootstrap_inner
(RaySubPool pid=15805, ip=<IP_ADDRESS>)     self.run()
(RaySubPool pid=15805, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/threading.py", line 910, in run
(RaySubPool pid=15805, ip=<IP_ADDRESS>)     self._target(*self._args, **self._kwargs)
(RaySubPool pid=15805, ip=<IP_ADDRESS>)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/learn/contrib/xgboost/tracker.py", line 355, in join
(RaySubPool pid=15805, ip=<IP_ADDRESS>)     while self.thread.isAlive():
(RaySubPool pid=15805, ip=<IP_ADDRESS>) AttributeError: 'Thread' object has no attribute 'isAlive'
(RaySubPool pid=3583) [10:59:53] task NULL got new rank 0
2022-03-24 10:59:54,331 ERROR session.py:1822 -- Task exception was never retrieved
future: <Task finished name='Task-110' coro=<_wrap_awaitable() done, defined at /home/ray/anaconda3/envs/ray/lib/python3.9/asyncio/tasks.py:684> exception=TypeError("ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''")>
Traceback (most recent call last):
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/asyncio/tasks.py", line 691, in _wrap_awaitable
    return (yield from awaitable.__await__())
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/deploy/oscar/session.py", line 106, in wait
    return await self._aio_task
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/deploy/oscar/session.py", line 950, in _run_in_background
    fetch_tileables = await self._task_api.get_fetch_tileables(task_id)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/services/task/api/oscar.py", line 100, in get_fetch_tileables
    return await self._task_manager_ref.get_task_result_tileables(task_id)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/context.py", line 188, in send
    result = await self._wait(future, actor_ref.address, message)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/context.py", line 83, in _wait
    return await future
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/context.py", line 74, in _wait
    await asyncio.shield(future)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/core.py", line 50, in _listen
    message: _MessageBase = await client.recv()
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/communication/base.py", line 262, in recv
    return await self.channel.recv()
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/ray/communication.py", line 209, in recv
    result = await object_ref
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/util/client/server/server.py", line 375, in send_get_response
    serialized = dumps_from_server(result, client_id, self)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/util/client/server/server_pickler.py", line 114, in dumps_from_server
    sp.dump(obj)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 620, in dump
    return Pickler.dump(self, obj)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/ray/communication.py", line 55, in __reduce__
    return _argwrapper_unpickler, (serialize(self.message),)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/serialization/core.py", line 361, in serialize
    gen_to_serial = gen.send(last_serial)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/core/base.py", line 140, in serialize
    return (yield from super().serialize(obj, context))
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/serialization/serializables/core.py", line 108, in serialize
    tag_to_values = self._get_tag_to_values(obj)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/serialization/serializables/core.py", line 101, in _get_tag_to_values
    value = field.on_serialize(value)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/utils.py", line 157, in on_serialize_nsplits
    new_nsplits.append(tuple(None if np.isnan(v) else v for v in dim_splits))
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/utils.py", line 157, in <genexpr>
    new_nsplits.append(tuple(None if np.isnan(v) else v for v in dim_splits))
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
Traceback (most recent call last):
  File "/home/ray/test_mars_xgb.py", line 42, in <module>
    model = xg_reg.fit(X_train, y_train, session=session)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/learn/contrib/xgboost/regressor.py", line 61, in fit
    result = train(
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/learn/contrib/xgboost/train.py", line 249, in train
    ret = t.execute(session=session, **run_kwargs).fetch(session=session)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/core/entity/executable.py", line 98, in execute
    return execute(self, session=session, **kw)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/deploy/oscar/session.py", line 1851, in execute
    return session.execute(
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/deploy/oscar/session.py", line 1647, in execute
    execution_info: ExecutionInfo = fut.result(
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/concurrent/futures/_base.py", line 445, in result
    return self.__get_result()
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/concurrent/futures/_base.py", line 390, in __get_result
    raise self._exception
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/deploy/oscar/session.py", line 1831, in _execute
    await execution_info
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/asyncio/tasks.py", line 691, in _wrap_awaitable
    return (yield from awaitable.__await__())
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/deploy/oscar/session.py", line 106, in wait
    return await self._aio_task
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/deploy/oscar/session.py", line 950, in _run_in_background
    fetch_tileables = await self._task_api.get_fetch_tileables(task_id)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/services/task/api/oscar.py", line 100, in get_fetch_tileables
    return await self._task_manager_ref.get_task_result_tileables(task_id)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/context.py", line 188, in send
    result = await self._wait(future, actor_ref.address, message)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/context.py", line 83, in _wait
    return await future
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/context.py", line 74, in _wait
    await asyncio.shield(future)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/core.py", line 50, in _listen
    message: _MessageBase = await client.recv()
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/communication/base.py", line 262, in recv
    return await self.channel.recv()
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/ray/communication.py", line 209, in recv
    result = await object_ref
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/util/client/server/server.py", line 375, in send_get_response
    serialized = dumps_from_server(result, client_id, self)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/util/client/server/server_pickler.py", line 114, in dumps_from_server
    sp.dump(obj)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 620, in dump
    return Pickler.dump(self, obj)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/ray/communication.py", line 55, in __reduce__
    return _argwrapper_unpickler, (serialize(self.message),)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/serialization/core.py", line 361, in serialize
    gen_to_serial = gen.send(last_serial)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/core/base.py", line 140, in serialize
    return (yield from super().serialize(obj, context))
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/serialization/serializables/core.py", line 108, in serialize
    tag_to_values = self._get_tag_to_values(obj)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/serialization/serializables/core.py", line 101, in _get_tag_to_values
    value = field.on_serialize(value)
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/utils.py", line 157, in on_serialize_nsplits
    new_nsplits.append(tuple(None if np.isnan(v) else v for v in dim_splits))
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/utils.py", line 157, in <genexpr>
    new_nsplits.append(tuple(None if np.isnan(v) else v for v in dim_splits))
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
(RaySubPool pid=3400) Main pool Actor(RayMainPool, 9ee3d50e43948f0f784697b809000000) has exited, exit current sub pool now.
(RaySubPool pid=3400) Traceback (most recent call last):
(RaySubPool pid=3400)   File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/oscar/backends/ray/pool.py", line 365, in check_main_pool_alive
(RaySubPool pid=3400)     main_pool_start_timestamp = await main_pool.alive.remote()
(RaySubPool pid=3400) ray.exceptions.RayActorError: The actor died unexpectedly before finishing this task.
(RaySubPool pid=3400)   class_name: RayMainPool
(RaySubPool pid=3400)   actor_id: 9ee3d50e43948f0f784697b809000000
(RaySubPool pid=3400)   pid: 3514
(RaySubPool pid=3400)   name: ray://ray-cluster-1648090782/0/1
(RaySubPool pid=3400)   namespace: b7b70429-e17c-486f-9172-0872403ed6ef
(RaySubPool pid=3400)   ip: <IP_ADDRESS>
(RaySubPool pid=3400) The actor is dead because because all references to the actor were removed.
A worker died or was killed while executing a task by an unexpected system error. To troubleshoot the problem, check the logs for the dead worker. RayTask ID: ffffffffffffffff6f1ccaae6135c700f75befbe09000000 Worker ID: 707d6a3f910fa005ec33fe7ae60ddef5cfc1b9eb67510f1bc0f19623 Node ID: 7c54d788f2585a26ce8ef92e01f7e774359a4f0636b4bcfcb84272f7 Worker IP address: <IP_ADDRESS> Worker port: 10043 Worker PID: 15700
Exception ignored in: <function _TileableSession.__init__.<locals>.cb at 0x7efbd9a75160>
Traceback (most recent call last):
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/core/entity/executable.py", line 52, in cb
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/concurrent/futures/thread.py", line 156, in submit
AttributeError: __enter__
Exception ignored in: <function _TileableSession.__init__.<locals>.cb at 0x7efbd9a75dc0>
Traceback (most recent call last):
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/site-packages/mars/core/entity/executable.py", line 52, in cb
  File "/home/ray/anaconda3/envs/ray/lib/python3.9/concurrent/futures/thread.py", line 156, in submit
AttributeError: __enter__

Minimized code to reproduce the error.

Expected behavior
A clear and concise description of what you expected to happen.
Additional context
Add any other context about the problem here.

Error is raised by mars.learn.contrib.xgboost.tracker.py which is copied from xgboost, we will update this file ASAP.


Teams adaptive cards automatically obscured in channels on the New Channels Experience/Teams 2.1 client

Teams 2.1 or the Teams 1.0 New Channel Experience requires the user to click on the "see more" button as most of an adaptive card is obscured.
This was not the case in

The Teams 1.0 client classic channels experience
Teams chat (1.0 or 2.1)

Teams 2.1 or the Teams 1.0 new channels experience, requiring the user to click on "see more" to view an entire adaptive card:

Teams 1.0 classic channel experience, renders nicely without obscuring any information.

Card JSON
{
    "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
    "type": "AdaptiveCard",
    "version": "1.4",
    "msteams": {
        "width": "full"
    },
    "body": [
        {
            "type": "Container",
            "items": [
                {
                    "type": "ColumnSet",
                    "columns": [
                        {
                            "type": "Column",
                            "width": "auto",
                            "items": [
                                {
                                    "type": "Image",
                                    "altText": "Alert",
                                    "url": "https://normanhurb-h.schools.nsw.gov.au/content/dam/doe/sws/schools/n/normanhurb-h/icons/exam.png",
                                    "width": "75px",
                                    "selectAction": {
                                        "type": "Action.OpenUrl",
                                        "url": "https://teams.microsoft.com/l/stage/2a527703-1f6f-4559-a332-d8a7d288cd88/0?context=%7B%22contentUrl%22%3A%22https%3A%2F%2Fschoolsnsw.sharepoint.com%2F%3Ab%3A%2Fs%2FN1XHSGBP%2Fmathematics%2FEaa5SGAOfe9ImIY94m_NJQ4BYrXhidPy87eYHN7cQl8egQ%22%2C%22websiteUrl%22%3A%22https%3A%2F%2Fschoolsnsw.sharepoint.com%2F%3Ab%3A%2Fs%2FN1XHSGBP%2Fmathematics%2FEaa5SGAOfe9ImIY94m_NJQ4BYrXhidPy87eYHN7cQl8egQ%22%2C%22name%22%3A%22Notification%20for%20Year%2009%202023%20Assessment%20Task%203%22%7D",
                                        "id": "10"
                                    }
                                }
                            ]
                        },
                        {
                            "type": "Column",
                            "width": "stretch",
                            "items": [
                                {
                                    "type": "TextBlock",
                                    "text": "**Assessment Task Notification**",
                                    "weight": "Bolder",
                                    "size": "Large",
                                    "wrap": true,
                                    "color": "Accent"
                                },
                                {
                                    "type": "TextBlock",
                                    "text": "2023 Year 09 Assessment Task 3",
                                    "wrap": true,
                                    "spacing": "None",
                                    "color": "Attention",
                                    "weight": "Bolder",
                                    "size": "Large"
                                }
                            ],
                            "verticalContentAlignment": "Center"
                        }
                    ]
                }
            ],
            "style": "warning",
            "bleed": true
        },
        {
            "type": "TextBlock",
            "text": "Dear <atTag><tagId>MDVhMGU2OWEtNDE4YS00N2MxLTljMjUtOTM4NzI2MWJmOTkxIyM1NzI3MjgxNi0yMjMxLTQ5ZjUtOTFiZS03NGY0NmY2Y2JhYzIjI3RRYWNHUXNwNEQ%3D</tagId><groupId>57272816-2231-49f5-91be-74f46f6cbac2</groupId></atTag>, please note that a new Assessment Task Notification is now available.",
            "wrap": true
        },
        {
            "type": "TextBlock",
            "text": "The following resources for revision will be provided no later than these dates:",
            "weight": "Bolder",
            "color": "Accent",
            "wrap": true,
            "separator": true
        },
        {
            "type": "FactSet",
            "facts": [
                {
                    "title": "Past Paper(s):",
                    "value": "Thursday, August 3, 2023"
                },
                {
                    "title": "Past Paper Solution(s):",
                    "value": "Monday, August 14, 2023"
                }
            ]
        },
        {
            "type": "Container",
            "bleed": true,
            "style": "emphasis",
            "items": [
                {
                    "type": "TextBlock",
                    "text": "Actions",
                    "wrap": true,
                    "weight": "Bolder",
                    "color": "Accent"
                },
                {
                    "type": "ActionSet",
                    "spacing": "None",
                    "actions": [
                        {
                            "type": "Action.OpenUrl",
                            "title": "Preview detailed notification for Year 09 2023 Assessment Task 3 (PDF)",
                            "url": "https://teams.microsoft.com/l/stage/2a527703-1f6f-4559-a332-d8a7d288cd88/0?context=%7B%22contentUrl%22%3A%22https%3A%2F%2Fschoolsnsw.sharepoint.com%2F%3Ab%3A%2Fs%2FN1XHSGBP%2Fmathematics%2FEaa5SGAOfe9ImIY94m_NJQ4BYrXhidPy87eYHN7cQl8egQ%22%2C%22websiteUrl%22%3A%22https%3A%2F%2Fschoolsnsw.sharepoint.com%2F%3Ab%3A%2Fs%2FN1XHSGBP%2Fmathematics%2FEaa5SGAOfe9ImIY94m_NJQ4BYrXhidPy87eYHN7cQl8egQ%22%2C%22name%22%3A%22Notification%20for%20Year%2009%202023%20Assessment%20Task%203%22%7D",
                            "id": "0",
                            "iconUrl": "https://normanhurb-h.schools.nsw.gov.au/content/dam/doe/sws/schools/n/normanhurb-h/icons/pdf.png"
                        },
                        {
                            "type": "Action.OpenUrl",
                            "title": "View Learning Programs",
                            "url": "https://schoolsnsw.sharepoint.com/:f:/s/293YP4GC/EpYRilAICVtDpEHcOejpFl0Ba6t73Hr7JzT0SI5j_rEBFA?e=SWDaQl",
                            "id": "1",
                            "iconUrl": "https://normanhurb-h.schools.nsw.gov.au/content/dam/doe/sws/schools/n/normanhurb-h/icons/sharepoint.png"
                        }
                    ]
                }
            ]
        }
    ]
}

Hello @z3019494 - Thanks for raising your query.
We will look into it and let you know the updates.

@z3019494 - We tried to repro this issue in both Teams new channel experience as well as in classic Teams and observed below behaviour.
In Teams Classic: (Rendering fully)

In Teams New channel experience:

Not to mention these cards aren't showing in full width either.
https://github.com/microsoft/AdaptiveCards/issues/8692

OMG it got fixed just then! The cards are no longer obscured!

We are closing this issue for now. Please feel free to reopen if required. Thanks!


issue with having .sapp-cache in home directory in NFS context

Just a note in case this hadn't come up as something you might address at some point.
I'm trying singularity on an Ubuntu 14.04 machine on which my home directory is NFS-mounted. I'm getting this error:
paciorek@smeagol:/tmp> ./echo.sapp
ERROR: Could not bind mount /accounts/gen/vis/paciorek/.sapp-cache/./echo/c/dev
Channeling a conversation about this with Ryan Lovett - it seems that the issue is that root can't do the bind mount because it can access a user's home directory on an NFS client.

Very interesting case I didn't think of! Of course, I would have hit this shortly as we implement it on our HPC resources, but let me brainstorm on this.
Do you know if it is possible to mount proc on top of an NFS mount?
Thanks!

my NFS v3 might be less restrictive: works for me (CentOS-6 x86_64 client)

Ryan Lovett here in Stats said:
It is possible to mount proc onto a directory on an NFS volume if the mounting user has permission on the directory. For example I made /var/nitedump/proc and mounted proc from one of our NFS client machines onto it, but couldn't do the same for ~arbitrary_user/proc.

Hi Chris,
Something that Ryan said made me curious: ".. if the mounting user has permission on the directory ..". Are you mounting/exporting your home directories with squash-root enabled? If so, this may have an issue with Singularity as it has some privilege escalation bits to implement the physical containerization, the bind mount to dev and full mount of proc.
BTW, tell Ryan I said hi. LOL
Thanks!

Bypassing the ChrisMQ... :)
Yes, we've got squash root. Would it be possible to use a local cache dir instead, e.g. /var/cache/sapp/$USER ? With /var/cache/sapp 0711 and /var/cache/sapp/$USER as whatever you need.

Hi Ryan,
Yes I can make the sapp-cache directory configurable, but it is beneficial to have it in the user's home directory such that it can be used across nodes. For example, with the latest version (as of like an hour or so ago), if you execute a SAPP file directly, it will invoke a different code pathway which will cause Singularity to use a temporary dir (in /tmp) for the container environment. It would be fairly easy to make that configurable.
I am wondering if root squash is what the root cause of the NFS problem. Do you have other mount options? What is your take?
Thanks!

root squash is definitely the cause of this problem. I don't want to disable it unless there is some other way of limiting client root access to user data. We're on NFSv3.
So some of the sapp cache data is relevant to the user on any node and some (whatever is getting bind mounted) is only relevant to the user on a specific node? If so can it be separated into home and machine specific locations (is that what your recent changes have done)? Or does each node need access to the other's data?

I just committed a solution to allow someone to custom configure their cache directory using the environment variable SINGULARITY_CACHEDIR. While this will work around the problem, as you eluded to it creates another if someone wishes to run parallel apps or run their apps on a different node within a shared home directory context.
Also, I added another bit of logic to do an additional stat() as root to see if the file system is still accessible, and if not dump a more appropriate error message.
I think the ticket can be closed now, but I'd like to see if the above two changes work for @paciorek.
On a side note, while root_squash may make it slightly more difficult for an attacker to traverse an NFS file system, it surely doesn't make it secure. Root could always create any number of users with matching UID's as that already on the file system and still get to any data they wish.

Here's what I am getting now. Some success some failure...
First, my complete install workflow:
git clone -b master --single-branch
https://github.com/gmkurtzer/singularity.git
cd singularity
export SINGULARITY_CACHEDIR=/tmp/paciorek
sh ./autogen.sh
./configure --prefix=/usr/local
make
sudo make install
sudo make install-perms
now test it:
$ cd /tmp
$ singularity build echo.sspec
$ ./echo.sapp blah
mktemp: failed to create directory via template
‘/tmp/paciorek/tmp/.sapprun-3189.XXXXXXXXX’: No such file or directory
SAPPDIR not given
$ singularity install /tmp/echo.sapp
Installing: /tmp/echo.sapp
$ singularity run echo blah
ERROR: 'echo' is not installed
$ singularity list
/usr/local/libexec/singularity/list.exec: 35: shift: can't shift that many
$ singularity install echo.sapp
Installing: echo.sapp
$ singularity run echo blah
blah
$ singularity list
/usr/local/libexec/singularity/list.exec: 35: shift: can't shift that many
On Thu, Dec 17, 2015 at 2:00 PM, Gregory M. Kurtzer <
<EMAIL_ADDRESS>wrote:

I just committed a solution to allow someone to custom configure their
cache directory using the environment variable SINGULARITY_CACHEDIR. While
this will work around the problem, as you eluded to it creates another if
someone wishes to run parallel apps or run their apps on a different node
within a shared home directory context.
Also, I added another bit of logic to do an additional stat() as root to
see if the file system is still accessible, and if not dump a more
appropriate error message.
I think the ticket can be closed now, but I'd like to see if the above two
changes work for @paciorek https://github.com/paciorek.
On a side note, while root_squash may make it slightly more difficult for
an attacker to traverse an NFS file system, it surely doesn't make it
secure. Root could always create any number of users with matching UID's as
that already on the file system and still get to any data they wish.
—
Reply to this email directly or view it on GitHub
https://github.com/gmkurtzer/singularity/issues/7#issuecomment-165592516
.

On a side note, while root_squash may make it slightly more difficult for an attacker to traverse an NFS file system, it surely doesn't make it secure. Root could always create any number of users with matching UID's as that already on the file system and still get to any data they wish.

Yes, or just su. So just slightly more difficult. root_squash is more effective in preventing sysadmins from making mistakes. A kerberized environment wouldn't allow this sort of thing, but then I think it'd get in your way too. (though I've no experience with krb+nfsv4)

@paciorek I made some changes that I think will fix this, but I haven't finished testing (because @bernardl is coercing me to goto our group holiday lunch). If you want to test,.. go for it, I think it will work now. With that said, I will test more when I return.

I cleaned a lot of stuff up, no doubt I created more bugs as I was solving them... But in any case, the latest commit will hopefully do good things.

Things look good for me now. The following all worked:
paciorek@xxxxx:/tmp> ./echo.sapp blah
blah
paciorek@xxxxx:/tmp> singularity install /tmp/echo.sapp
Installing: /tmp/echo.sapp
paciorek@xxxxx:/tmp> singularity run echo blah
blah
paciorek@xxxxx:/tmp> singularity list
CONTAINER NAME         SIZE (k) SUMMARY
echo                       2568
paciorek@xxxxx:/tmp> singularity delete echo
Deleted container cache: echo
Removing previous SAPP container cache
paciorek@xxxxx:/tmp> singularity install echo.sapp
Installing: echo.sapp
paciorek@xxxxx:/tmp> singularity run echo blah
blah
On Fri, Dec 18, 2015 at 4:06 PM, Gregory M. Kurtzer <
<EMAIL_ADDRESS>wrote:

I cleaned a lot of stuff up, no doubt I created more bugs as I was solving
them... But in any case, the latest commit will hopefully do good things.
—
Reply to this email directly or view it on GitHub
https://github.com/gmkurtzer/singularity/issues/7#issuecomment-165924149
.

Excellent, thanks Chris! I am going to close this issue, but feel free to reopen if it comes up again.
Thanks!


HDDS-3368. Ozone filesystem jar should not include webapps folder

What changes were proposed in this pull request?

Exclude the unwanted server dependencies from filesystem jar

What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3368
How was this patch tested?
Tested the filesystem jar in a cluster with Hive on Tez and verified that hive was successfully able to write to Ozone bucket using o3fs uri.

@bharatviswa504 @elek @adoroszlai Please review

@bharatviswa504 Yes, created this jira to track that effort - https://issues.apache.org/jira/browse/HDDS-3376

Thank You @vivekratnavel for the contribution.


Update yourkit-java-profiler to 2015-build-15064

Thank you for the contribution. It was merged directly as 39174c0de5faf88fe870b2078f1204ac91f78caa to keep commit history cleaner. Your contribution is still credited to you.


Re-generated yarn file to remove security vulnerabilities

Fixing dependency vulnerabilities

If the issue was related to yarn version. We should update package.json
  "engines": {
    "node": ">=8.1.0",
    "npm": ">=5.4.2",
    "yarn": ">=0.27.5"
  },

Vulnerabilities were fixed by merging in react16 branch


GEODE-8421: replace cleaning of region with destroy/create

Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[*] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[*] Has your PR been rebased against the latest commit within the target branch (typically develop)?

[*] Is your initial contribution a single, squashed commit?

[*] Does gradlew build run cleanly?

[ ] Have you written or updated unit tests to verify your changes?

[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:
Please ensure that once the PR is submitted, check Concourse for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to<EMAIL_ADDRESS>

this PR appears to be abandoned, can it be closed?