Aborting scan with 4 ^C

Assuming now that ^C^C^C makes a pause at next checkpoint, that ^C^C^C^C will abort the scan.  Got this printout - did not abort cleanly.
In [210]: RE(ct(num=1800,delay=2))
Transient Scan ID: 64607
Persistent Unique Scan ID: 'b972415d-7198-4338-84b7-b80a3225c3ed'
+-----------+------------+------------+------------+------------+------------+------------+------------+------------+------------+---------------+
|   seq_num |       time |   sclr_ch2 |   epu1_gap |   epu2_gap | mono_tempa | mono_tempb |  grt1_temp |  grt2_temp |  ring_curr | diag6_monitor |
+-----------+------------+------------+------------+------------+------------+------------+------------+------------+------------+---------------+
|         1 | 11:28:20.6 |       4624 |    57.1840 |    30.8355 |    302.754 |    302.750 |       31.4 |       28.6 |     199.75 |     177898557 |
|         2 | 11:28:24.1 |       4610 |    57.1840 |    30.8355 |    302.754 |    302.752 |       31.6 |       28.7 |     199.75 |     176485203 |
|         3 | 11:28:27.7 |       4616 |    57.1840 |    30.8355 |    302.753 |    302.751 |       31.4 |       28.7 |     199.75 |     177054705 |
|         4 | 11:28:31.2 |       4612 |    57.1840 |    30.8355 |    302.752 |    302.752 |       31.4 |       28.6 |     199.69 |     176228190 |
|         5 | 11:28:34.7 |       4619 |    57.1840 |    30.8355 |    302.751 |    302.754 |       31.4 |       28.7 |     199.69 |     177529023 |
|         6 | 11:28:38.3 |       4621 |    57.1840 |    30.8355 |    302.753 |    302.753 |       31.4 |       28.7 |     199.63 |     177957256 |
|         7 | 11:28:41.9 |       4612 |    57.1840 |    30.8355 |    302.753 |    302.752 |       31.5 |       28.6 |     199.63 |     176966899 |
|         8 | 11:28:45.5 |       4600 |    57.1840 |    30.8355 |    302.753 |    302.754 |       31.4 |       28.7 |     199.63 |     176676989 |
|         9 | 11:28:49.2 |       4591 |    57.1840 |    30.8355 |    302.753 |    302.756 |       31.4 |       28.7 |     199.57 |     177566139 |
|        10 | 11:28:52.8 |       4610 |    57.1840 |    30.8355 |    302.753 |    302.755 |       31.4 |       28.6 |     199.57 |     177829971 |
|        11 | 11:28:56.4 |       4602 |    57.1840 |    30.8355 |    302.752 |    302.750 |       31.4 |       28.7 |     199.51 |     175941839 |
|        12 | 11:29:00.0 |       4596 |    57.1840 |    30.8355 |    302.751 |    302.752 |       31.4 |       28.7 |     199.51 |     177662621 |
|        13 | 11:29:03.6 |       4604 |    57.1840 |    30.8355 |    302.751 |    302.750 |       31.4 |       28.7 |     199.45 |     175583165 |
^CA 'deferred pause' has been requested. The RunEngine will pause at the next checkpoint. To pause immediately, hit Ctrl+C again in the next 10 seconds.
Deferred pause acknowledged. Continuing to checkpoint.
^C^CAborting....
+-----------+------------+------------+------------+------------+------------+------------+------------+------------+------------+---------------+
generator ct ['b97241'] (scan num: 64607)
Out[210]: ['b972415d-7198-4338-84b7-b80a3225c3ed']

In [211]: Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/home/xf23id1/conda_envs/collection/lib/python3.4/site-packages/IPython/lib/inputhookqt4.py", line 90, in inputhook_qt4
    try:
KeyboardInterrupt

If you suspect this is an IPython bug, please report it at:
    https://github.com/ipython/ipython/issues
or send an email to the mailing list at<EMAIL_ADDRESS>
You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.

Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
    %config Application.verbose_crash=True

How reproducible is this?  It looks like the can had already aborted and returned by the time you hit the last ^C which somehow ended up getting raised inside of the inputhook method that IPython installed.

Right, that one was more ^C than you should have needed. One to pause at next checkpoint, two to pause ASAP, three to abort. See docs.

Yes, I was tired. It was not 4^C -  I miss counted.

OK, no worries. Closing this issue then.


How to set up StaticRouter when we are using ReactRotuer

Hi,
I have tried as below to setup static router , but I need to pass request url  for location, how can i setup this.

import { createReactAppExpress, handleUniversalRender } from '@cra-express/core';
import { StaticRouter } from 'react-router';
const path = require('path');
const React = require('react');

const { default: App } = require('../../src/components/app/app');
const clientBuildPath = path.resolve(__dirname, 'client');
const context = {};
const app = createReactAppExpress({
  clientBuildPath,
  universalRender: handleUniversalRender(
    <StaticRouter location={req.url} basename="/" context={context}>
      <App />
    </StaticRouter>
  )
});

module.exports = app;

https://github.com/antonybudianto/cra-universal/wiki/Integrate-React-Router


Small fixes

use bundle exec jekyll serve to use local jekyll
change gems to plugins because it's deprecated
remove events

This can get merged after #744, it should have no conflicts.

Tested ok on the following platforms:
| Edge | IE 11 | Chrome | Firefox | Opera | Vivaldi
-- | -- | -- | -- | -- | -- | --
Windows 7 | - | OK | OK | OK | OK | OK
Windows 10 | OK | OK | OK | OK | OK | OK
Ubuntu 18.04 | - | - | OK | OK | OK | OK
CentOS 7 | - | - | - | OK | OK | OK


Does Keras 3 work with tf.distribute.MultiWorkerMirroredStrategy?

This user guide shows Keras 3 usage with tf.distribute.MirroredStrategy, which is only useful for single-node multi-GPU data parallel training.
However, tf.distribute.MultiWorkerMirroredStrategy is required for multi-node data parallel training.
This example on Tensorflow's docs does not work with keras-nightly-3.7.0.dev2024120303, tf-nightly-2.19.0.dev20241203, though this problem has been around for a few releases of tensorflow (since tensorflow==2.16 when the default Keras version got bumped to Keras 3; see https://github.com/tensorflow/tensorflow/issues/72388).
Question: Does Keras 3 support tf.distribute.MultiWorkerMirroredStrategy for TF distributed training, or does it only support tf.distribute.MirroredStrategy?

Welcome to the party, pal: https://github.com/keras-team/keras/issues/20329

Hi @justinvyu,
Thanks for reporting this. You can use tf.distribute.MirroredStrategy to implement data parallelism, it seems tf.distribute.MultiWorkerMirroredStrategy is not supported with keras 3 yet as mentioned in this documentation. You can read more about data parallelism here.

Thanks @dhantule. Do you know when the tf.distribute.MultiWorkerMirroredStrategy support is slated on the roadmap? Is there somewhere that I can read about the roadmap?

Hi @justinvyu, data parallelism and distributed tuning can be combined, we can run multiple trials of training and leverage data parallelism to speed up the training process. If you have 8 workers with 2 GPUs on each worker, you can run 8 parallel trials with each trial training on 2 GPUs by using tf.distribute.MirroredStrategy.

hi @dhantule we need to be able to train a single model across multiple nodes still, even without distributed tuning.

Keras 3 does not work with tf.distribute.MultiWorkerMirroredStrategy.
Whether support with be added is a question for the Keras team at Google to answer. @jeffcarp, do you know?
Here's what I know:

PSS training was a very niche feature. My impression was that it was only used internally at Google, mostly for legacy reasons (it used to be the only realistic way to do massive multi GPU training on Borg). But I guess I was wrong? This is the first time I hear about it being used outside Google. PSS is basically never the right idea, even if you have multiple machines.
Users of PSS training at Google are all using Keras 2 + TF at this time.
There is work underway for PSS training with Keras 3 on JAX. I believe it involves a custom callback.
Not sure if that's also true for TF.

My overall take is, you should probably not be using PSS. Even at Google where some people use it, I have never seen a valid use case for it. If you problem is, "I have many machines, each with 1 or more GPUs, and I want to train a single model on all of them", then the answer is Keras 3 + JAX + the keras.distribution.ModelParallel API. It will do a much better job at utilizing your devices. And it works in multi-node settings, to be clear.


Disable TLS1.0/1.1 on BlobStore server

Describe the bug
BlobStore server listening on 25250 should disable TLS1.0/TLS 1.1
Currently there is no such configuration option for the blobstore nginx: https://github.com/cloudfoundry/bosh/blob/master/jobs/blobstore/templates/blobstore.conf.erb.
Which means it uses the nginx default "ssl_protocols TLSv1 TLSv1.1 TLSv1.2"  according to: http://nginx.org/en/docs/http/configuring_https_servers.html#:~:text=The directives ssl_protocols and ssl_ciphers,1 TLSv1.
Currently only the director api specifies ssl_protocols in its nginx config. Which defaults to TLSv1.2: https://github.com/cloudfoundry/bosh/blob/master/jobs/director/spec#L94.
Additional context
Disable TLS 1.0/1.1 is a security requirement from many customers. Considering this is a simple fix, I would recommend to prioritize it as a critical issue to address.

Hi ,
any chance to fix this issue?


crypto.livepeer.tv Channel for ETH startups

Hello Folks! simple idea.
we crew from Livepeer.tv in Berlin are receiving a lot of requests from ETH Community Startups
to stream their Meetups etc. Some also want us to fly to other destinations to do it.
So I believe this is a demand that LP could help solving.
LP.tv Berlin has a second channel called Crypto.livepeer.tv
I think it would an awesome thing to get ETH startups to use crypto.livepeer.tv
as a dedicated channel for their happenings and livestreams. In a perfect scenario, every startup within the ETH community will have a www.startup.io/livestream section on their website, where an embedded Crypto.livepeer.tv runs non-stop. Sponsoring each others, directly website to website
without passing through YT.
a Calendar for Crypto.livepeer.tv?

book a slot feature
login/txn to prove identity
disclaimer "LP is not responsible for malicious content/third party violations etc"
release of a Stream key valid for a specific time only.

LG xx
Mat
What does this pull request do? Explain your changes. (required)
Create a login/identity/calendar based system for crypto.livepeer.tv
in order to make the channel available to Crypto Startups to stream their happenings directly.
Specific updates (required)
crypto.livepeer.tv/Bookaslot sub-page
calendar page
fill-in fields with company, email contact etc.
automatic system/txn to prove identity
How did you test each of these updates (required)
I'm not a dev
Does this pull request close any open issues?
no
Screenshots (optional):
Checklist:

[ x] I have read the CONTRIBUTING document.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.

Hey there,
Yes indeed that is the idea for Crypto.Livepeer.tv! Awesome that projects
are paying you to fly out too. There is a separate calendar for
Crypto.livepeer.tv, and its here:
https://calendar.google.com/calendar/embed?src=livepeer.org_7p9cr4j343rla039ev8d4pu78s%40group.calendar.google.com&ctz=America%2FNew_York
Content is solely crypto - related, and projects including Status,
CryptoNYC, Truebit Art project, Aragon, Decentraland etc have been on
there, running their own town halls, conferences/meetups, or doing
interviews with us.
I just shared the calendar with you, and Chris has permissions to schedule
events on there. Maybe you could create a google form page that sends sign
ups directly to Chris' inbox so he can add the events and manage any
overlap? We also use it here in NYC, so I'll make sure we add any dates
we're using it so as not to overlap.
Raffi
ᐧ
On Thu, Jul 26, 2018 at 2:09 PM, Matlemad<EMAIL_ADDRESS>wrote:

Hello Folks! simple idea.
we crew from Livepeer.tv in Berlin are receiving a lot of requests from
ETH Community Startups
to stream their Meetups etc. Some also want us to fly to other
destinations to do it.
So I believe this is a demand that LP could help solving.
LP.tv Berlin has a second channel called Crypto.livepeer.tv
I think it would an awesome thing to get ETH startups to use
crypto.livepeer.tv
as a dedicated channel for their happenings and livestreams. In a perfect
scenario, every startup within the ETH community will have a
www.startup.io/livestream section on their website, where an embedded
Crypto.livepeer.tv runs non-stop. Sponsoring each others, directly
website to website
without passing through YT.
a Calendar for Crypto.livepeer.tv?

book a slot feature
login/txn to prove identity
disclaimer "LP is not responsible for malicious content/third party
violations etc"
release of a Stream key valid for a specific time only.

LG xx
Mat
What does this pull request do? Explain your changes. (required)
Specific updates (required)

How did you test each of these updates (required)
Does this pull request close any open issues?
Screenshots (optional):
Checklist:

I have read the CONTRIBUTING document.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added tests to cover my changes.

You can view, comment on, or merge this pull request online at:
https://github.com/livepeer/livepeerjs/pull/157
Commit Summary

Add ENS endpoint in SDK and updated GraphQL schema
Add GraphQL query for Broadcaster using ENS name
If channel is not an address try doing an ENS name lookup
Move mock GraphQL object functions to @livepeer/graphql-sdk to be
imported by other packages
Change GraphQL schema for Account to include ensName string and use
AccountBroadcasterQuery in @livepeerjs/player
feat(sdk): ENS reverse lookups, ETH addr / ENS name interop in rpc
method params
feat(graphql-sdk): Add ensName to many types, id / ens name interop
refactor(player): Update Channel query and render method
feat(explorer): ENS support
chore: Update lockfile
refactor(explorer/player): Simplify mock util function imports
feat(sdk): Support custom ENS registries in sdk options

File Changes

M packages/explorer/flow-typed/index.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-0 (1)
M packages/explorer/src/components/App.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-1 (10)
M packages/explorer/src/components/TranscoderCard.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-2 (6)
M packages/explorer/src/enhancers/index.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-3 (2)
M packages/explorer/src/utils.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-4 (72)
M packages/explorer/src/views/Account/enhance.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-5 (6)
M packages/explorer/src/views/Account/index.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-6 (26)
M packages/explorer/src/views/AccountBroadcasting/enhance.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-7 (2)
M packages/explorer/src/views/AccountDelegating/enhance.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-8 (2)
M packages/explorer/src/views/AccountOverview/enhance.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-9 (2)
M packages/explorer/src/views/AccountTranscoding/enhance.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-10 (2)
M packages/explorer/src/views/Approve/enhance.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-11 (8)
M packages/explorer/src/views/Bond/enhance.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-12 (8)
M packages/explorer/src/views/ClaimEarnings/enhance.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-13 (8)
M packages/explorer/src/views/Transcoders/enhance.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-14 (4)
M packages/graphql-sdk/coverage.svg
https://github.com/livepeer/livepeerjs/pull/157/files#diff-15 (2)
M packages/graphql-sdk/schema.json
https://github.com/livepeer/livepeerjs/pull/157/files#diff-16 (172)
M packages/graphql-sdk/src/fragments/Account.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-17 (1)
M packages/graphql-sdk/src/fragments/Broadcaster.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-18 (3)
M packages/graphql-sdk/src/fragments/Delegator.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-19 (1)
M packages/graphql-sdk/src/fragments/Job.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-20 (1)
M packages/graphql-sdk/src/fragments/Transcoder.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-21 (7)
M packages/graphql-sdk/src/index.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-22 (76)
M packages/graphql-sdk/src/index.test.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-23 (84)
M packages/graphql-sdk/src/index.test.js.md
https://github.com/livepeer/livepeerjs/pull/157/files#diff-24 (228)
M packages/graphql-sdk/src/index.test.js.snap
https://github.com/livepeer/livepeerjs/pull/157/files#diff-25 (0)
M packages/graphql-sdk/src/mock-sdk.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-26 (53)
M packages/graphql-sdk/src/queries.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-27 (19)
M packages/graphql-sdk/src/resolvers/Account.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-28 (12)
M packages/graphql-sdk/src/resolvers/Account.test.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-29 (1)
M packages/graphql-sdk/src/resolvers/Broadcaster.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-30 (10)
M packages/graphql-sdk/src/resolvers/Delegator.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-31 (10)
M packages/graphql-sdk/src/resolvers/Job.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-32 (14)
M packages/graphql-sdk/src/resolvers/Query.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-33 (12)
M packages/graphql-sdk/src/resolvers/Query.test.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-34 (14)
M packages/graphql-sdk/src/resolvers/Transcoder.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-35 (10)
M packages/graphql-sdk/src/types/Account.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-36 (3)
M packages/graphql-sdk/src/types/Broadcaster.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-37 (3)
M packages/graphql-sdk/src/types/Delegator.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-38 (3)
M packages/graphql-sdk/src/types/Job.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-39 (3)
M packages/graphql-sdk/src/types/Query.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-40 (18)
M packages/graphql-sdk/src/types/Transcoder.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-41 (3)
M packages/player/.env.development
https://github.com/livepeer/livepeerjs/pull/157/files#diff-42 (6)
M packages/player/src/views/Channel/index.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-43 (89)
M packages/player/src/views/Embed/index.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-44 (37)
M packages/sdk/src/index.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-45 (215)
M packages/sdk/src/index.test.js
https://github.com/livepeer/livepeerjs/pull/157/files#diff-46 (29)
M yarn.lock
https://github.com/livepeer/livepeerjs/pull/157/files#diff-47 (54)

Patch Links:

https://github.com/livepeer/livepeerjs/pull/157.patch
https://github.com/livepeer/livepeerjs/pull/157.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/livepeer/livepeerjs/pull/157, or mute the thread
https://github.com/notifications/unsubscribe-auth/AN9lGu5gUkonpxHjpF9TFq6ERUKQQoxZks5uKgXAgaJpZM4ViTg5
.

Hey Raffi, thanks a lot! I was aware there was already a calendar for Crypto. :)
What I have in mind is actually something more designed and more articulated on the architecture side, that's the reason why I opened a Pull Request :P
a GOOGLE CALENDAR has some some limitations,
the first of many being that someone has to take care of it,
and organise it/resolve/contact the applicants if there are irregularities in the booking or in proving Identity.
The best scenario would be to have a crypto.livepeer.tv PAGE
with the player on top (+ embedding code)
and the infos and link to booking form right below.
BOOKING FORM
Let me link you to an example I bumped into recently, as I'm looking for venues for a tour in September.
http://www.theearlybird.at/at/en/booking
it's a bar in Innsbruck, Austria, they have this clear (but in German!) form where they lead you through the rules of the self booking, and at the end there's a calendar widget that shows the availabilities.
I believe that in our case this very last part could definitely be connected to the already existent Google Calendar for Crypto.
ID verification
we might still need a certain degree of human verification to make sure that behind e.g. StartupXYZ there is a active startup, member of the ETH Community (and not villains).
But I also think that a simple transaction could solve the ID problem.
Maybe the StartupXYZ could be asked to do a gas txn, copy the txn number somewhere (on their website?) as a stamp?
Temporary Stream Key
This is, I suppose, the most tricky part.
So, once the whole booking and ID verification is accomplished, Crypto should release privately and securely a streaming key that allows the applicant to access to Crypto.livepeer.tv for the reserved slot and proceed to stream their Meetup (if the stream happens 18:00 to 20:00, it's suggested to reserve 19:00 to 21:00)
Let me know if it makes more sense.
Hugs!
M

@Matlemad who worked on this PR?  I see a lot of ENS integrations - want to understand what's happening here.

@ericxtang I see a lot of contributions too, but handling them is unfortunately beyond my knowledge.
I've only translated what I saw as a need and a desire from many actors in the ETH and BC community
to use Livepeer instead of YT, and proposed a temporary quick solution to get these people sharing and determining the palimpsest of crypto.livepeer.tv in an autonomous way.
Please let me know how I can help further!
xM

@iSasFTW yes I know and... my bad. Shall I move it into Issues?

Yop! and sorry for the confusion created!


daemon: improve server start-up to reclaim old socket

I accidentally submitted my change to the daemon without the comment about making the directory, so I decided to fix the code to do the todo. And while I was there, it was easy to make it handle this case, as well.

[Current coverage][cc-pull] is 72.45% (diff: 34.00%)

Merging [#64][cc-pull] into [master][cc-base-branch] will decrease coverage by 2.54%

@@             master        #64   diff @@
==========================================
  Files            33         35     +2
  Lines          1516       1590    +74
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           1137       1152    +15
- Misses          335        387    +52
- Partials         44         51     +7

Powered by Codecov. Last update [7a6f1b8...d68b817][cc-compare]
[cc-base-branch]: https://codecov.io/gh/scootdev/scoot/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/scootdev/scoot/compare/7a6f1b8dbc375aab6467b1b287eb3befb051fd01...d68b8172be27508da19754d127bb8456ab70653e?src=pr
[cc-pull]: https://codecov.io/gh/scootdev/scoot/pull/64?src=pr

LGTM

lgtm


Fix issue #315, a regression on custom params and headers (PR #248)

Thanks for the contribution 👍


JSONDecodeError when uploading a valid CSV

When attempting to upload a CSV training set for my model I receive a JSONDecodeError error. I tried uploading my smaller validation set too, but it also failed. I'm not entirely sure why JSON decoders are even being ran against a CSV file.
At first I thought maybe the CSV was invalid, but it checks out. I am not sure how to debug this problem.
Any help is greatly appreciated! Thank you.
Valid CSV
$ csvclean ~/training_set.csv
No errors.

Example CSV data
col_one,col_two
TRUE,"Lorem ipsum dolor sit amet, consectetur adipiscing elit"
FALSE,"Ut id ex luctus ""with quoted text inside"" vitae tincidunt nibh"
TRUE,"Nam ligula nibh, dapibus eget justo vitae"
FALSE,"Cras sed molestie enim. Etiam facilisis erat id bibendum"

Upload attempt
$ autonlp upload --project my_project \
	--split train \
	--col_mapping col_one:target,col_two:text \
	--files ~/training_set.csv

> INFO    Uploading files for project: my_project
> INFO    🗝 Retrieving credentials from config...
> INFO    ☁ Retrieving project 'my_project' from AutoNLP...
> INFO    🔄 Refreshing project status...
> INFO    🔄 Refreshing uploaded files information...
> INFO    🔄 Refreshing models information...
> INFO    🔄 Refreshing cost information...
> INFO    ✅ Successfully loaded project: 'my_project'!
> INFO    Mapping: {'col_one': 'target', 'col_two': 'text'}
Traceback (most recent call last):
  File "/usr/local/bin/autonlp", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/autonlp/cli/autonlp.py", line 57, in main
    details = err.response.json().get("detail")
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 900, in json
    return complexjson.loads(self.text, **kwargs)
  File<EMAIL_ADDRESS>line 346, in loads
    return _default_decoder.decode(s)
  File<EMAIL_ADDRESS>line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File<EMAIL_ADDRESS>line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Environment
$ autonlp --version
0.3.1

$ python -V
Python 3.9.5

$ pip -V
pip 21.1.3

Hello @fjarrett !
Thanks for reporting this issue.
The error is most likely raised by the huggingface_hub dependency that we use to interact with HuggingFace's models and datasets hub. More precisely, I suspect the error is raised from this function in AutoNLP, which clones the project's dataset repo on your machine. Indeed, cloning dataset repos is broken in the latest huggingface_hub release (see this issue ).
We just released AutoNLP 0.3.2 that pins an anterior version of the huggingface_hub package (namely, 0.12.0). Could try updating AutoNLP and retrying to see if it solves your issue?

@SBrandeis Your hunch was right, I upgraded to 0.3.2 and the uploads worked. I am training our project models now. Thank you for the assist!

@SBrandeis it looks like our training failed, I don't see an error code/message so not sure how to debug. Any chance someone can take a look and see what could be the issue? We have used this exact dataset on Amazon Comprehend in the past.
📁 training_set_out.csv (id # 385)
   • Split:             train
   • Processing status: ✅ Success!
   • Last update:       2021-06-30 13:58 Z
📁 validation_set_out.csv (id # 386)
   • Split:             valid
   • Processing status: ✅ Success!
   • Last update:       2021-06-30 13:59 Z

~~~~~~~~~~~~ Models ~~~~~~~~~~~

+----+--------+--------+--------------------+--------------------+
|    |   ID   | Status |   Creation date    |    Last update     |
+----+--------+--------+--------------------+--------------------+
| ❌ | 302893 | failed | 2021-06-30 14:03 Z | 2021-06-30 14:12 Z |
| ❌ | 302894 | failed | 2021-06-30 14:03 Z | 2021-06-30 14:12 Z |
| ❌ | 302895 | failed | 2021-06-30 14:03 Z | 2021-06-30 14:12 Z |
| ❌ | 302896 | failed | 2021-06-30 14:03 Z | 2021-06-30 14:12 Z |
| ❌ | 302897 | failed | 2021-06-30 14:03 Z | 2021-06-30 14:12 Z |
| ❌ | 302898 | failed | 2021-06-30 14:03 Z | 2021-06-30 14:12 Z |
| ❌ | 302899 | failed | 2021-06-30 14:03 Z | 2021-06-30 14:12 Z |
| ❌ | 302900 | failed | 2021-06-30 14:03 Z | 2021-06-30 14:12 Z |
| ❌ | 302901 | failed | 2021-06-30 14:03 Z | 2021-06-30 14:12 Z |
| ❌ | 302902 | failed | 2021-06-30 14:03 Z | 2021-06-30 14:12 Z |
+----+--------+--------+--------------------+--------------------+

Awesome 😄 I'm closing this issue, feel free to open another one if you encounter another problem !

@fjarrett looking into this now!

@SBrandeis thank you! and sorry for the head fake 😸

I had a look at the parameters of your project - the problem is that you're trying to train a model that's not compatible with 🤗 Transformers' TextClassification pipeline.
You can browse the TextClassification-compatible models here: https://huggingface.co/models?pipeline_tag=text-classification
If you want to try out that specific model on your use case, you might want to have a look at 🤗's Inference API if you haven't already: https://huggingface.co/inference-api
Let me know if it helps!

@SBrandeis I tried again this time using --hub_model textattack/facebook-bart-large-MNLI instead since that model was tagged for TextClassification but the training failed again

@SBrandeis trying again with --hub_model roberta-large-mnli this time 🤞

That might not work either. For text-classification its best to select models which are not finetuned on a downstream task. Try: roberta-large :)

@SBrandeis ok trying that now

@abhishekkrthakur @SBrandeis I tried roberta-large but that also failed. I finally resorted to training without any --hub_model specified and that did work. Could it be that fine-tuning existing hub models is broken right now?
FAILED facebook/bart-large-mnli
FAILED textattack/facebook-bart-large-MNLI
FAILED roberta-large-mnli
FAILED roberta-large

Oh, it seems I was wrong. The roberta-large run did, in fact, succeed. I must have been looking at the wrong project. Thank you both for your help! I learned a lot. 🙌


Support dockerfile build parameter as alternative to image

docker compose supports the "build" parameter which lets you supply a directory which holds a dockerfile. When running a pod, it should build the image automatically to the daemon.
https://docs.docker.com/compose/yml/#build

Hi @darylteo
Really thanks to your suggestion, we will consider this enhancement.
Regards,


FINERACT-1858: Backport Flyway migrations to Liquibase for 1.7.x

Backport FINERACT-1846 for 1.7.x releases.

Hello Aleks,
Quick question the v1.7.3 binaries are available in Github? Apache Fineract.
Regards
Victor
El mar, 21 feb 2023 a las 0:23, Aleksandar Vidakovic (<
@.***>) escribió:

Merged #2926 https://github.com/apache/fineract/pull/2926 into 1.7.3.
—
Reply to this email directly, view it on GitHub
https://github.com/apache/fineract/pull/2926#event-8567220313, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ALD2ZAW2EVDRM66KXPC3TFDWYRNNVANCNFSM6AAAAAAUGF7AEI
.
You are receiving this because your review was requested.Message ID:
@.***>


Added additional check for empty object when upserting to Pinecone Index

Langchain checks for null properties and deletes these before upserting to the Index. I've added an additional check for empty objects in the metadata, as these are not handled when the array is flattened.
I had issues when upserting to Pinecone, and I realised this was because of the fact that sometimes there were empty objects as properties in the PDF metadata. The proposed change fixed this issue.

Thanks!


Rename tf.nn.swish to tf.nn.silu to give appropriate credit

The swish was originally coined the "SiLU" in https://arxiv.org/pdf/1606.08415.pdf and https://arxiv.org/abs/1702.03118 long before the swish paper. Renaming other peoples' exact same ideas is unacceptable and tensorflow's naming convention implicitly erases the research and work of people outside of Google.
This request inspired by a recent discussion, but this problem has been brought up every few months for the past few years. In light of recent efforts to make the ML community more equitable and fair, this is a no-brainer and long overdue.
Will this change the current api? How?
tf.nn.swish ought to be renamed tf.nn.silu and both of the aforementioned papers will be cited in the documentation. At the very least they should be documented in the swish documentation already.

Thank you for bringing this to our attention. Due to backwards compatibility constraints, we cannot remove tf.nn.swish, but we will a expose the same functionality as tf.nn.silu to give proper credit to the earlier invention.


Update user guide to include list of food items for find

Codecov Report

Merging #149 (069e6dd) into master (e8303f9) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #149   +/-   ##
=========================================
  Coverage     76.05%   76.05%
  Complexity      567      567
=========================================
  Files            91       91
  Lines          1754     1754
  Branches        145      145
=========================================
  Hits           1334     1334
  Misses          372      372
  Partials         48       48

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


error with 4 terminlas

I configured the color, icon and name in 4 terminals, a day later when I used the returned terminal, the name of the terminal that I left open is changed to the default name.
I use WSL on win10 with python3
now:

yesterday:

update: I run a program in the terminal, stop it and switch to another terminal and the name of the first terminal is changed

I believe you're using an old version of VS Code bc cannot repro this. Pls create a new issue with your system's information if you can repro this in insider's.


Auth

This is the first commit towards having Authorizer be an external concept and configurable by client applications.

Moves the state of the art forward. Whilst there is 100% coverage on Auth, there are no unit tests for it. Good enough for a merge, but please add unit tests ASAP.


Error occurred when executing command: yarn run dev

I've done some code and played with this template for days, and it works fine. But once I restart my vs code, when I run npm run electron-dev, it shows error
[0] Error occurred when executing command: yarn run dev
[0] Error: spawn cmd.exe ENOENT
[0]     at ChildProcess._handle.onexit (node:internal/child_process:283:19)
[0]     at onErrorNT (node:internal/child_process:476:16)
[0]     at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
[1] Error occurred when executing command: yarn run electron
[1] Error: spawn cmd.exe ENOENT
[1]     at ChildProcess._handle.onexit (node:internal/child_process:283:19)
[1]     at onErrorNT (node:internal/child_process:476:16)
[1]     at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
[1] yarn run electron exited with code -4058
[0] yarn run dev exited with code -4058

What is it? and how to fix it?

Can you try and reinstall the dependencies once, also check if yarn is installed on your system and can be accessed through cmd or console whatever it is. Also mention the name of OS, OS version, Node version and Yarn version you are using.


Trading interface getTrades() returns not a table

As the title, here is my code.
-- Assuming td=peripheral.wrap("left") is already defined at the beginning

while true do
    print("Trying to get trades...")
    local success, tradesData = pcall(td.getTrades)
    if success then
        print("Success:", success)
        print("Type of trades:", type(tradesData))
        print("Trades data:", tradesData)
        if type(tradesData) == "table" then
            -- If the type of tradeData is table, then use directly.
            local trades = tradesData
            print("Trades found.")
            print("Trades data:", textutils.serialize(trades))
            -- Looking for knockback II enchanted book.
            local knockbackTradeFound = false
            for _, trade in ipairs(trades) do
                for _, offer in ipairs(trade) do
                    if offer and offer.id == "minecraft:enchanted_book" and offer.enchanted_book.level == 2 then
                        knockbackTradeFound = true
                        break
                    end
                end
                if knockbackTradeFound then
                    break
                end
            end
            if knockbackTradeFound then
                print("Knockback trade found!")
                break
            end
        else
            print("Invalid trades data.")
        end
    else
        print("Failed to get trades.")
    end
    os.sleep(0.1)
end

And its output messages are:"Trying to get trades...""Success:true""Type of trades:boolean""Trades data:true"and"Invalid trades data."
If this means that getTrades() returns not a table？Or there are some problems in my code. THX for pointing out.

Alright, I know it.The return not includes NBT data.


1.20.4 Compatibility

Basically, the title.
This mod makes the GUI for the Smithing Table look so cool but unfortunately, it's not compatible with the latest version of Minecraft.

I am working on it

Done, the new version should become available soon


Keras versioning error

Error using Keras version 2.1.3 and tensorflow version 1.5.0
https://github.com/gwaygenomics/tybalt/blob/84c6079f98ac4aa9269eead0afbd424369a94d20/scripts/vae_pancancer.py#L163

Thanks @ajlee21 - if possible, can you post the error here?
Tybalt currently uses Keras 2.0.6 and tensorflow 1.0.1 but its probably time for an update and for this to be fixed.

Using TensorFlow backend.
/home/alexandra/anaconda2/envs/python3/lib/python3.6/site-packages/ipykernel_launcher.py:135: UserWarning: Output "custom_variational_layer_1" missing from loss dictionary. We assume this was done on purpose, and we will not be expecting any data to be passed to "custom_variational_layer_1" during training.

ValueError                                Traceback (most recent call last)
 in ()
139 hist = vae.fit(np.array(rnaseq_train_df), shuffle=True, epochs=epochs, batch_size=batch_size,
140                validation_data=(np.array(rnaseq_test_df), np.array(rnaseq_test_df)),
--> 141                callbacks=[WarmUpCallback(beta, kappa)])
142
143 # Save training performance
~/anaconda2/envs/python3/lib/python3.6/site-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
1612                 sample_weight=val_sample_weight,
1613                 check_batch_axis=False,
-> 1614                 batch_size=batch_size)
1615             if self.uses_learning_phase and not isinstance(K.learning_phase(), int):
1616                 val_ins = val_x + val_y + val_sample_weights + [0.]
~/anaconda2/envs/python3/lib/python3.6/site-packages/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_batch_axis, batch_size)
1428                                     output_shapes,
1429                                     check_batch_axis=False,
-> 1430                                     exception_prefix='target')
1431         sample_weights = _standardize_sample_weights(sample_weight,
1432                                                      self._feed_output_names)
~/anaconda2/envs/python3/lib/python3.6/site-packages/keras/engine/training.py in _standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
53             raise ValueError('Error when checking model ' +
54                              exception_prefix + ': '
---> 55                              'expected no data, but got:', data)
56         return []
57     if data is None:
ValueError: ('Error when checking model target: expected no data, but got:', array([[5.000000e+00, 2.440000e+02, 4.000000e+00, ..., 1.486718e+06,
7.000000e+00, 6.000000e+00],
[0.000000e+00, 2.880000e+02, 0.000000e+00, ..., 3.668960e+05,
0.000000e+00, 2.000000e+00],
[7.000000e+00, 1.620000e+02, 4.000000e+00, ..., 6.256400e+05,
3.000000e+00, 2.000000e+00],
...,
[1.300000e+01, 3.045000e+03, 6.000000e+00, ..., 5.851519e+06,
4.100000e+01, 4.800000e+01],
[7.000000e+00, 5.060000e+02, 2.000000e+00, ..., 1.172282e+06,
2.000000e+00, 6.000000e+00],
[2.000000e+00, 6.970000e+02, 1.000000e+00, ..., 3.982010e+05,
1.000000e+00, 0.000000e+00]]))


VPKEdit CLI has missing dependencys or im just a idiot with linux

./vpkeditcli: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by ./vpkeditcli)
this is what it says its missing

Try running:
sudo apt update
sudo apt install libstdc++6

still missing that version here is the output using strings
root@localhost:~# strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_3.4.20
GLIBCXX_3.4.21
GLIBCXX_3.4.22
GLIBCXX_3.4.23
GLIBCXX_3.4.24
GLIBCXX_3.4.25
GLIBCXX_3.4.26
GLIBCXX_3.4.27
GLIBCXX_3.4.28
GLIBCXX_DEBUG_MESSAGE_LENGTH

Ah just one off
Can I ask what distro and version you're running?

here is the os-release files output
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian

im going to upgrade to Debian 12 and test it out

I've been looking for a way to upgrade that package for Debian 11, but updating to Debian 12 would definitely fix it
I run that on my laptop that I use for Linux testing

It Works on Debian 12

The application relies on modern C++ features, and frankly I don't know Linux well enough to see if I can get it to run with older glibcxx versions, so I'm going to mark this as a wontfix


Cb 2.51.0 merge2

See detailed description in the commit message.

@aszegedi Is it enough to delete just the conflicted file or are there some other changes needed?

Thank you @nadamdb for your help on this! The integration-test/src/main/java/com/sequenceiq/it/cloudbreak/testcase/e2e/environment/AzureEnvironmentWithCustomerManagedKeyTests.java is not present on the CB-2.52.0 or master and not needed there indeed (this is 2.51.0 only code change). So we can accept the related changes for this from these branches, that means remove the file and the related commit content from here.


Error "Cannot find module 'crypto'" when using hedera-wallet-connect

Hello,
I am using hedera-wallet-connect and I encounter an error related to a missing module 'crypto' when attempting to run the application,

Steps to Reproduce:
Run the application using the provided setup.
Navigate to the login page.(the error occurs on every page using hedera-wallet-connect under the hood, not only login)
Observe the error message stating "Cannot find module 'crypto'".
Additional Information:
This issue seems to be related to the hedera-wallet-connect library and also same behavior when using hashconnect .
Environment:
Node.js Version: v21.7.1
NextJS Version: 14.1.0
hedera-wallet-connect Version: 1.0.6
hashconnect Version: 3.0.13
Please let me know if further information or assistance is required to address this issue.
Thank you.

Downgrading Hashconnect to "^0.2.4" worked, but I don't think that's a suitable solution.

@oynozan I attempted before to resolve the problem by using that solution, but it resulted in the loss of features that I need, which are present in versions beyond 3.0.0

I faced with the same problem when I downgraded, so it's not a suitable solution as I mention.

could be an SSR issue, could you share a repro?

I've had the same issue here (forgot to search for similar issues before submitting it 🙈): https://github.com/hashgraph/hedera-wallet-connect/issues/190
There's a repo linked in the issue.

just try using dynamic imports with ssr disabled, or installing a crypto pollyfill as dep.

just try using dynamic imports with ssr disabled, or installing a crypto pollyfill as dep.

You're right, this does resolve the issue. There's an example of it mentioned at the bottom of the description in ticket #190 for anyone else reading this.
For anyone coming to this repo using something like NextJS we should be able to handle this at the library level and have full support for SSR built in like a lot of other libraries are currently offering. Then there wouldn't be a need to dynamically import.

For anyone coming to this repo using something like NextJS we should be able to handle this at the library level and have full support for SSR built in like a lot of other libraries are currently offering. Then there wouldn't be a need to dynamically import.

Yes, you could check if window is defined before executing any browser dependant function 👍

@TKarima22 does this PR solve this issue for you : https://github.com/hashgraph/hedera-wallet-connect/pull/224.
You can test by using a pre-release version of the library built off of that PR - npm i<EMAIL_ADDRESS>

closing as completed in v1.3.2. If you're still experiencing issues please feel free to re-open


Error during installation

Hello :),
this is more a help request than a error report for this helpful package.
If I run it locally it works fine.
When I want to run it with a Gitlab-Runner via the Powershell it throws an error:

I would also guess that it is because of writing permission according to the error message but I already run it with admin rights for other script parts which also need it.
Does someone have a clue what could be wrong, what I could check?
I would appreciate any help and advice.
Best regards
Edit:
Here is also the -Verbose call output but in my perspective it does not give more information for me about what the cause could be:

Finally found the issue after nearly two months ^^; .
The gitlab-runner is running in the background as "System" user which root folder is in C:\Windows\System32...
The module is storing them as default in the root directory which is there.
Those execution files can't be run in this place.
The command Install-UnitySetupInstance allows a input configuration -Cache which changes this target to a custom one.
For me C:.unitysetup
Kudos to @jwittner for taking this in consideration.
Thanks for you work here.


[staging-next] python3Packages.promise: fix tests

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 117230 at 2e90e558 run on x86_64-linux 1

2 packages failed to build:

python38Packages.fastapi
python39Packages.fastapi

8 packages built successfully:

python38Packages.graphene
python38Packages.graphql-server-core
python38Packages.promise
python38Packages.starlette
python39Packages.graphene
python39Packages.graphql-server-core
python39Packages.promise
python39Packages.starlette

Note that build failures may predate this PR, and could be nondeterministic or hardware dependent.
Please exercise your independent judgement.

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 117230 run on x86_64-linux 1

2 packages failed to build and already failed to build on hydra master:

python38Packages.fastapi: plain log  |  pretty log
python39Packages.fastapi: log was empty

8 packages built:

python38Packages.graphene
python38Packages.graphql-server-core
python38Packages.promise
python38Packages.starlette
python39Packages.graphene
python39Packages.graphql-server-core
python39Packages.promise
python39Packages.starlette


fix(s3): consistent s3 bucket name usage

what

consistent s3 bucket name usage

why

otherwise policies fail miserably

@qwelias thanks for the PR.
yes it will fix the issue with the 63-char limit.
But we are updating our modules to use context.tf pattern, which has already that functionality so all modules get it automatically.
See this as an example https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/context.tf#L156
We'll update this module as well

Nice, feel free to close it if the update is coming soon. Tho I don't see how merging it in now may hurt.


fix problem with dot files

The solution for issue #9  Problem with dot files

@ximet Thanks! I think the issue has to do with files being in the source directory along with non-markdown files being in the source directories.
This fix rather checks for dot files in the source directories which might not solve the problem completely.

@kbrsh I think, if you need resolve the problem completely, you'll need to add glob npm-package to your repository. But now, I cannot use your application without that hot-fix

@ximet I just fixed the issue in 31c8736e7f09a0a9a38d135cefefd4ad8255aa67. I'm currently working on the new release.


Look into the no_plugins configuration

WARNING  (root) The following configurations are in conda but unrecognized by conda-ops: ['no_plugins']

This option is in it's own Plug-in section. It's put in the 'rest of the pile' configuration options that get tracked (not whitelist, not blacklist).


Agora sometimes emits a socket error

Nov 13 08:06:13 eu-002 docker[14457]: sock error 104!

It seems to originate from here:
https://github.com/vibe-d/eventcore/blob/a462db4e04ea4c2f4f227dbf5fc74c605e469340/source/eventcore/drivers/posix/sockets.d#L412
I'm not sure why it's using the print function in https://github.com/vibe-d/eventcore/blob/a462db4e04ea4c2f4f227dbf5fc74c605e469340/source/eventcore/internal/utils.d#L8, is this supposed to be a non-recoverable error?

Update: This error is spurious and not reproducible.

Any thoughts on this issue?

Does it still happen or not?

It does, but happens at random times and the error # is various.

If it's not blocking anything you can unassign from this issue. But we should keep it open until it can be resolved by someone who can tackle this.

does this happen in the rest client or server?

I wonder if this 104 error is just a sideeffect of some agora instances crashing and the client side would still try to read from the (unproperly closed) socket. Did any crash happen around the time when this error message appeared? Also I saw the same problem WITHOUT using docker and just using faucet, it would probably be better to run it outside of docker for the test.
I also checked that after fixing this: vibe-d/vibe.d#2506 which would crash agora server, I stopped seing this 104 issue.
Also this ticket also talking about crashing and seeing this 104 error: vibe-d/vibe.d#1714

I wonder if this 104 error is just a sideeffect of some agora instances crashing and the client side would still try to read from the (unproperly closed) socket.

Yes I think this might be the reason. I remember other nodes crashing at the time.

I also checked that after fixing this: vibe-d/vibe.d#2506 (which regularly crashed agora server), I stopped seing this 104 issue.

Perfect. Then let's just update Agora with the latest vibe.d submodule and close the issue. Thank you for fixing this!

@omerfirmak

thx @omerfirmak for solving the original #2506 issue together

thx @omerfirmak for solving the original #2506 issue together

It was a pleasure :)

After integrating https://github.com/vibe-d/vibe.d/issues/2506, we might want to increase our timeout duration as well. To minimize the overhead of dropping connections.

Is there anything else that needs to be updated other than vibe.d & vibe-core? I'm getting errors:

submodules/vibe.d/data/vibe/data/serialization.d(487,8): Error: no property formattedWrite for type std.array.Appender!string

It was building fine with only v0.9.2 + the fix we proposed. We are on master now, vibe.d master might be broken.

https://github.com/vibe-d/vibe.d/blame/e350f3221c477fb7a3c9a1a209b305894d8786aa/data/vibe/data/serialization.d#L487
Weirdly, it is missing a import std.format : formattedWrite;
I will open a PR for it.

might also be worth adding a unit test that specifically triggers compilation of that branch
we already have a unit tests that triggers this branch I think:
https://github.com/vibe-d/vibe.d/blame/e350f3221c477fb7a3c9a1a209b305894d8786aa/data/vibe/data/serialization.d#L484

Is there anything else that needs to be updated other than vibe.d & vibe-core? I'm getting errors:

submodules/vibe.d/data/vibe/data/serialization.d(487,8): Error: no property formattedWrite for type std.array.Appender!string

should be fixed by adding the import as @omerfirmak suggested
https://github.com/vibe-d/vibe.d/pull/2510
added some extra tests too

sock error 54! is often emitted in a local environment with latest versions of agora and stoa. It happens in the beginning.

sock error 54! is often emitted in a local environment with latest versions of agora and stoa. It happens in the beginning.

Do we still see this?

agora crashing?
https://github.com/bosagora/agora/issues/1362#issuecomment-741802077

Not seen for a while, closing.

talked about this with Ku, and it only happens during restart(when some agora nodes stops/some still do requests), so this is similar to the case when agora instances crashing


Fix relative heights of the editor widgets

Spotted in https://github.com/European-XFEL/DAMNIT/pull/119#discussion_r1376379338.

Correct, I actually have no idea why I changed that in the first place :upside_down_face: But it should be as small as possible in the case where there are no errors and doesn't need to be larger.

OK, then LGTM :+1:


Use of Simple-AR in react

Thank you for your Simple-AR it works great out of the box!
However, I have been working for many hours trying to use Simple-AR in a React app but failed. May I know if you have any plan to make it work for react?

@pyrostudiosg hi! Thank you for your inquiry. I will add it to my action list. Currently I am finalizing Unity package and will try to work on React.


Speed up tests

Coverage remained the same at 91.901% when pulling 34df286b997339c72e3c8a67669a6afb1788ea55 on trim-tests into 98f3b7ac65e4983e24db4aa288b6ce6efffa1d0c on master.


[WIP] Add support for directory hashing using IPFSHashonly

Proposed changes
This PR updates aea.helpers.base.IPFSHashonly tool to hash directories as well.
Fixes
If it fixes a bug or resolves a feature request, be sure to link to that issue.
Types of changes
What types of changes does your code introduce to agents-aea?
Put an x in the boxes that apply

[ ] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist
Put an x in the boxes that apply.

[x] I have read the CONTRIBUTING doc
[ ] I am making a pull request against the develop branch (left side). Also you should start your branch off our develop.
[ ] Lint and unit tests pass locally with my changes and CI passes too
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have checked that code coverage does not decrease.
[ ] I have added necessary documentation (if appropriate)
[ ] Any dependent changes have been merged and published in downstream modules

Further comments
If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...

refactor:
https://github.com/valory-xyz/open-aea/blob/294b92df8d704e169df55214c8a9d8a5f7d233ac/aea/helpers/ipfs/base.py#L126-L133
https://github.com/valory-xyz/open-aea/blob/294b92df8d704e169df55214c8a9d8a5f7d233ac/aea/helpers/ipfs/base.py#L141-L145
https://github.com/valory-xyz/open-aea/blob/294b92df8d704e169df55214c8a9d8a5f7d233ac/aea/helpers/ipfs/base.py#L194-L198
e.g.
def create_link(hash, size, name) -> PBLink:
    link = merkledag_pb2.PBLink()
    link.Hash = hash
    link.Tsize = size
    link.Name = name
    return link

Codecov Report

Merging #82 (05d3f83) into main (ec0a276) will increase coverage by 0.09%.
The diff coverage is 65.33%.

@@            Coverage Diff             @@
##             main      #82      +/-   ##
==========================================
+ Coverage   96.76%   96.86%   +0.09%
==========================================
  Files         317      307      -10
  Lines       24763    24245     -518
==========================================
- Hits        23962    23485     -477
+ Misses        801      760      -41

Flag
Coverage Δ

unittests
96.86% <65.33%> (+0.09%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

aea/helpers/ipfs/base.py
82.19% <64.86%> (-17.81%)
:arrow_down:

aea/configurations/base.py
100.00% <100.00%> (ø)

aea/__init__.py
100.00% <0.00%> (ø)

packages/fetchai/protocols/gym/dialogues.py
100.00% <0.00%> (ø)

packages/fetchai/connections/gym/__init__.py
100.00% <0.00%> (ø)

packages/fetchai/contracts/__init__.py

packages/fetchai/skills/gym/dialogues.py

.../aea-ledger-fetchai/aea_ledger_fetchai/__init__.py

...s/aea-ledger-fetchai/aea_ledger_fetchai/fetchai.py

...s/aea-ledger-fetchai/aea_ledger_fetchai/_cosmos.py

... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ec0a276...05d3f83. Read the comment docs.

Build completed!
    _     _____     _
   / \   | ____|   / \
  / _ \  |  _|    / _ \
 / ___ \ | |___  / ___ \
/_/   \_\|_____|/_/   \_\

v1.5.0

===========================================================================================
|  ComponentId                       |  IPFSHash                                          |
===========================================================================================
|  (protocol, open_aea/signing:1.0.0)|  QmT4MmP1MYDfk9fNwRDYBnT4pp1tms62XkGW957dMrDcwb    |
|  (protocol, fetchai/default:1.0.0) |  QmRx9V1FeuUc3VGi3mmJhZk2pK6cn341zhauM9kzL9FyKJ    |
|  (connection, fetchai/stub:0.21.0) |  QmXFpXZEYWQJ4r1ZadwsBiz2fRYWp1YtbeurFPH9sLHrXF    |
|  (skill, fetchai/echo:0.19.0)      |  QmTkhfryfN2EjZy8EjwrJVfA84YEcHK1VP9Fib2bCQ1S5C    |
===========================================================================================

open_aea/protocols/signing,QmT4MmP1MYDfk9fNwRDYBnT4pp1tms62XkGW957dMrDcwb
fetchai/protocols/default,QmRx9V1FeuUc3VGi3mmJhZk2pK6cn341zhauM9kzL9FyKJ
fetchai/connections/stub,QmXFpXZEYWQJ4r1ZadwsBiz2fRYWp1YtbeurFPH9sLHrXF
fetchai/skills/echo,QmTkhfryfN2EjZy8EjwrJVfA84YEcHK1VP9Fib2bCQ1S5C

LGTM

another stylistic thing: I would pull out the class methods (and static) methods that have nothing to do with the instance / state. Classes are overused because people default to them, whereas regular functions are perfectly fine to use.


Compile error on centos 6

/usr/local/lib/libboost_thread.a(thread.o):thread.cpp:(.text+0x30c): more undefined references to clock_gettime' follow collect2: ld returned 1 exit status make[2]: *** [tests/cpp-netlib-utils_thread_pool] Error 1 make[2]: Leaving directory /root/cpp-netlib-0.9.4/build'
make[1]: *** [libs/network/test/CMakeFiles/cpp-netlib-utils_thread_pool.dir/all] Error 2
make[1]: Leaving directory `/root/cpp-netlib-0.9.4/build'
make: *** [all] Error 2

You're using a very old version of cpp-netlib. Please use the latest release (0.11.2) from http://cpp-netlib.org/.

Solved using latest version.


Add arg parameter to simplify format

I've have some problem trying format the ids since the mode is always the last argument,  I try pass the ids or * a lot of times with Django 1.9 but the code explode ask me for a model in the correct format.
Let me know if it's the wrong solution.

I realize this is related to https://github.com/davedash/django-fixture-magic/pull/48, Let me know if I made this wrong.


Title and content indicates Europe, first line indicates Brazil

Pleas change Brazil to Europe.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: cd14df17-76ba-883b-7a17-c50c7c87af0f
Version Independent ID: c3560cb9-c651-3425-d2ee-34df04335a7f
Content: Europe - Finance & Operations | Dynamics 365 | #MSDyn365FO
Content Source: articles/financials/localizations/europe.md
Service: dynamics-ax-applications
GitHub Login: @ShylaThompson
Microsoft Alias: shylaw

Thanks @MarkElenbaas  for pointing out this error. This will be fixed by end of day today.


Concern about parent as a second prototype chain

This was a concern we discussed regularly when @littledan was the Decorators champion. We've generally wanted to avoid a second prototype chain-like behavior with metadata as it doesn't stand up to someone calling Object.setPrototypeOf on the class constructor, or to return obj within a class constructor.
If we move forward with an opaque immutable object, I propose that we do not include a parent property. This would have several implications:

Outside of a decorator: To read "inherited" metadata from a class you would need to walk the class's prototype chain.
Inside of a decorator: You would either have no access to "inherited" metadata, or we could provide the superclass constructor in the decorator's context (from which you could read that constructor's [Symbol.metadata] property).

There's a tradeoff between convenience in the common case (normal class inheritance) vs defensiveness against uncommon cases (changing the class prototype, the override trick). Given just how unusual the uncommon cases are in my experience, I would be inclined to make the common case more convenient, personally. But I could be persuaded that this tradeoff should come out the other way, particularly if there's concrete, reasonable cases we expect to come up where this would break.
(Also: I think it makes sense that you still get the parent class's metadata when using the return override trick? The base class is still the base class, even if it chooses to do something unusual for the instances it constructs; the metadata is about the class, not its instances. Concretely: if the base class is e.g. returning a Proxy around the instance it just constructed, or returning cached previously-constructed instances, or similar, then the metadata still applies. And if it's doing something more exotic than that, I think all hope of declarative programming is lost.)

or we could provide the superclass constructor in the decorator's context

How is that better than the current parent design? It seems like that would be no more robust against someone changing out the [[Prototype]] after the fact, nor against the return override trick.

Maybe metadata should be bundled with prototype and manipulated by a global api Metadata like Reflect. We don't need to care about the changing of prototype chain because users should take care it.
Also I don't think metadata is a decorator behavior. That's over design.

@ruojianll there are a lot of barriers to adding a global API, it seems unlikely that it would gain consensus. Reflect specific is contentious because some members have asserted that it should only have methods which relate to objects (e.g. the same as Proxy handlers).


handle client python errors

This change introduces logging for kube-shell that redirects errors to ~/.kube/shell/errors.log. All client-python errors will be caught and go here.

thanks @vogxn


The Big Book of Healthcare

The description of twelve countries' healthcare's systems. How each handle different cases, how is it financed, how much it costs in total for each of the participants.
There's too much myth in this area.

This sounds great and it could pair well with a site focused on [[statistics]]. [[metasj]] might be able to point to existing [[underlays]] in this space.
Why twelve?

Why twelve?

A rough estimate based on one chapter per country, half-dozen chapters for intro, conclusions and such, around 15-25 pages per chapter.

This video Sistemas de Saúde no Mundo explain some healthcare systems and has references to larger comparative studies.


Exclude SpatialRefSys table from export task

https://rollbar.com/dfe/teacher-vacancies/items/2713/

Review app deployed to https://teaching-vacancies-review-pr-2975.london.cloudapps.digital


Delete useless file : docs.md

What is this PR for?
Since we changed documentation viewing method after 0.5.0 release, we don't need docs.md file anymore.
What type of PR is it?
Improvement
Todos

[x] - delete docs.md file

Is there a relevant Jira issue?
No.
How should this be tested?
You may find docs.html in Zeppelin-0.5.0 documentation section, but after this version, there is no usage this file.
Screenshots (if appropriate)
Questions:

Does the licenses files need update? No
Is there breaking changes for older versions? No
Does this needs documentation? No

LGTM. Merge if there're no more discussions


Crashes on Forge 1.20.1 with Sinytra Connector

Hi, love your mod! I wanted to use it in my forge pack with Sinytra Connector but it crashes on start up.
debug.log
latest.log
copypastedlogfromprism.txt

Hey, thanks for the compliment!
The mod is made for fabric, so it won't work for forge unfortunately.
I currently have no plans on porting the mod to forge, but I might look into it in the future.
Greetings,
Retro

Is it okay if you can at least try and see if you can get it to work with sinytra connector? It let's you okay fabric mods on forge and it usually works out of the box in most cases. Sometimes it only need minor adjustments to get it to work.

Ah I see! I will have a look into that, however I expect to be busy this week, so it may take some time.

I have located the issue, and a new version which should be compatible should be available on both CurseForge and Modrinth soon!
Greetings,
Retro


AnyUrl doesn't accept file URLs with implicit host

Initial Checks

[X] I have searched GitHub for a duplicate issue and I'm sure this is something new
[X] I have searched Google & StackOverflow for a solution and couldn't find anything
[X] I have read and followed the docs and still think this is a bug
[X] I am confident that the issue is with pydantic (not my code, or another library in the ecosystem like FastAPI or mypy)

Description
https://github.com/pydantic/pydantic/issues/1983 appear again.
Example Code
from pydantic import AnyUrl, FileUrl
from pydantic.main import BaseModel, Field

class MoveSharedUrl(AnyUrl):
    allowed_schemes=['smb', 'ftp','file' ]

class Test(BaseModel):
    url: MoveSharedUrl

Test(url = "file:///aaa/bbb/serverFile/recvFiles/")
Test(url= "smb://<IP_ADDRESS>/serverFile/recvFiles/")

Python, Pydantic & OS Version
pydantic version: 1.10.11
            pydantic compiled: True
                 install path: C:\Python311\Lib\site-packages\pydantic
               python version: 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]
                     platform: Windows-10-10.0.19044-SP0
     optional deps. installed: ['devtools', 'dotenv', 'email-validator', 'typing-extensions']

Affected Components

[ ] Compatibility between releases
[X] Data validation/parsing
[ ] Data serialization - .dict() and .json()
[ ] JSON Schema
[ ] Dataclasses
[ ] Model Config
[X] Field Types - adding or changing a particular data type
[ ] Function validation decorator
[ ] Generic Models
[ ] Other Model behaviour - construct(), pickling, private attributes, ORM mode
[ ] Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.

Selected Assignee: @hramezani

Thanks @honglei for reporting this 🙏
As you know, Pydantic V2 is out and we are not working actively on Pydantic V1.
We can consider an easy patch for fixing this problem on V1 if you would like to prepare the patch.
BTW, the working example on V2 is:
from typing_extensions import Annotated

from pydantic import AnyUrl, UrlConstraints
from pydantic.main import BaseModel

class Test(BaseModel):
    url: Annotated[AnyUrl, UrlConstraints(allowed_schemes=['smb', 'ftp','file'])]

Test(url = "file:///aaa/bbb/serverFile/recvFiles/")
Test(url= "smb://<IP_ADDRESS>/serverFile/recvFiles/")

We’re no longer actively developing Pydantic V1 (although it will continue to receive security fixes for the next year or so), so we’re closing issues solely related to V1. If you really think this shouldn’t be closed, please comment or create a new issue 🚀.


ETT data

See https://drive.google.com/drive/folders/1Gv1MXjLo5bLGep4bsqDyaNMI2oQC9GH2 for data used in SCINet https://github.com/cure-lab/SCINet which is performing well, seemingly, at https://paperswithcode.com/sota/time-series-forecasting-on-etth1-24

The Electricity Transformer Temperature (ETT) is a crucial indicator in the electric power long-term deployment. This dataset consists of 2 years data from two separated counties in China. To explore the granularity on the Long sequence time-series forecasting (LSTF) problem, different subsets are created, {ETTh1, ETTh2} for 1-hour-level and ETTm1 for 15-minutes-level. Each data point consists of the target value ”oil temperature” and 6 power load features. The train/val/test is 12/4/4 months.


Wolicheng.WoMic version 4.7

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?

[x] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

@OfficialEsco i doubt they will DisplayVersion and Publisher beacause it dosent seems like devs are very active on the project, but i tried to email them anyway lets see


OCPQE-22219 add test jobs definition for 4.16 release

Select prow jobs for 4.16 nightly/stable from automated release job pool
https://github.com/openshift/release/blob/master/ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.16__automated-release.yaml
https://github.com/openshift/release/blob/master/ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.16__automated-release-stable-4.16-upgrade-from-stable-4.16.yaml
/assign @liangxia @jianzhangbjz @jhou1 @chengzhang1016
Can you review and comment. TODO: enable job controller job when this PR is merged.

/lgtm

/approve


Rancher CLI for go-prompt

Rancher cli is currently using go-prompt to auto prompt its command. Could you add it to the list of your repo? :) Again thanks for this awesome library

Thanks! I added.


broken on gitpods

Chrome devtools

gz#4419

Hi @gengjiawen
Thanks for reaching out
Can you please share some more details?
Are you not able to use Tabnine at all? Did it use to work before or is it the first time you're using Tabnine?

Used to work. Not sure why this occured.

@gengjiawen - can you please update to the latest version and try again ?

@gengjiawen - can you please update to the latest version and try again ?

Fixed. Thanks. Please consider test it on gitpods before release.


Add to Ecosystem

Please consider submitting a PR to add this lib to Ecosystem doc.

@gaearon will do!

Closed by https://github.com/rackt/redux/pull/713


runtime: scavengelist disabled on 64k page size systems (ppc64)

I am sorry for the imprecise nature of this issue. The symptoms are random SIGSEGV's after processes have been running for several minutes. The most visible effect of this is go test std will segfault half way through the test run, making it impossible to complete a CI build.
Last year @mwhudson and I saw a similar issue with gccgo-4.9 compiled programs which was eventually traced to the page size of the machine, most ppc64 machines use 64k pages, being out of sync with the page size of that the runtime uses.
In tracing some processes today I have found occurences of munmap(2) and madvise(2) calls that are not using multiples of 64k
munmap(0x3fff82b40000, 26532)           = 0

More worrying, I've found examples that don't use a multiple of 4096 bytes!
munmap(0x3fff80e10000, 102680)          = 0

/cc @randall77

Linux, right?
The runtime currently uses 8K pages, so everything should be a multiple of
8K even if the OS pages are 4K.
I don't see how an madvise call can possibly get a non-multiple of 8K.  It
is only called from sysUnused, and that is only called with a second
argument that is ... << _PageShift.
There are a few places where munmap uses non-8K multiples.  It's called
from sysFree, and sysFree (and sysAlloc, for that matter) is called from
some places with sizes that aren't necessarily page sizes.  Maybe
sysAlloc/sysFree should round to the page size before calling mmap/munmap?
It may not matter, but I'm not sure what mmap/munmap do with partial pages.
You might want to try setting _PageShift to 16 and see if that helps
ppc64le.  We probably don't want to leave that there, but it would be a
good experiment.
On Tue, Feb 24, 2015 at 9:39 PM, Dave Cheney<EMAIL_ADDRESS>wrote:

I am sorry for the imprecise nature of this issue. The symptoms are random
SIGSEGV's after processes have been running for several minutes. The most
visible effect of this is go test std will segfault half way through the
test run, making it impossible to complete a CI build.
Last year @mwhudson https://github.com/mwhudson and I saw a similar
issue with gccgo-4.9 compiled programs which was eventually traced to the
page size of the machine, most ppc64 machines 64k pages, being out of sync
with the page size of that the runtime uses.
In tracing some processes today I have found occurences of munmap(2) and
madvise(2) calls that are not using multiples of 64k
munmap(0x3fff82b40000, 26532)           = 0
More worrying, I've found examples that don't use a multiple of 4096 bytes!
munmap(0x3fff80e10000, 102680)          = 0
/cc @randall77 https://github.com/randall77
—
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/9993.

Yes, linux/ppc64le.
I'll try the page shift hack and also out some asserts in to check
everything is aligned to the page shift.
On 25 Feb 2015 17:22, "Keith Randall"<EMAIL_ADDRESS>wrote:

Linux, right?
The runtime currently uses 8K pages, so everything should be a multiple of
8K even if the OS pages are 4K.
I don't see how an madvise call can possibly get a non-multiple of 8K. It
is only called from sysUnused, and that is only called with a second
argument that is ... << _PageShift.
There are a few places where munmap uses non-8K multiples. It's called
from sysFree, and sysFree (and sysAlloc, for that matter) is called from
some places with sizes that aren't necessarily page sizes. Maybe
sysAlloc/sysFree should round to the page size before calling mmap/munmap?
It may not matter, but I'm not sure what mmap/munmap do with partial pages.
You might want to try setting _PageShift to 16 and see if that helps
ppc64le. We probably don't want to leave that there, but it would be a
good experiment.
On Tue, Feb 24, 2015 at 9:39 PM, Dave Cheney<EMAIL_ADDRESS>wrote:

I am sorry for the imprecise nature of this issue. The symptoms are
random
SIGSEGV's after processes have been running for several minutes. The most
visible effect of this is go test std will segfault half way through the
test run, making it impossible to complete a CI build.
Last year @mwhudson https://github.com/mwhudson and I saw a similar
issue with gccgo-4.9 compiled programs which was eventually traced to the
page size of the machine, most ppc64 machines 64k pages, being out of
sync
with the page size of that the runtime uses.
In tracing some processes today I have found occurences of munmap(2) and
madvise(2) calls that are not using multiples of 64k
munmap(0x3fff82b40000, 26532) = 0
More worrying, I've found examples that don't use a multiple of 4096
bytes!
munmap(0x3fff80e10000, 102680) = 0
/cc @randall77 https://github.com/randall77
—
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/9993.

—
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/9993#issuecomment-75911749.

The plot thickens. With some asserts in place (and _PageShift still 13), someone passed a request for 1439992 bytes to sysAlloc
##### Building packages and commands for linux/ppc64le.
unaligned 1439992
panic: unaligned memory access
fatal error: panic on system stack

runtime stack:
runtime.throw(0x39cfd0, 0x15)
        /home/ubuntu/go/src/runtime/panic.go:511 +0xc0 fp=0x3fffce49e848 sp=0x3fffce49e830
runtime.gopanic(0x2ad460, 0xc208002000)
        /home/ubuntu/go/src/runtime/panic.go:338 +0xec fp=0x3fffce49e8c8 sp=0x3fffce49e848
runtime.checklen(0x15f8f8)
        /home/ubuntu/go/src/runtime/mem_linux.go:24 +0xe0 fp=0x3fffce49e900 sp=0x3fffce49e8c8
runtime.sysAlloc(0x15f8f8, 0x512550, 0x3fffce49e9e8)
        /home/ubuntu/go/src/runtime/mem_linux.go:62 +0x14 fp=0x3fffce49e938 sp=0x3fffce49e900
runtime.stkbucket(0x1, 0x120, 0x3fffce49ea28, 0x5, 0x20, 0xf0d01, 0x3fffce49ea60)
        /home/ubuntu/go/src/runtime/mprof.go:144 +0x60 fp=0x3fffce49e9e0 sp=0x3fffce49e938
runtime.mProf_Malloc(0xc208000000, 0x120)
        /home/ubuntu/go/src/runtime/mprof.go:237 +0xec fp=0x3fffce49eb60 sp=0x3fffce49e9e0
runtime.profilealloc(0x4e67e0, 0xc208000000, 0x120)
        /home/ubuntu/go/src/runtime/malloc.go:749 +0x120 fp=0x3fffce49eb88 sp=0x3fffce49eb60
runtime.mallocgc(0x120, 0x351fa0, 0x0, 0x2adc60)
        /home/ubuntu/go/src/runtime/malloc.go:654 +0x548 fp=0x3fffce49ec58 sp=0x3fffce49eb88
runtime.newobject(0x351fa0, 0x4e5ea0)
        /home/ubuntu/go/src/runtime/malloc.go:698 +0x80 fp=0x3fffce49ec80 sp=0x3fffce49ec58
runtime.malg(0x8000, 0x4e61a0)
        /home/ubuntu/go/src/runtime/proc1.go:1992 +0x30 fp=0x3fffce49ecb8 sp=0x3fffce49ec80
runtime.mpreinit(0x4e67e0)
        /home/ubuntu/go/src/runtime/os1_linux.go:169 +0x34 fp=0x3fffce49ecd0 sp=0x3fffce49ecb8
runtime.mcommoninit(0x4e67e0)
        /home/ubuntu/go/src/runtime/proc1.go:115 +0x118 fp=0x3fffce49ecf8 sp=0x3fffce49ecd0
runtime.schedinit()
        /home/ubuntu/go/src/runtime/proc1.go:57 +0x98 fp=0x3fffce49ed40 sp=0x3fffce49ecf8
runtime.rt0_go(0x3fffce49f702, 0x3fffce49f70b, 0x3fffce49f70c, 0x3fffce49f70f, 0x3fffce49f713, 0x0, 0x3fffce49f717, 0x3fffce49f71e, 0x3fffce49f730, 0x3fffce49f740, ...)
        /home/ubuntu/go/src/runtime/asm_ppc64x.s:69 +0xa4 fp=0x3fffce49ed80 sp=0x3fffce49ed40

Instrumenting sysAlloc may have been a bit aggressive, but this assertion in sysFree is very worrying -- it's not even aligned to an 8k page, and the remaining 7.3k will be discarded by the operating system and probably zeroed when we touch it next.
##### Building packages and commands.                                                                                [74/9576]
runtime
# runtime
unaligned 58152
panic: unaligned memory access
fatal error: panic on system stack

runtime stack:
runtime.throw(0x391570, 0x15)
        /home/ubuntu/go/src/runtime/panic.go:511 +0xc0 fp=0x3fffd0feb108 sp=0x3fffd0feb0f0
runtime.gopanic(0x2f7b00, 0xc20b6e1f60)
        /home/ubuntu/go/src/runtime/panic.go:338 +0xec fp=0x3fffd0feb188 sp=0x3fffd0feb108
runtime.checklen(0xe328)
        /home/ubuntu/go/src/runtime/mem_linux.go:24 +0xe0 fp=0x3fffd0feb1c0 sp=0x3fffd0feb188
runtime.sysFree(0x3fff7b930000, 0xe328, 0x513700)
        /home/ubuntu/go/src/runtime/mem_linux.go:88 +0x2c fp=0x3fffd0feb1e0 sp=0x3fffd0feb1c0
runtime.gcCopySpans()
        /home/ubuntu/go/src/runtime/mgc.go:607 +0xd8 fp=0x3fffd0feb208 sp=0x3fffd0feb1e0
runtime.gcMark(0xb831317ec7648)
        /home/ubuntu/go/src/runtime/mgc.go:452 +0x8c fp=0x3fffd0feb348 sp=0x3fffd0feb208
runtime.gc.func2()
        /home/ubuntu/go/src/runtime/mgc.go:371 +0x40 fp=0x3fffd0feb380 sp=0x3fffd0feb348
runtime.systemstack(0x4d9030)
        /home/ubuntu/go/src/runtime/asm_ppc64x.s:236 +0x90 fp=0x3fffd0feb388 sp=0x3fffd0feb380
runtime.mstart()
        /home/ubuntu/go/src/runtime/proc1.go:682 fp=0x3fffd0feb388 sp=0x3fffd0feb388

but this assertion in sysFree is very worrying -- it's not even aligned to an 8k page

what's wrong with it?
it is ok to mmap/munmap less than a page

You might want to try setting _PageShift to 16

_PageSize is only for heap and is an internal detail
don't change it for the sake of OS
if we need to tune something then it is _PhysPageSize

My understanding is mmap, munmap etc always work on page sized quantities.
The result is if you madvise DONT_NEED a 4k section on a 64k system, the
kernel will round this up to 64k and drop 60 odd kB of data that the gc
thinks is active.
We hit exactly this problem using gccgo on 64kb systems last year.
Sorry for the brevity (phone)
On 26 Feb 2015 00:25, "Dmitry Vyukov"<EMAIL_ADDRESS>wrote:

but this assertion in sysFree is very worrying -- it's not even aligned to
an 8k page
what's wrong with it?
it is ok to mmap/munmap less than a page
You might want to try setting _PageShift to 16
_PageSize is only for heap and is an internal detail
don't change it for the sake of OS
if we need to tune something then it is _PhysPageSize
—
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/9993#issuecomment-75958714.

Is the problem only with madvise or with mmap/munmap as well?

We saw the issue with the scavenger madvising unused 4k pages of the heap,
and the kernel rounding that up to 64k pages.
On Thu, Feb 26, 2015 at 3:21 PM, Dmitry Vyukov<EMAIL_ADDRESS>wrote:

Is the problem only with madvise or with mmap/munmap as well?
—
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/9993#issuecomment-76120132.

I see.
As always man page is super helpful, it talks about everything except the important parts:
The Linux implementation requires that the address addr be page-aligned, and allows length to be zero.  If there are some parts of  the  specified  address  range that are not mapped, the Linux version of madvise() ignores them and applies the call to the rest (but returns ENOMEM from the system call, as it should).
If we saw that it happens in reality, then I guess we need to fix it.
However, mmap/munmap with unaligned size are OK, don't touch them. Also don't change _PageSize and don't use it outside of heap code. _PhysPageSize is what you need.

I believe the problem is here
https://github.com/golang/go/blob/master/src/runtime/mheap.go#L731
Attached is the patch we sent to gccgo last year, but reading over it I am unsure if it is safe to use here. I'm not sure it is safe to widen the range of memory freed as there is no signal to the GC that we've freed a larger area than the scavenge free list wanted us to do.
diff -r 3a53301d24d7 libgo/runtime/mheap.c
--- a/libgo/runtime/mheap.c	Tue Apr 22 16:43:35 2014 -0700
+++ b/libgo/runtime/mheap.c	Thu Apr 24 21:18:35 2014 -0700
@@ -387,7 +387,7 @@
 static uintptr
 scavengelist(MSpan *list, uint64 now, uint64 limit)
 {
-	uintptr released, sumreleased;
+	uintptr released, sumreleased, start, end, pagesize;
 	MSpan *s;

 	if(runtime_MSpanList_IsEmpty(list))
@@ -400,7 +400,17 @@
 			mstats.heap_released += released;
 			sumreleased += released;
 			s->npreleased = s->npages;
-			runtime_SysUnused((void*)(s->start << PageShift), s->npages << PageShift);
+
+			start = s->start << PageShift;
+			end = start + (s->npages << PageShift);
+
+			// Round start up and end down to ensure we
+			// are acting on entire pages.
+			pagesize = getpagesize();
+			start = ROUND(start, pagesize);
+			end &= ~(pagesize - 1);
+			if(end > start)
+				runtime_SysUnused((void*)start, end - start);
 		}
 	}
 	return sumreleased;

not sure it is safe to widen the range of memory freed

as far as I see, you only shrink the region from both ends
we will need to ensure that _PageSize <= _PhysPageSize on platforms where SysUnused is not a no-op
also I suspect that such change can make scavenger less efficient on platforms with _PhysPageSize > _PageSize, because even consecutive unused pages can end up being not madvised (though, of course, it is better than crashing)

Hey all, we're interested in creating a Go Mobile library, but this issue is currently a blocker for us. On arm64 devices this issue means that the Go GC will increasingly own memory within an iOS (and presumably Android) process and never give it back, which can eventually get to the point where the memory available to the rest of the process is too small to run normal operations and OS will kill the app due to memory pressure.
More discussion here: https://groups.google.com/forum/#!topic/golang-nuts/-7ehtvFzJj0
Thanks!

As a first step, we can certainly make scavengelist or sysUnused round the boundaries in to _PhysPageSize. That will at least help with scavenging memory that backed large allocations and, as free spans get coalesced, it will eventually release coalesced spans that are sufficiently large (at least most of the time).
I have some ideas for how to make the scavenger work well in environments where the heap's granularity is finer than the system's granularity. Our current heuristic of releasing a free span only once no page in that span has been used for 5 minutes, in addition to being ridiculously conservative, is a good way to turn the heap in to Swiss cheese. To a first order, the age of a particular span is irrelevant (there is some weak temporal locality). What matters is not which spans we release, but how many we release, so we could chose how many using a temporal heuristic, but chose which spans to release using a spacial heuristic. If we bias this toward large ranges, we'll free memory much more effectively on platforms with large physical pages.

@davecheney, would you mind stress testing CL 22066 on a PPC or ARM64? Based on your original post, it sounds like go test std is sufficient to trigger the problem. I suspect setting GODEBUG=scavenge=1 should stress it much harder.

Thanks Austin. I've asked @mwhudson to take a look as I'm travelling ATM.
On Fri, 15 Apr 2016, 06:17 Austin Clements<EMAIL_ADDRESS>wrote:

@davecheney https://github.com/davecheney, would you mind stress
testing CL 22066 on a PPC or ARM64? Based on your original post, it sounds
like go test std is sufficient to trigger the problem. I suspect setting
GODEBUG=scavenge=1 should stress it much harder.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/9993#issuecomment-210131275

I'll test this. To be clear, it should be a system where "getconf PAGESIZE" says 65536? (The arm64 machine I have access to appears to have 4k pages, but the ppc64 one is 64k)

Thanks! Yes, in that case you should stress test on the ppc64. Though it would be good to make sure it passes all.bash on the arm64 just to test the case where physical pages are smaller than sys.PhysPageSize.

GODEBUG=scavenge=1 is quite slow :-)

The tests didn't pass cleanly with scavenge=1 but I sort of get the feeling that's not the fault of that CL: https://gist.github.com/mwhudson/21badf1d67536a9fc3719d19cc5865b6
I'm running the same test on master now.

GODEBUG=scavenge=1 is quite slow :-)

Yeah. :) Unfortunately, that also turns the forced GC rate way up, which is probably where a lot of the slowdown is coming from.
I agree that those failures don't look likely to be related to this CL.
As a baseline, I tried GODEBUG=scavenge=1 go test std and just go test std on linux/amd64 on master (12e3b18, the branch point of CL 22066) and in both cases got one failure:
--- FAIL: TestWaitGroupMisuse2 (2.09s)
        waitgroup_test.go:114: Should panic
        waitgroup_test.go:84: Unexpected panic: <nil>
FAIL
FAIL    sync    2.398s

Also, with scavenge=1, math/big took 297 seconds, so I'm not too surprised it timed out on arm64 and ppc64. The other failures in the gist you posted all looked networking-related.

The test run on master had the same failures so yeah. Test passed afaict.

Thanks! I'll push these in a little bit.


Sharedflows Create Breaking update

Hello,
A few months ago I worked on setting up a Gitlab CI/CD pipeline to create and deploy an Apigee shared flow successfully. Today, this pipeline is failing.
I believe this feature has been updated between versions. Locally I have version v1.122 which works with the commands I'm using in the pipeline.
For context, I'm using the following command to install on my docker container in the pipeline, which downloads the latest version: v1.124:
curl -L https://raw.githubusercontent.com/apigee/apigeecli/main/downloadLatest.sh | sh -
The command that used to save my shared flow (but still works locally) no longer works on the pipeline:
apigeecli sharedflows create -f ./sharedflowbundle -n $SHAREDFLOW_NAME -o afrl-datahub -t $token
These are the options that are listed now:
Usage:
  apigeecli sharedflows create [command]
Available Commands:
  bundle      Creates a sharedflow in an Apigee Org
  github      Creates a sharedflow from a GitHub rep

I'm using GitLab and not github so that option doesn't apply. I'm looking into the command to create a bundle, which I'm assuming is a zip file of the ./sharedflowbundle directory.
Alternatively, is there a way to install a specific version of apigeecli? Any help will be apprecaited.

I realized that I just needed to add the command bundle to it and I got it to work:
- apigeecli sharedflows create bundle -f ./sharedflowbundle -o afrl-datahub -t $token -n $SHAREDFLOW_NAME
sorry!


Split Community page into two tabs - Tags and Users

Popular tags and popular users

How do you define a popular user? This is not a metric that has been yet discussed.

Most followed?

OK, most followed is an easy metric that doesn't require any new infra.


pppos-client: Refactor modem component (IDFGH-4671)

The idea of this refactoring is to centralize all methods (& handle) which are defined in specifications (GPRS, 3GPP TS 27.007 & V.25TER) in the esp_modem_dce_service files (.c & .h). That can remove duplicate lines.
This PR hasn't been tested on all modems, but a similar refactor has been tested on SIM7000. It can be considered a an proposition which needs to be fully reviewed.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks for your contribution.

Can confirm these changes also works on the BG96 and BG95 modems.
I am much in favour of this work towards a generic 3GPP driver. It really would help clean up the driver code and thus make it more reliable.

@jonathandreyer Thanks for your contribution again, have closed just now.


[plugin system] Add a utility function (addFontMetrics) to extend fontMetrics

Store current font metrics object inside a metricMap variable.
Also export a function called addFontMetrics from fontMetrics.js and expose it on the main katex object so plugins can add their own font metrics to library in run time.
Checkout #762 and #729

Please take a look at persian-katex-plugin this is a complete implementation of Persian formulas for KaTeX using this branch.
You can also find out how I generated the metrics using generateDefaultMetrics.js and ultimately extractMetricsFromTTF.js.
I'm not sure how to add proper tests because these test need to add new fonts and metrics to the KaTeX object.
I also would be very happy if I could add the generateMetrcis.js as a utility function to the main library.

@HosseinAgha I had a look at your plugin.  I'm still a little confused about which classes are being set on the spans containing characters from the Vazir font.  In katex.less we define classes like .texttt that result in the font-family being set to KaTeX_Typewriter;.  How does this work with your plugin?

Answering my own question.  I checked out your repo and ran the test app.  It looks like Vazir-Regular is being set as one of the classes on the span containing the number ۹.  This is super cool!  I'm happy to see it was too much work.  I'll spend some more time reviewing this PR this evening and get back to you regarding your question about how to test this.

Thank you @kevinbarabash for your review.
I'll start fixing the issues tonight.
As there is a significant time zone difference between us I think you will get them tomorrow :)

@kevinbarabash Is it OK if we warn the user if one of the default KaTeX metrics was being overwritten?
something like (in setFontMetrics):
if (metricMap.hasOwnProperty(fontName)) {
  console.warn(`metrics for ${fontName} is being overwritten by a plugin`)
}

@HosseinAgha thanks picking this up again.  There are some changes described in my comments above that I think would be helpful in making this more robust and easier for other people to use it.  I approved this PR though b/c I want to get this on to master so that we can continue to iterate on it.

I added some tests as you requested and fix the conflicts I think we are ready to merge.
@kevinbarabash Thank you for your review.

@HosseinAgha thanks for adding tests.


feature(federation): preview items to ingest, code refactor

Issue
https://gravitee.atlassian.net/browse/APIM-5150
https://gravitee.atlassian.net/browse/APIM-5095
https://gravitee.atlassian.net/browse/APIM-5091
Description

Additional context

📚  View the storybook of this branch here

should go to 4.4, closing


Fix coverage drop on param_mapping

Description
The coverage for the param mapping file dropped due to an additional check:
        temp[path[-1]] = value
        if path[-1] == "intent":
            del temp["pipeline"]
        if path[-1] == "transformer":
            del temp["parameters"]

@jichaogp I think this was introduced in this commit https://github.com/georgianpartners/foreshadow/commit/6a5aa870103911185df26cfb647297ecee5196b1 . Could you add an example test case to cover these lines?

Will do.

I had to try-catch the deletion of the pipeline key when implementing the command line driver as the standard workflow for performing optimization includes removing the overridden pipeline from json (and will throw a KeyError here). When making these test cases you should probably either try / catch that line yourself or wait until PR #49 is merged else someone will have to come back to this again.

As a follow up on this issue, I noticed that the param_mapping.py as a whole was not explicitly covered, it would be good to add unit tests for the methods in this file.s