Az/apicenter2

Related command

Description

Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change
[Component Name 2] az command b: Add some customer-facing feature

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

[ ] I adhere to the Error Handling Guidelines.

Thank you for your contribution! We will review the pull request and get back to you soon.

@arpishahmsft recommend create an extension which is much flexible to release.

[like]  Arpi Shah reacted to your message:

From: kai ru @.>
Sent: Monday, October 30, 2023 3:39:19 AM
To: Azure/azure-cli @.>
Cc: Arpi Shah @.>; Mention @.>
Subject: Re: [Azure/azure-cli] Az/apicenter2 (PR #27712)
@arpishahmsfthttps://github.com/arpishahmsft recommend create an extension which is much flexible to release.
—
Reply to this email directly, view it on GitHubhttps://github.com/Azure/azure-cli/pull/27712#issuecomment-1784426221, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A4WSSFVSCUPSZZYZXHU6BSTYB4OOPAVCNFSM6AAAAAA6TOUEIKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBUGQZDMMRSGE.
You are receiving this because you were mentioned.Message ID: @.***>

Please fix CI issues.

/azp run


xcor_gsea() missing parM() function

I've been working on implementing PATH, and have been running into errors with the xcor_gsea function. To troubleshoot, I have worked through the "PATH analysis of glioblastoma" vignette with the data available here, on github, but have found the same error when trying to run xcor_gsea(). The error text is:
Error in parM(gene.data, weight.matrix) : could not find function "parM"
I cannot find what package this function would belong to, so I'm unsure what could have a version issue or other troubleshooting need.

Hi, thanks for flagging this issue. I've just pushed code for the parM
function, which was not in the repo. Hopefully that solves the problem. If
not, let me know and I'll see what I can do to fix it.
On Fri, Jun 9, 2023 at 2:57 PM Sadie VanHorn @.***>
wrote:

I've been working on implementing PATH, and have been running into errors
with the xcor_gsea function. To troubleshoot, I have worked through the
"PATH analysis of glioblastoma" vignette with the data available here, on
github, but have found the same error when trying to run xcor_gsea(). The
error text is:
Error in parM(gene.data, weight.matrix) : could not find function "parM"
I cannot find what package this function would belong to, so I'm unsure
what could have a version issue or other troubleshooting need.
—
Reply to this email directly, view it on GitHub
https://github.com/landau-lab/PATH/issues/1, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB2IIY4Z3UDWDKVXQ5AE74DXKNWZVANCNFSM6AAAAAAZBADDYM
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

It's close - one new error with the parM() function:
Error in select(., Z) : unused argument (Z)
Again, this is just in the vignette, copied directly from github.

I think the select() function call inside xcor_gsea() must've been masked. Specified that it is from the package dplyr now.

great! It is giving the correct output - Thanks for your help 😃


MD-WIDGETS: Allow extra space in the markdown file.

Description
I am often nicely formatting my raw markdown files. So for example I might have:
%% EmptyLine

1. This is Question 1. The following four options belong to the same group.
   Only one option can be selected.

%% Radio(Question 1,1,Option 1 This is a long option that may wrap to
the next line if it needs to. When wrapping it does so indented to
align the paragraph.

%% Radio(Question 1,2,Option 2)

%% Radio(Question 1,3,Option 3)

%% Radio(Question 1,4,Option 4)

%% EmptyLine

This is rendered poorly as:

The error for Option 2 is in the MD - missing ) for Option 1.

Close as completed and merged.


Feature Request: Sparse ILU Factorization

Are there plans in the near future to include Sparse ILU Factorization routines?
If this is a good first issue, I would be interested in contributing.

I think no one's working on it. We welcome your contributions 😄
https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.spilu.html


Some local integration tests failing

Bug description
Several integration tests are failing on master when run in local/native mode. e.g.
crwilson@crwilson:~/go/src/istio.io/istio/tests/integration$ go test ./... 2>&1 | grep -e "^FAIL\|^ok\|^\?"
ok  	istio.io/istio/tests/integration/conformance	(cached)
ok  	istio.io/istio/tests/integration/framework	(cached)
?   	istio.io/istio/tests/integration/framework/mycomponent	[no test files]
ok  	istio.io/istio/tests/integration/galley	(cached)
ok  	istio.io/istio/tests/integration/istioctl	0.847s
ok  	istio.io/istio/tests/integration/mixer	(cached)
ok  	istio.io/istio/tests/integration/mixer/k8s	0.442s
ok  	istio.io/istio/tests/integration/mixer/policy	2.375s
ok  	istio.io/istio/tests/integration/mixer/telemetry/logs	1.557s
ok  	istio.io/istio/tests/integration/mixer/telemetry/metrics	1.967s
ok  	istio.io/istio/tests/integration/pilot	(cached)
ok  	istio.io/istio/tests/integration/pilot/interception	0.633s
ok  	istio.io/istio/tests/integration/pilot/locality	(cached)
?   	istio.io/istio/tests/integration/pilot/outboundtrafficpolicy	[no test files]
FAIL
FAIL	istio.io/istio/tests/integration/pilot/outboundtrafficpolicy/allowany	4.888s
ok  	istio.io/istio/tests/integration/pilot/outboundtrafficpolicy/registryonly	1.642s
ok  	istio.io/istio/tests/integration/pilot/trafficshifting	1.123s
ok  	istio.io/istio/tests/integration/qualification	(cached)
?   	istio.io/istio/tests/integration/security/authn	[no test files]
ok  	istio.io/istio/tests/integration/security/authn/jwt	0.827s
FAIL
FAIL	istio.io/istio/tests/integration/security/authn/permissive	4.622s
ok  	istio.io/istio/tests/integration/security/citadel	0.738s
ok  	istio.io/istio/tests/integration/security/healthcheck	0.722s
?   	istio.io/istio/tests/integration/security/rbac/util	[no test files]
ok  	istio.io/istio/tests/integration/security/rbac/v1/group	1.290s
ok  	istio.io/istio/tests/integration/security/rbac/v1/grpc	1.756s
ok  	istio.io/istio/tests/integration/security/rbac/v1/path	2.040s
ok  	istio.io/istio/tests/integration/security/rbac/v2/basic	1.732s
ok  	istio.io/istio/tests/integration/security/rbac/v2/group	0.704s
FAIL
FAIL	istio.io/istio/tests/integration/security/reachability	4.345s
ok  	istio.io/istio/tests/integration/security/sds_citadel_flow	(cached)
ok  	istio.io/istio/tests/integration/security/sds_ingress/multi_mtls_gateway_invalid_secret	(cached)
ok  	istio.io/istio/tests/integration/security/sds_ingress/multi_tls_gateway_invalid_secret	(cached)
ok  	istio.io/istio/tests/integration/security/sds_ingress/multiple_mtls_gateway	(cached)
ok  	istio.io/istio/tests/integration/security/sds_ingress/multiple_tls_gateway	(cached)
ok  	istio.io/istio/tests/integration/security/sds_ingress/single_mtls_gateway_compound_secret	(cached)
ok  	istio.io/istio/tests/integration/security/sds_ingress/single_mtls_gateway_separate_secret	(cached)
ok  	istio.io/istio/tests/integration/security/sds_ingress/single_tls_gateway	(cached)
?   	istio.io/istio/tests/integration/security/sds_ingress/util	[no test files]
ok  	istio.io/istio/tests/integration/security/sds_vault_flow	(cached)
?   	istio.io/istio/tests/integration/security/util	[no test files]
?   	istio.io/istio/tests/integration/security/util/connection	[no test files]
?   	istio.io/istio/tests/integration/security/util/secret	[no test files]
?   	istio.io/istio/tests/integration/telemetry/tracing	[no test files]
ok  	istio.io/istio/tests/integration/telemetry/tracing/clienttracing	0.697s
ok  	istio.io/istio/tests/integration/telemetry/tracing/servertracing	0.434s

I've confirmed that these failures were introduced in https://github.com/istio/istio/pull/14614. Taking a quick look at the diff, my guess is that we disabled some native-specific skips & workarounds that are leading to the failures appearing.
Affected product area (please put an X in all that apply)
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[X] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Expected behavior
Steps to reproduce the bug
Tests work fine in 1620478d1 but some fail in 303ed43df and beyond.
Full test log

The first failure is fixable by re-adding RequireEnvironment(environment.Kube). in traffic_allow_any_test.go. The other two seem to have the same error cropping up:
* Error response from daemon: Failed to Setup IP tables: Unable to enable SKIP DNAT rule:  (iptables failed: iptables --wait -t nat -I DOCKER -i br-d5efbdc45f6a -j RETURN: iptables v1.8.2 (nf_tables): Chain 'DOCKER' does not exist
        (exit status 1))

@howardjohn you seemed to be investigating some errors ... is it possible there is some overlap here?

The first failure is fixable by re-adding RequireEnvironment(environment.Kube).

That isn't a real fix that is just skipping the test
I have seen the same error in a different context before. AFAIK it was caused by an update to gLinux. A reboot should fix it @sushicw

After rebooting, tests all pass on master. Looks like I was chasing a red herring. Thanks @nmittler  and @howardjohn !


Suggestion for dynamic projections

Hi,
I found a use case where I need to have projections made dynamically, basically just by specifying an arbitrary subset of fields that I want. My solution was to dynamically create new Model and pass it to the project() function. I was wondering if you would accept something like this as a PR? Would it be useful for Beanie? Or would you suggest another way of doing this? Thanks!
Right now it I am using it like this:
Product.find_many().project(create_submodel(Product, fields=["name", "description"]))

If made as part of Beanie, could it be like this?
Product.find_many().project(fields=["name", "description"])

PoC source with an example:
from typing import List, Union, Dict, Type, Optional
from pydantic import create_model, BaseModel

def create_submodel(
    model_type: Type[BaseModel], fields: List[Union[str, Dict]]
) -> Type[BaseModel]:
    """
    Creates a new model based on specified model_type with only a subset of it's fields.

    :param model_type:
    :param fields:
    :return:
    """
    model_fields = model_type.__fields__
    model_field_names = set(model_fields.keys())
    submodel_fields = {}
    for field in fields:

        if isinstance(field, str):

            if field not in model_field_names:
                raise ValueError(
                    f"field '{field}' not defined in model {model_type.__name__}"
                )

            submodel_fields[field] = (
                model_fields[field].type_,
                model_fields[field].field_info,
            )

        elif isinstance(field, dict):
            field.values()
            field_name = list(field.keys())[0]

            if field_name not in model_field_names:
                raise ValueError(
                    f"field '{field_name}' not defined in model {model_type.__name__}"
                )

            submodel = create_submodel(
                model_fields[field_name].type_, field[field_name]
            )
            submodel_fields[field_name] = (submodel, model_fields[field_name].field_info)

    return create_model(f"{model_type.__name__}SubModel", **submodel_fields)

if __name__ == "__main__":

    class Category(BaseModel):
        name: str
        description: str

    class Product(BaseModel):
        name: str
        description: Optional[str] = None
        price: float
        category: Category

    ProductSubModel = create_submodel(
        Product, fields=["name", "price", {"category": ["name"]}]
    )

    product_with_less_info = ProductSubModel.parse_obj(
        {"name": "Banana", "price": 3.14, "category": {"name": "Fruit"}}
    )

    print(product_with_less_info.json(indent=2))

# RESULT:
# {
#   "name": "Banana",
#   "price": 3.14,
#   "category": {
#     "name": "Fruit"
#   }
# }

Hey @gethvi ,
This feature looks interesting.
I think it would be nicer to have smth like model factory (as you have in your example) instead of providing the list of the fields to the project method. In this case, it will be simpler to add additional API like:
from typing import Union, Set, Dict, Any

from pydantic import BaseModel

def cut_model(
        name: str,
        model: BaseModel,
        include: Union[Set, Dict[str, Any]],
        exclude: Union[Set, Dict[str, Any]]
) -> BaseModel:
    ...

I have to think more about the implementation and interface, but the idea is really good. Thank you very much.

Hello,
@gethvi I really like your solution, but it would also be interesting to handle use cases such as:
class B(BaseModel):
    z: int
    y: str = None

class A(BaseModel):
    x: str
    w: Dict[str, B]

C = create_submodel(A, fields=["x", {"w": {"foo": {"z"}}}])
results = A.find_many().project(projection_model=C)

then , results[0].dict() would look like:
{"x": "value_of_x", "w": {"foo": {"z": 1}}}

Your current implementation fails with:
    def create_submodel(
        model_type: Type[BaseModel], fields: List[Union[str, Dict]]
    ) -> Type[BaseModel]:
        """
        Creates a new model based on specified model_type with only a subset of it's fields.

        :param model_type:
        :param fields:
        :return:
        """
        model_fields = model_type.__fields__
        model_field_names = set(model_fields.keys())
        submodel_fields = {}
        for field in fields:

            if isinstance(field, str):

                if field not in model_field_names:
>                   raise ValueError(
                        f"field '{field}' not defined in model {model_type.__name__}"
                    )
E                   ValueError: field 'foo' not defined in model B

Do you think your solution could be adapted to fit this use case ? I didn't manage to do it by myself.
Thanks

#202
#204
i hope this suggestion can fix this 2 issuse too.

This is a bit old, but I'm running into the same need today.  I have a fairly complicated nested model I need to be able to pull arbitrary values from.  In native mongodb syntax you can do stuff like:
projection = {Model.name: 1, Model.config.nested.data: 1, Model.config.nested.settings: 1}

So honestly I'd love if you could use Beanie's project() method with a native mongo dict like that.
Building up a Pydantic model with so many sub-models is quite clumsy to have to do by hand.


Docker container hosting Shibboleth IdP crashes within 8 hours on hosted Docker container

(1) Run Docker container for Shibboleth IdP directly on Ubuntu16.04/VMWare
Run the 2nd separate Docker container for OpenLDAP on the same Ubuntu16.04/VMWare
Run the 3rd separate Docker container for MySQL on the same Ubuntu16.04/VMWare
Execute the command
docker run -p 8443:8443 --name idp1 --link ldap:ldap --link mysql:mysql idq/shibboleth-idp:001
Experimental result:
Docker container hosting Shibboleth IdP runs reliably on Ubuntu16.04/VMWare for more than 1 week
(2) Run Docker container for Shibboleth IdP on Ubuntu16.04/Docker container
Run the 2nd separate Docker container for OpenLDAP on the same Ubuntu16.04/Docker container
https://github.com/UniconLabs/dockerized-idp-testbed/tree/master/ldap
Run the 3rd separate Docker container for MySQL on the same Ubuntu16.04/Docker container
Execute the command
docker run -p 8443:8443 --name idp1 --link ldap:ldap --link mysql:mysql idq/shibboleth-idp:001
Experimental result:
Docker container hosting Shibboleth IdP crashes on Ubuntu16.04/Docker container within 8 hours. However, both the Docker container hosting OpenLDAP and the Docker container hosting MySQL run reliably on the same Ubuntu16.04/Docker container.
We have to re-execute the command to start the Docker container hosting Shibboleth IdP again
8443: 8443 host:container
Remarks:
(I) We use HAProxy to proxy internal port 8443 of Shibboleth IdP host to external port 443.
For example, IP address/Port of internal Shibboleth IdP host is <IP_ADDRESS>:8443
is proxies to
IP address/Port of external Shibboleth IdP host is <IP_ADDRESS>:443
(II) Once we re-start HAProxy, Docker container hosting Shibboleth IdP crashes immediately. However, other Docker containers which also rely on the same HAProxy run reliably.
https://en.wikipedia.org/wiki/HAProxy
https://github.com/UniconLabs/dockerized-idp-testbed/tree/master/httpd-proxy

@winstonhong,
Were you ever able to get to the bottom of this? I know folks using this that haven't had the same issue.
Thanks,
John
P.S. Sorry for the long delay in responding.


Develop

Current coverage is 94.39% (diff: 25.00%)

Merging #36 into master will decrease coverage by 4.35%

@@             master        #36   diff @@
==========================================
  Files            10         11     +1
  Lines           319        339    +20
  Methods          81         87     +6
  Messages          0          0
  Branches          0          0
==========================================
+ Hits            315        320     +5
- Misses            4         19    +15
  Partials          0          0

Powered by Codecov. Last update 91820df...c75df3f


Fix error variants/messages in trustchain-ion utils

Closes #187.

Looks great, seems good to merge once we've run the tests including ignored ones.


Make this robust in the face of more packages

So far I've tested this on exactly one package: @observablehq/plot.
It would be good to test this on some other packages too! Help very much appreciated here.
Any packages that don't work can be used to expand the test suite. It's all in triple-string-literals in https://github.com/simonw/download-esm/blob/0.1a0/tests/test_download_esm.py at the moment, but it would make sense to turn those HTTPX fixtures into actual files on disk as it expands.

It works with Preact!
Fetch preact and htm:
download-esm htm
download-esm preact

Save this as index.html:
<script type="module">
import { h, Component, render } from './preact-10-13-2.js';
import htm from './htm-3-1-1.js';

// Initialize htm with Preact
const html = htm.bind(h);

function App (props) {
    return html`<h1>Hello ${props.name}!</h1>`;
}

render(html`<${App} name="World" />`, document.body);
</script>

Run python3 -m http.server and visit http://localhost:8000/ and:

Got this error trying to fetch codemirror:
% download-esm codemirror codemirror
codemirror-6-0-1.js
codemirror-view-6-10-1.js
codemirror-language-6-6-0.js
codemirror-autocomplete-6-6-0.js
codemirror-commands-6-2-3.js
codemirror-state-6-2-0.js
codemirror-lint-6-2-1.js
style-mod-4-0-3.js
w3c-keyname-2-2-6.js
codemirror-search-6-4-0.js
crelt-1-0-5.js
codemirror-view-6-9-5.js
codemirror-view-6-10-1.js
lezer-highlight-1-1-4.js
lezer-common-1-0-2.js
Traceback (most recent call last):
  File "/Users/simon/.local/pipx/venvs/download-esm/lib/python3.11/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
    yield
  File "/Users/simon/.local/pipx/venvs/download-esm/lib/python3.11/site-packages/httpcore/backends/sync.py", line 28, in read
    return self._sock.recv(max_bytes)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File<EMAIL_ADDRESS>line 1263, in recv
    return self.read(buflen)
           ^^^^^^^^^^^^^^^^^
  File<EMAIL_ADDRESS>line 1136, in read
    return self._sslobj.read(len)
           ^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: The read operation timed out

Sort of got Monaco working:
download-esm monaco-editor

<script type="module">
import * as monaco from './monaco-editor-0-37-1.js';

const editor = monaco.editor.create(document.body, {
	value: ['def foo(bar):\n    print("baz")'].join('\n'),
});
editor.getModel().updateOptions({ tabSize: 4 });
</script>

One catch: if I try to add syntax highlighting with language: "python" I get these errors in the console:

Mentioned this in https://simonwillison.net/2023/May/2/download-esm/


Spliting the JoinGame and CreateGame commands and enabling SetupPorts…

… to be called from the Coordinator
This will enable proxy agents to join games independently

Hey there! Thanks so much for putting in the work to create this pull request. I have some feedback -

The examples, in this case bot_mp should remain proxy free. They should represent the minimal amount of work required to run. In this case, building and running the single bot_mp executable.

Similarly, the proxy example is really meant as a starting point. Making it threaded and having it start the game makes it far more complicated for a new user wishing to implement a proxy to digest. I think the functionality you added is definitely necessary for a working proxy but should really be versioned somewhere outside of our examples under your name, so you can get the credit too :).

It's a bit pedantic but your changes don't follow our coding standard.

I would suggest making your own repo for the proxy and submoduling our project. It is fairly easy to implement a cmake file to utilize it. I would be happy to help you with that. The core change to this PR is definitely something I'd be willing to accept. The JoinGame and CreateGame being abstracted into its own functions.

Hey Antony
yeh, sorry the pull request should have only included the first commit with the coordinator changes.  I think you're correct that we should leave the proxy and bot_simple alone.   Sorry about that.
I'll drop this one for the moment and make another request with just the changes to the coordinator instead.  I think that seems more sane :)


Wrong with docker usage!

docker run --name=gogs -p 10022:22 -p 10080:3000 -v /var/gogs:/data gogs/gogs
3000 is the port inside docker
10080 is the port exported.
But when you create a new repo, it has a url with port 3000, which can not be accessed!
How to solve this problem.
I think this is a common issue, and the page below is misleading.
https://github.com/gogits/gogs/tree/master/docker

You should run docker run --name=gogs -p 10022:22 -p 10080:10080 -v /var/gogs:/data gogs/gogs
If you had run docker run --name=gogs -p 10022:22 -p 10080:3000 -v /var/gogs:/data gogs/gogs, open /var/gogs/gogs/conf/app.ini and change the server port to 10080.

I didnt understand what is wrong... for me it works totally fine i create the server like this
docker run --name=gogs -p 10022:22 -p 10080:3000 -v /var/gogs:/data gogs/gogs

and access it on MY_IP:10080  as expected..

But when you create a new repo, it has a url with port 3000, which can not be accessed!
Because you didn't read README carefully to setup current Application URL in install page.


How can i setup create function?

I have defined create function in my nuxt.config.js file.
axios is not imported in nuxt.config.js and i have an error:

ReferenceError: axios is not defined

  feed: [{
      path: '/rss_export', // The route to your feed.
      async create (feed) {
        feed.options = {
          title: 'title',
          description: 'describtion',
        }

        const posts = await axios.get('http://localhost:3000/v1/news/').data

        posts.forEach(post => {
          feed.addItem({
              title: 'test'
          })

          feed.addCategory('funny')

          feed.addContributor({
              // some details
          })
        })
      },
      cacheTime: 1000 * 60 * 15,
      type: 'rss2'
    }
  ],

what is the best way to setup create function? Give me code example please

Hey @pashtetbezd
This looks fine so far, but you need to import axios on top of your file:
const axios = require('axios')

// Your nuxt config here

thnx! it works


trigger build

Fix https://github.com/open-cluster-management/backlog/issues/15232
Signed-off-by: haoqing0110<EMAIL_ADDRESS>

/approve /lgtm


Clicking on the URL in FLZ Token Display error

/explore/tokens/FLZ
Clicking on the URL in FLZ Token Display error
This XML file does not appear to have any style information associated with it. The document tree is shown below.

AccessDenied
Access Denied
3Z3AXJHRNABPP6Q2
pe3Ngtzbz66eILcgTe/DYbOpF4Cd+/6yq4ny/KcDwXuIS8iGscn8GWvPRN+CmC58igirIjmmNOU=

Clear your cache..

Clear your cache..

iphone13 IOS cleared cache still error
AccessDeniedAccess DeniedMZEM4ET54BG90TRY8RIQ9cTVJtHWWZQYVYCUy/7xq33haIzM+WOluPAG8gXsH6OxM+Cej3lAGoqNbRILSdAg8rgtSbc=

Go to https://smartclaimz.io not a page reload.  The page works
It's an amplify issue

/explore/tokens/FLZ
That isn't a valid url anyway.. where did you get that url?
Unless you mean
Click Explore menu-> click Tokens -> click Find Pools for FLZ

yes click on the url i expect to go to freeloaderz.io

You do..  i just did it.

I see what you are talking about now.   It wasn't clear

clear as mud

All good man!
Fixed URL reference #119


Update README.md

Correct link to root README.md which was incorrectly pointing to Bitcoin Core repo.

thanks!


T4 setting to achieve maximum performance

hi @nvpohanh, could you please confirm if there are extra settings to those shown below to get maximum performance on T4 GPU with MLPerf inference benchmarks?

Set Transparent Huge Pages (THP) to always (for Server scenario)
Turn off the ECC RAM
Set GPU Max Clock and  Memory Clock rate
Set Cooling efficiency

Yes, please let me know if you still don't get the reported performance numbers.

Hi @nvpohanh, I am still getting performance issues, please see below:
on RTX6000:
Mobilenet | Server Scenario: ~43,594 img/sec ( Nvidia has reported ~47,775-49,775 img/sec on RTX8000)
on T4:
Resnet-50- | Server scenario: ~4,782 img/sec (Nvidia has reported ~5,193 img/sec)

It is expected that RTX6000/8000 is slightly slower than TitanRTX. To track down, could you share which clock frequency and power level that the GPU stabilizes at during the inference? You can monitor that by running nvidia-smi dmon -s pc concurrently with the harness.
About T4, it would be super helpful to understand the GPU temperature. You can use the same nvidia-smi dmon -s pc command to monitor that.

@nvpohanh, I will run the tests again, meanwhile could you please provide the reference MLPerf inference performance numbers for RTX6000?

Unfortunately we didn't submit RTX6000 numbers for MLPerf Inference v0.5.

Hi @nvpohanh,  does Nvidia MLPerf inference scales automatically with RTX8000/6000?. I am running the inference with 3 GPU's; however, seems it is not scaling linearly, see below:
on 1x_RTX6000:
Mobilenet | Server Scenario: ~43,594 img/sec
on 3x_RTX6000:
Mobilenet | Server Scenario: ~93,724.99 img/sec  (expected: 43,594x3= ~130,781 )

It is expected that RTX6000/8000 is slightly slower than TitanRTX. To track down, could you share which clock frequency and power level that the GPU stabilizes at during the inference? You can monitor that by running nvidia-smi dmon -s pc concurrently with the harness.

Hi @nvpohanh, does Nvidia MLPerf inference scale automatically with RTX8000/6000?. I am running the inference with 3 GPU's; however, seems it is not scaling linearly, see below:
on 1x_RTX6000:
Mobilenet | Server Scenario: ~43,594 img/sec
on 3x_RTX6000:
Mobilenet | Server Scenario: ~93,724.99 img/sec (expected: 43,594x3= ~130,781 )

Hi @nvpohanh, does the Nvidia MLPerf inference system work with multi-GPU on any system?, or does it work multi-GPU only with the systems used for submission v0.5?

I'm running BERT experiments on an AWS G4 g4dn.4xlarge instance using a single T4. The supported clocks are a bit lower than in @vilmara's case:
$ sudo nvidia-smi -q -d SUPPORTED_CLOCKS

==============NVSMI LOG==============

Timestamp                           : Tue Jun 16 16:19:30 2020
Driver Version                      : 440.33.01
CUDA Version                        : 10.2

Attached GPUs                       : 1
GPU 00000000:00:1E.0
    Supported Clocks
        Memory                      : 5001 MHz
            Graphics                : 1590 MHz
            Graphics                : 1575 MHz
...

But at the maximum frequency the GPU is not really stable, dropping down to ~900 MHz in a SingleStream run:
$ sudo nvidia-smi -ac 5001,1590
...
================================================
MLPerf Results Summary
================================================
SUT name : PySUT
Scenario : Single Stream
Mode     : Performance
90th percentile latency (ns) : 98016803
Result is : VALID
  Min duration satisfied : Yes
  Min queries satisfied : Yes

================================================
Additional Stats
================================================
QPS w/ loadgen overhead         : 11.27
QPS w/o loadgen overhead        : 11.27

Min latency (ns)                : 81513330
Max latency (ns)                : 101580923
Mean latency (ns)               : 88718991
50.00 percentile latency (ns)   : 87841138
90.00 percentile latency (ns)   : 98016803
95.00 percentile latency (ns)   : 98051350
97.00 percentile latency (ns)   : 98064832
99.00 percentile latency (ns)   : 98093310
99.90 percentile latency (ns)   : 101015009

It's quite stable at ~800 MHz:
$ sudo nvidia-smi -ac 5001,795
$ python3 run.py --backend=pytorch --scenario=SingleStream
================================================
MLPerf Results Summary
================================================
SUT name : PySUT
Scenario : Single Stream
Mode     : Performance
90th percentile latency (ns) : 100668289
Result is : VALID
  Min duration satisfied : Yes
  Min queries satisfied : Yes

================================================
Additional Stats
================================================
QPS w/ loadgen overhead         : 10.13
QPS w/o loadgen overhead        : 10.13

Min latency (ns)                : 97998490
Max latency (ns)                : 103667665
Mean latency (ns)               : 98672758
50.00 percentile latency (ns)   : 98098138
90.00 percentile latency (ns)   : 100668289
95.00 percentile latency (ns)   : 101665011
97.00 percentile latency (ns)   : 102236103
99.00 percentile latency (ns)   : 102918025
99.90 percentile latency (ns)   : 103634703

but a bit faster at 900 MHz:
$ sudo nvidia-smi -ac 5001,900
$ python3 run.py --backend=pytorch --scenario=SingleStream
================================================
MLPerf Results Summary
================================================
SUT name : PySUT
Scenario : Single Stream
Mode     : Performance
90th percentile latency (ns) : 99973436
Result is : VALID
  Min duration satisfied : Yes
  Min queries satisfied : Yes

================================================
Additional Stats
================================================
QPS w/ loadgen overhead         : 10.57
QPS w/o loadgen overhead        : 10.57

Min latency (ns)                : 88573483
Max latency (ns)                : 109587347
Mean latency (ns)               : 94590988
50.00 percentile latency (ns)   : 94583175
90.00 percentile latency (ns)   : 99973436
95.00 percentile latency (ns)   : 101330092
97.00 percentile latency (ns)   : 102890999
99.00 percentile latency (ns)   : 104760879
99.90 percentile latency (ns)   : 107273294

I have an impression that the T4 on AWS has not-so-good cooling... Could you try nvidia-smi dmon -s pc to see what's the GPU temperature? If it reaches >75C, something needs to be improved.

@nvpohanh Yes, that's the case. I suspect this VM instance takes a slice of a larger machine. Perhaps the neighbours are maxing out their GPUs :).


ng set --global pacakgeManager=yarn Unexpected token y in JSON at position 0

@angular/cli: 1.0.0-beta.31
node: 7.5.0
os: win32 x64 - Windows 10

Repro steps.
Try to set the global packageManger to yarn
ng set --global pacakgeManager=yarn

The log given by the failure.
Unexpected token y in JSON at position 0
SyntaxError: Unexpected token y in JSON at position 0
    at JSON.parse (<anonymous>)
    at resolve (C:\Users\mark.pieszak\AppData\Roaming\npm\node_modules\@angular\cli\commands\set.js:55:39)
    at Class.run (C:\Users\mark.pieszak\AppData\Roaming\npm\node_modules\@angular\cli\commands\set.js:30:16)
    at Class.<anonymous> (C:\Users\mark.pieszak\AppData\Roaming\npm\node_modules\@angular\cli\ember-cli\lib\models\command.js:134:17)
    at process._tickCallback (internal/process/next_tick.js:103:7)

Not sure if intended but your repro steps have a typo
pacakgeManager
should be:
packageManager
I tried to repo this issue and it worked correctly with the typo change:
Ricardos-MacBook-Pro:Projects ricardovaranda$ ng get packageManager --global
default
Ricardos-MacBook-Pro:Projects ricardovaranda$ ng set packageManager=yarn --global
Ricardos-MacBook-Pro:Projects ricardovaranda$ ng get packageManager --global
yarn

Wow I thought something changed in beta31 to cause it. I completely didn't see that typo! You're 100% right, that fixes it 👍👍

I encountered this issue without a typo
⇒  ng set --global packageManager=yarn
Unexpected token u in JSON at position 0
SyntaxError: Unexpected token u in JSON at position 0
    at JSON.parse (<anonymous>)
    at /Users/andreiluca/Projects/my-app/node_modules/angular-cli/commands/set.js:46:39
    at Promise (<anonymous>)
    at Class.run (/Users/andreiluca/Projects/my-app/node_modules/angular-cli/commands/set.js:31:16)
    at Class.<anonymous> (/Users/andreiluca/Projects/my-app/node_modules/angular-cli/ember-cli/lib/models/command.js:152:17)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:169:7)

Same problem, same solution for several cli version. Uninstall from npm.

I know this issue has been closed but I was still having problems with it. JSON.parse wouldn't parse yarn and I had to wrap it in quotes to be read. Here's the command that worked for me:
ng set --global  packageManager='"yarn"'


Update Arethusa Widget in order to fix Safari side bar issue

Fixes https://github.com/perseids-publications/treebank-template/issues/56
Update arethusa-widget, which decreases the sidebar width to 34%. See https://github.com/perseids-tools/arethusa-widget/commit/cab65cfbe9f6309ae1196776ef6b7f98bafbea71 (commit description below):

There is an interaction with the display: table; content: " "; CSS in
the :before and :after around #main-body and #sidepanel. It
looks like this pattern is called clearfix and is designed to fix
issues with floats.

In Safari, this CSS causes there to be a pseudo-element with a width of 1%,
which means that if the sum of widths of the elements adds to 100%,
the rightmost float will drop to the next line.

I don't know if removing the clearfix would cause more
problems on other browsers. To avoid potentially causing even more issues
with a bugfix, this commit just changes the width of the #main-body and
#sidepanel elements to add to 99% instead of 100%. It will look
slightly worse on non-Safari browsers, but will no longer be broken on
Safari.

Also remove widget.css since all the necessary CSS is being included by the arethusa-widget library.

Everything seemed to work without it when I tested in Safari, Chrome, and Firefox, but I thought this was a safer solution since there may be other parts of Arethusa that rely on it or other browsers.


exe file in package?

This seems a bit extreme. The readme says only Mac and Linux are supported, so why not keep the Windows experiment on a branch?
There's also windows-specific packages for the same problem, such as win-audio and node-audio-windows, but both are limited to windows in package.json. Not sure if this is enforced for transient dependencies. Would prevent development on non-windows platforms if it is. Maybe add as peer dep?

Windows is not experimental, it's fully supported. I agree that it's not optimal to ship binaries, but at least the file is only 11KiB...
If you have another approach that would work, I would love to see it!


set associated head sha on pull request event

fixes #206
Signed-off-by: CrazyMax<EMAIL_ADDRESS>

This might seem as a fix in versioning, but it's a breaking change. For example, this breaks all our workflows which depend on the built image sha
Now we have to include extra conditions as the sha won't be the same when there's a PR and a push to main. It would have been nice to increase the version to 5.0.0 considering this would break most scenarios being based on GITHUB_SHA or ${{ github.sha }}
This now requires implementations to use ( ${{ github.event.pull_request.head.sha }} || {{ github.sha }} ) for a single pull to work on both GitHub events (PRs and push)

@gustavovalverde From what I see on https://github.com/ZcashFoundation/zebra/pull/5390, you're pushing an image on PR and others jobs depend on it using github.sha as tag.
I didn't think about this use case as pushing on PR is not possible in majority of cases as secrets can't be exposed. But in your case I see you're using google-github-actions/auth action to connect to your provider using the OIDC token so you're able to push to its registry.
Even if you can share image between jobs without pushing them, it still sounds like a valid use case to use a registry. I will revert this change, sorry about that.


Version out of date

Version 6 is not supported, atleast for gameRecord.py, have not tested with other files.

Thank you for reporting the issue.
I tried to fix it here: https://github.com/advancedfx/afx-sfm-scripts/releases/tag/v1.7.2


Webform - get fields type and label; then handle submission

Hello,
How to retrieve all the {type, label} fields of an webform?
Then, I want to manage the submission?
Thx

There is Rest module for interacting with Drupal Webform:
https://www.drupal.org/project/webform_rest
For me completely fine works embeeding Webforms via iframe, thanks to this i can manage/change easily webform in Drupal. This techniquie requires some JS script to be running on Drupal as well to make iframe adjust properly to wrapper size. Consider also enabling Jquery Validation plugin for Webform to make client + server side validation.
Enable module in Drupal

Configure certain Webform to be shared:
https://www.youtube.com/watch?v=oMgOprfJTWA
React Component for embeeding Webforms within iframe:

Please open a new issue if this is still an unanswered question for you. Thanks!


Add instructions about how to subscribe to dev list

Added instructions on how to subscribe.
@shangxinli

Maybe add a link to the archive as well?
<EMAIL_ADDRESS>

Thanks Vinoo


7B model OOM with 4x4096(96GB)

Hi
I try to train a full 7B model with the DPO script on 4x4096(96Gb), without using Lora or quantization (it is a full model training). But I did train with fp16. Hosting 7b model takes 50GB (Adam optimizer), so my 4x4096 setup should be fine.
I used the dpo script from your example https://github.com/huggingface/trl/blob/main/examples/research_projects/stack_llama_2/scripts/dpo_llama2.py, and I only change the Lora and remove the quantization part.
But if I used 1xA100(80GB), it works fine. Do you know what is the issue with the 4x4096 setup?

Hi @vegasvv
thanks for the issue!
I believe you need to split up your model across all available GPUs, can you load both the reference and active models using device_map="auto" ?

Thanks @younesbelkada i did try with auto for device_map, but it won't let me. It was I can't use auto in distributed model

ah yes , correct, in that case you need to load the training protocol with the technique mentioned here: https://github.com/huggingface/accelerate/issues/1840#issuecomment-1683105994
instead of using Accelerator().process_index you can use PartialState().process_index. please refer to that comment for the explanation on why this hack is needed for distributed training

ah yes , correct, in that case you need to load the training protocol with the technique mentioned here: huggingface/accelerate#1840 (comment) instead of using Accelerator().process_index you can use PartialState().process_index. please refer to that comment for the explanation on why this hack is needed for distributed training

I see, so, I basically do this?
If the model does not fit entirely into a single GPU, you can continue using device_map="auto" but instead of running the script with accelerate launch xxx or python -m torch.distributed.run xxx run it simply with python xxx.py. ?
Or I use device_map=PartialState().process_index but still use accelerate launch xxx?

ah yes , correct, in that case you need to load the training protocol with the technique mentioned here: huggingface/accelerate#1840 (comment) instead of using Accelerator().process_index you can use PartialState().process_index. please refer to that comment for the explanation on why this hack is needed for distributed training

I see, so, I basically do this?
If the model does not fit entirely into a single GPU, you can continue using device_map="auto" but instead of running the script with accelerate launch xxx or python -m torch.distributed.run xxx run it simply with python xxx.py. ?
Or I use device_map=PartialState().process_index but still use accelerate launch xxx? > ah yes , correct, in that case you need to load the training protocol with the technique mentioned here: huggingface/accelerate#1840 (comment) instead of using Accelerator().process_index you can use PartialState().process_index. please refer to that comment for the explanation on why this hack is needed for distributed training
I see, so, I basically do this?
If the model does not fit entirely into a single GPU, you can continue using device_map="auto" but instead of running the script with accelerate launch xxx or python -m torch.distributed.run xxx run it simply with python xxx.py. ?
Or I use device_map=PartialState().process_index but still use accelerate launch xxx?
@younesbelkada

Hi @vegasvv , can you please try method 2

I try, it is still OOM unfortunately..

Hmm, can you make sure gradient checkpointing is set to True ?

If I use method 1, how to start multi-machine @younesbelkada

Hi @zhongshsh, just run it with python xxx.py, for method 2, first run accelerate config, make sure to select multi-GPU and run accelerate run xxx.py

@younesbelkada I try Method 1, python xxx.py when running in multi-machine env, but got error message:
ValueError: You can't train a model that has been loaded with `device_map='auto'` in any distributed mode. Please rerun your script specifying `--num_processes=1` or by launching with `python {{myscript.py}}`.

Hmmm are you sure you ran with python xxx.py ? Are you using python -m torch.distributed.run ?

not sure. I ran with python xxx.py, but in my case of multi node, maybe python xxx.py would be implicitly changed to python -m torch.distributed.run


feat(chat command): Word editing

[x] Command to backtrack by 1 word.
[x] Command to generate 1 word. Can already do this by setting /sentence_token_limit 1 and pressing enter until a full word is generated.
[x] Command to insert 1 word. Can already do this by setting /sentence_token_limit 1 and typing the word you want to insert with a preceding space.

I guess the commands should support more than just 1 word. Token-level editing should be supported too.

Simplified this feature to just the token & word deletion commands. We can get by without special insert and generate commands.

Ugh, I meant to type "feat(chat command)" on that commit. Oh well, can't fix it now.


anchor platform: correct quotes description

Updates the Quotes description to remove the incorrect mention of crypto-to-crypto trading pairs and adds more context.

Preview is available here:http://developers-pr110.previews.kube001.services.stellar-ops.com

Preview is available here:http://developers-pr110.previews.kube001.services.stellar-ops.com

Preview is available here:http://developers-pr110.previews.kube001.services.stellar-ops.com


fix repo

Issues associated with this PR
fix Introduction.md repo
Sign the CLA
Make sure you have signed the CLA
Solutions
You should show your solutions about the issues in your PR, including the overall solutions,
details and the changes. At this time, Chinese is allowed to describe these.
Code Style

Make sure Goimports has run
Show Golint result

Codecov Report

Merging #784 into master will decrease coverage by 0.05%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #784      +/-   ##
==========================================
- Coverage   51.54%   51.48%   -0.06%
==========================================
  Files         205      205
  Lines       17020    17020
==========================================
- Hits         8773     8763      -10
- Misses       7355     7360       +5
- Partials      892      897       +5

Impacted Files
Coverage Δ

pkg/stream/stream.go
46.66% <0%> (-13.34%)
:arrow_down:

pkg/buffer/iobuffer.go
75.31% <0%> (-2.52%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4df4c82...100bfd7. Read the comment docs.

please sign the CLA first. @ianwoolf

looks like i already sign CLA. why the status not sync? or i miss something?

looks like i already sign CLA. why the status not synced? or i miss something?

maybe it is the github's bug.

maybe a new commit will restart the check

ok  i create a new commit


Set colors

Great add-on! This makes it much easier to distinguish windows.
It would be nice to add a feature that allows colors to be reassigned on existing windows. Clicking the "colorful windows" button in the toolbar could provide a color palette to redefine the current window. For instance, I would like my email window to be black, my documentation window to be green, etc.

+1!!
Came looking for this.

+1
This add-on would be much more useful if (a) the color per window could be selected by the user and (b) the user could elect to disable special colors for any given window.

Any idea when this feature will be available

I made an extension that does this: https://github.com/jftuga/foxcolorbox
I have mentioned Colorful-window-theme in the Acknowledgements section of my README.  I wanted to learn extension writing, so I decided to start from the ground up, but I definitely borrowed a lot of ideas from this project.


When do you plan provide Zeus compatibility?

Currently no plans, please refer to https://github.com/myfreescalewebpage/meta-somlabs.

Hi,
I am working on zeus branch.
Preliminary support is available here:
https://github.com/marcinbis/somlabs-bsp/tree/zeus
https://github.com/marcinbis/meta-somlabs/tree/zeus
Regards
On 11.01.2020 15:25, Michał Wołowik wrote:

Currently no plans, please refer to
https://github.com/myfreescalewebpage/meta-somlabs.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/SoMLabs/meta-somlabs/issues/5?email_source=notifications&email_token=AASIHJLSIEWISTD5WXKC7YTQ5HJEBA5CNFSM4KFR6WDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIWDBCA#issuecomment-573321352,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AASIHJLNLFR3JLK5EERHHY3Q5HJEBANCNFSM4KFR6WDA.

--
Marcin Bis
BIS-LINUX.COM | mobile: +48 607 075 848

Thank you. I’m going to try.
On Sat, 11 Jan 2020 at 19:48, Marcin Bis<EMAIL_ADDRESS>wrote:

Hi,
I am working on zeus branch.
Preliminary support is available here:
https://github.com/marcinbis/somlabs-bsp/tree/zeus
https://github.com/marcinbis/meta-somlabs/tree/zeus
Regards
On 11.01.2020 15:25, Michał Wołowik wrote:

Currently no plans, please refer to
https://github.com/myfreescalewebpage/meta-somlabs.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<
https://github.com/SoMLabs/meta-somlabs/issues/5?email_source=notifications&email_token=AASIHJLSIEWISTD5WXKC7YTQ5HJEBA5CNFSM4KFR6WDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIWDBCA#issuecomment-573321352>,

or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AASIHJLNLFR3JLK5EERHHY3Q5HJEBANCNFSM4KFR6WDA
.

--
Marcin Bis
BIS-LINUX.COM | mobile: +48 607 075 848
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/SoMLabs/meta-somlabs/issues/5?email_source=notifications&email_token=ACXOLEYJE34TA2WHEIE3Q6TQ5IA7HA5CNFSM4KFR6WDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIWHFTI#issuecomment-573338317,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACXOLE47SHVFIZACOPQVKRTQ5IA7HANCNFSM4KFR6WDA
.


Generate operator mapping between Photon execs and CPU execs to be used in plan parser

Ahead of qualification tool support for Photon, we need to create mapping of Photon operators to CPU operators to then estimate speedup.

Here is the set of Photon-specific operators that show up in the event log analysis output from our qualification tool:
PhotonBroadcastExchange
PhotonBroadcastHashJoin
PhotonBroadcastNestedLoopJoin
PhotonFilter
PhotonProject
PhotonSubqueryBroadcast
PhotonAdapter
PhotonAgg
PhotonBroadcastAdapter
PhotonGroupingAgg
PhotonGroupingAggWithRollup
PhotonResultStage
PhotonShuffleExchangeSink
PhotonShuffleExchangeSource
PhotonShuffledHashJoin
PhotonShuffleMapStage
PhotonUnionShuffleExchangeSink
PhotonUnionShuffleMapStage

The first 6 have direct mappings to CPU equivalents by dropping the Photon prefix:
PhotonBroadcastExchange
PhotonBroadcastHashJoin
PhotonBroadcastNestedLoopJoin
PhotonFilter
PhotonProject
PhotonSubqueryBroadcast


Waterbody vis

Proposed changes
New RWE notebook showing two animations for waterbodies.
Checklist (replace [ ] with [x] to check off)

[X] Remove any unused Python packages from Load packages
[X] Remove any unused/empty code cells
[X] Remove any guidance cells (e.g. General advice)
[X] Ensure that all code cells follow the PEP8 standard for code. The jupyterlab_code_formatter tool can be used to format code cells to a consistent style: select each code cell, then click Edit and then one of the Apply X Formatter options (YAPF or Black are recommended)
[X] Include relevant tags in the first notebook cell and re-use tags if possible
[X] Ensure appropriate colour schemes  have been used to maximise accessibility for vision impairment. Test your images or learn more with Coblis or TPGI
[X] Clear all outputs, run notebook from start to finish, and save the notebook in the state where all cells have been sequentially evaluated

After rerunning the notebook, it's working. Bravo @mickwelli.

Split this into 2 notebooks and added further examples, requesting re-review from @nanaboamah89

@mickwelli , I have gone through the notebooks. For the Waterbodies Seasonal Animations notebook Instead of having two examples in the notebook, having one example is fine, and in the conclusion or at the analysis parameter section, we can let the use know they can run test it Bakolori Dam by providing the latitude and longitude to them an example.

Thanks Michael, the notebooks look good with two different examples of how the timeseries data can be used and visualised.
Notebook 1: Seasonal animations
A couple of minor edits:

Analysis parameters - "water extent over a shorter time period" (change shorter to short)
Inspect timeseries - might be useful to add explanation for geohash listing/identification (and add link to waterbodies documentation and dataset notebook)

For the timeseries with rainfall - can you increase offset between the two parameters (i.e. increase max value of each axis) for both occurrences of the graph.
I still feel there is too much changing (3 things) to fully interpret what is happening in the animation. and recommend keeping the chart parameters static but indicating where in the timeseries the the geomad corresponds to
Notebook 2: long term
Can you check the geometry of the annual WOfS in the animation - it looks stretched S-N compared to the summary above

Thanks @lisarebelo, I've made the following revisions:
Long term notebook:

Added fixed aspect ratio to plots and animation, WOfS was actually stretched E-W, it's a bit illusory.

Seasonal notebook:

Added text under Inspect Timeseries providing background on the Waterbodies service, linking to Datasets notebook and docs.
Edited analysis parameters text.
Increased space on y axes for animation
Tried to overcome visual crowding by having the charts static in transparent aesthetic, with solid lines and bars coming in with animation to show timing. Interested in your thoughts on this.
Fixed up data referencing issue for Bakolori Dam, I think it's quite a neat example now.


"this_thread_block()" is missing in hip_cooperative_groups.h

"this_thread_block()" is missing in hcc_detail/hip_cooperative_groups.h. It is working for Nvidia
machines due to the inclusion of Cuda cooperative_groups.h but not on AMD machines.

This is deliberate.  The feature is not implemented or released yet.


Specify minimum required Python version

Hello,
thank you for sharing this module.
It might be worth specifying the minimum required Python version. Your use of typing.Literal requires Python 3.8 or newer. This could be specified with python_requires=">=3.8" in the setup() call in setup.py.
Best,
Juraj

I added the line ion the setup.py. Thank you for the suggestion!


How to Add A White Flash Event?

Describe your problem here.
I want to know how to add a white flash in psych engine by source code or by build if we can
Are you modding a build from source or with Lua?
Lua
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
No

With Lua, use the cameraFlash() function
To create a white flash event you would do cameraFlash("game", "0xFFFFFFFF", 1, true), "game" being the camera (camGame) which you could change to hud or other, "1" being the duration of the flash, and "true" being if it's forced

With Lua, use the cameraFlash() function To create a white flash event you would do cameraFlash("game", "0xFFFFFFFF", 1, true), "game" being the camera (camGame) which you could change to hud or other, "1" being the duration of the flash, and "true" being if it's forced

thanks

but we have to write the code in the stage's lua?

or in custom events?

custom events


lynis: 3.1.1 -> 3.1.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/CISOfy/lynis/releases.
meta.description for lynis is: Security auditing tool for Linux, macOS, and UNIX-based systems
meta.homepage for lynis is: https://cisofy.com/lynis/
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 3.1.2 with grep in /nix/store/5a8gviinkkklfi206nkxyp7bk7jx451s-lynis-3.1.2
found 3.1.2 in filename of file in /nix/store/5a8gviinkkklfi206nkxyp7bk7jx451s-lynis-3.1.2

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

lynis

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/5a8gviinkkklfi206nkxyp7bk7jx451s-lynis-3.1.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A lynis https://github.com/r-ryantm/nixpkgs/archive/ad41323c9df3c55a9fa1598c4cb3a1daf9bfe735.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/ad41323c9df3c55a9fa1598c4cb3a1daf9bfe735#lynis

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/5a8gviinkkklfi206nkxyp7bk7jx451s-lynis-3.1.2
ls -la /nix/store/5a8gviinkkklfi206nkxyp7bk7jx451s-lynis-3.1.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

lynis

Maintainer pings
cc @ryneeverett for testing.

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.

@ryneeverett seems to work for me.


Add authentication capability to the Web UI

Right now the web UI is completely open for anyone to connect and modify things.
A useful feature sometime in the future would be the ability to require a password.
If its feasible, it would be nice if would allow any of the Linux users who are in a certain group to log in. Or, by default, it could just be the 'pi' user.

@akuker What would you like to see happen when you access the webapp without logging in? Shall you see only a login screen, or shall we have a 'read-only' view where you can inspect everything but not carry out any actions (similarly to how Jenkins does it).
On the other end of the spectrum, perhaps we only block access to endpoints that modify the file system, and allow anonymous users to f.e. attach or detach devices, look at logs and change log levels, etc.
Thoughts?

I like the idea of going into read-only mode. But, we should also disable downloading files while in read-only mode. Basically if you're not logged in, you can see what is connected, but that's it.

@akuker Sounds like a plan that I can buy into. I was thinking of using Flask-Login https://flask-login.readthedocs.io/en/latest/ . Once we figured out a login flow, it should just be a matter of adding the @login_required decorators to the endpoints that we want to protect (pretty much all of them, except index, drive_list, send_pwa_files, show_logs, and maybe device_info.)
@erichelgeson For your visibility. Let us know if you have other recommended libraries!

Why not keep it simple and have it opt in? Maybe just ask the user if they want to secure it with a password and setup a simple basic auth: https://nginx.org/en/docs/http/ngx_http_auth_basic_module.html

What about using PAM to authenticate users to make the setup a little easier? That would be my vote. We could still have a configuration option to enable or disable password requirements.
There's the issue of sending un-encrypted passwords over the network though. But maybe that could be a feature for the next release - to add optional https support. (So that we can still use Netscape and other old browsers)

PR using simplepam and a webform in the header for authenticating with Linux users on the system. https://github.com/akuker/RASCSI/pull/483

@rdmark @akuker I have a related question/proposal on authentication/security. IMO RaSCSI is not security-critical, but adding a decent level of authentication to the RaSCSI core might be useful. This is what I suggest: An additional optional parameter that denotes a file with a password. This file must be readable by root only and RaSCSI has to check this when it is started. All RaSCSI operations are only executed if a new "password" parameter is passed with each operation. It's good to have named parameters, so we can add this optional new parameter without losing backwards compatibility.
This approach has a loophole, though: The password has to be sent in cleartext via the remote interface. But as already mentioned, RaSCSI is not security critical, and any more elaborate solution is probably too complicated to be implemented at all.

@rdmark @akuker I would appreciate some feedback.

@uweseimet Will the clients have to pass the password in their command payloads, or is this done internally in the rascsi process only?

@rdmark The password would be an additional parameter to each interface operation. rascsi cannot do that, because this would defeat the purpose. The client has to authenticate against the service, i.e. rascsi, not the service against itself.
rasctl would most likely have an additional option, which triggers rascsl to wait for a password input. This would be the same as with the mysql command line client.
As already mentioned, this feature would be optional, i.e. the password parameter would only be required if authentication is configured, i.e. if a password file was specified when starting rascsi.

@uweseimet Understood, so this would be a replacement for the 'magic word' (RASCSI) that you introduced a few months ago. How would the client know whether or not to pass the password parameter? Or maybe I'm overthinking this, and both server and clients would just look for the existence of that password file on the file system, and if it exists take the auth code paths.

@rdmark No, the magic word is just a technical feature on a low level, to ensure that you do not connect to and crash the service by accident.
The client does not know anything about the token file. It does not even know where the file is located, because that's specified when starting rascsi (see my initial comment above). It must definitely not use the file the server uses, because this would render everything useless. The client might not even run on the same machine as rascsi.
All in all this is just like with any other token-based approach. It's up to the client to know where to get the token from, but the client never uses the same source for the token as the server. For rasctl user input is required, the web ui might also require user input, or would be configured with a token or a token file, or woudl not use this feature at all. (The standard rascsi setup should IMO not require a token.) Other clients might store the token (entered by the user) in a database, e. g. an app.
In general, any authentication not enforced by rascsi but by other means is meaningless. If, for instance, there was some kind of authentication in the web ui only, it would not prevent any other client (e. g. rasctl) from having full access to rascsi.

I created https://github.com/akuker/RASCSI/issues/528 for the authentication feature.

Client implementation of token authentication feature here https://github.com/akuker/RASCSI/pull/554


void-variable: evil-multiedit-state-map error when invoking evil-multiedit

What did you expect to happen?
It was working before I upgrade today.
When I invoke evil-multiedit-match-symbol-and-next It should work properly but it is erroing with:
let*: Error in private config: "multiedit.el", (void-variable evil-multiedit-state-map)

What actually happened?
 It wont display that error but also wont find any match for anything I search.
Steps to reproduce
M-x evil-multiedit-match-symbol-and-next
System Information
http://ix.io/3FD6

This issue should now be resolved on Doom's side, however, the error seems to be coming from /home/matheus/.doom.d/multiedit.el. It's likely you have a define-key using the old evil-multiedit-*-state-map keymaps. Use evil-multiedit-mode-map instead.
Anyhow, let me know if the issue isn't resolved and I'll reopen this issue. Thanks for bringing it to my attention!

Thanks, that was fast! Indeed I had to replace evil-multiedit-state-map with evil-multiedit-mode-map. Also seems that evil-multiedit-insert-state-map no longer exists.
Another thing I noticed is that evil-multiedit-[next|prev] is not working but evil-multiedit-match-and-next sorta replaces it anyway. Maybe thats an issue for multiedit's repo.
Anyway it is working now.

Also seems that evil-multiedit-insert-state-map no longer exists.

That's right. It's all in one keymap now (evil-multiedit-mode-map). You can bind insert state keys there too:
(map! :map evil-multiedit-mode-map :i "C-n" #'a-command)
;; or without Doom:
(evil-define-key 'insert evil-multiedit-mode-map (kbd "C-n") #'a-command)

Another thing I noticed is that evil-multiedit-[next|prev] is not working but evil-multiedit-match-and-next sorta replaces it anyway. Maybe thats an issue for multiedit's repo.

This may need to be reported to evil-multiedit, but it appears to be working fine in vanilla Doom with the out-of-the-box keybinds. Make sure you're not somehow overriding these from your private config.


how to predict time by using AFT model?

Is it possible to predict survival times directly, as in the following torchlife code?

from torchlife.model import ModelAFT

model = ModelAFT('Gumbel')
model.fit(df)
surv_prob = model.predict(df)
mode_time = model.predict_time(df)

Dear @abebe9849.
Thank you for your interest and inquiry.
At the moment we do not have a built-in function to predict survival time. However, in the Weibull AFT, the survival times follow a Weibull distribution. So you could easily predict survival time by generating random samples from the Weibull distribution.
Using synthetic data, this is how you could do it
import torch
from torchsurv.loss.weibull import survival_function

_ = torch.manual_seed(42)

# Generate synthetic data
n = 50
time = torch.randint(low=1, high=100, size=(n,))
log_params = torch.randn((n, 2))

# Compute the survival probability from weibull log parameters
surv_prob = survival_function(log_params, time)  # shape = (n,n)

# Generate predicted survival time
m = 10000  # number of predictions
log_scale, log_shape = log_params.unbind(1)
predicted_time = torch.distributions.weibull.Weibull(
    torch.exp(log_scale), torch.exp(log_shape)
).sample(
    (m,)
)  # shape = (m,n)
predicted_time_mean = predicted_time.mean(dim=0)  # shape = (n)

I hope this is useful.
Melodie


Implement "xdm" subcommand

Implement eXport DMN Model (xdm) subcommand.

Implemented in version 0.4.0.


Update to Material Components Web 4.0

Hey, thanks for this nice project. =)
This package still uses Material Components Web 3.10. But version 4.0 of the Material Components Web was released yesterday (2019-11-02) with some quite big changes.

Lol yep. Takes a bit more than a day to get an update out after them, I usually trail 2-4 weeks but thanks for the heads up. I’ve been talking  with some people from that team for a while so I’m excited to see what’s new.

And here we go again. I am not sure what's more complicated. Rocket science or building a robust select component :D To be fair, I think the changes to the select very much make sense.

I wish they would build new ones instead of continuing to break old ones, but I enjoy the challenge :)

I agree totally with the fact that new components would be more welcome than breaking old ones, specially when these breakages have removal of features (for example, the new mdc-select doesn't support multiple selections, which was supported by the old native select component..).

@fjorgemota got an empty PR going for this, I use their changelog as the checklist for checking and cross referencing so that should give you some visibility into this. Still hoping I can keep this to a non breaking release, I'm going to figure out a way to keep native selects around ;).

@jamesmfriedman Nice! If you want I can just put that checklist on this issue, as a meta-issue, so everything can be tracked here, instead of on the PR.
The native selects update should be the harder to do right, I think. The other changes are a bit...small, it seems.

RMWC is undergoing another massive transformation (about the 3rd one since it's been alive). Converted the whole internal api to hooks, and got rid of the abstractions to make everything look like idiomatic react. Published a 6.0.0 alpha release which you can install with rmwc@next or the individual components.
The mdc 4.x.x updates will be going in along with it, but had to get the api moved over first. Feedback welcomed.

As 5.x.x is out for mdc, would that be the target for version 6.0.0 of this?

I did see that the other day. Yes that will be the target version for 6.0.0, I want to make sure I have all of the revisions in place from mdc 4 first.

Update on this, branch is approaching release. Got everything updated to 5.x, just bug squashing and testing.

Fixin in 6.0.0.


Inspector controls 'flicker' or disappear while typing

Describe the bug
While typing code, some inspector controls rapidly change state, including ones not related to the property being edited
To Reproduce

Open this test project https://utopia.pizza/p/a645be4c-alpine-climb/
Edit the hr and set its color prop to yellow or orange

Observed: the inspector 'flickers', some row / controls seem to disappear and then quickly re-appear - see attached video
Expected behaviour
No flickering, inspector rows stay the same.
Video
https://user-images.githubusercontent.com/2945037/113146160-3ece0100-9227-11eb-9e86-45770f58cb86.mp4

This is caused by the UID of the selection changing. The only way to fix this is to attempt to retain UIDs during the parsing of the code somehow, which requires discussion.

Closed by #1102


docs(headers): fix markdown header

Coverage remained the same at 100.0% when pulling cdab757f60683837ad581a53193fe6a55d15062c on vvo:patch-1 into c5b3a4206d654077e462b90bb83c59c1e925cdc7 on keithmorris:develop.

@vvo Thanks for the update!


Debugger UX should show stack for current thread instead of a treeview with all threads

Originally by @davidfowl
Debugging any ASP.NET Core application using devkit (I turned off just my code as well), I see something like this:

Can we make it show the thread we're stepping show up there and not all threads? I don't know if that means have a thread switcher UX or another "multiple threads with stacks UX".

The feature here is for VS Code to provide a way to not show all threads and their CallStacks, just a single CallStack of the current thread and something to toggle between threads.
Related: https://github.com/dotnet/vscode-csharp/issues/7216

Is this basically just https://github.com/microsoft/vscode/issues/116109? I see that @davidfowl said they are different requests, but they seem at least closely related to me

It's similar on the surface but not practically. Visual Studio's default debugging experience supports multiple threads and a "current stack trace". These are usually physical threads and the number of them is usually manageable in a UI like a dropdown.
For the uninitiated in Visual Studio experience, here's a video for what exists:
https://github.com/microsoft/vscode/assets/95136/1ee443b7-1132-49fc-afe5-8cfa99ebfe31
Notice the callstack window is just the stack and you are able to switch threads. Which changes both the callstack and the editor. This is a one at a time view. This is an enhancement to what exists in VS code.
What this issue is about is more like the Parallel tasks view or the tasks window in Visual Studio. Lets call it the "green thread" view. The main difference is the scale. You don't have 1M threads to switch between, you have 10K "green threads/goroutines/tasks", and that might be completely normal. These "green threads" are logical threads with logical stacks and can share the same view, but the better view for these types of systems is one that groups common stacks together to manage the scale.
See https://learn.microsoft.com/en-us/visualstudio/debugger/using-the-parallel-stacks-window?view=vs-2022#tasks-view for more information


api: stunreachability is not supported by list_measurements API

When performing this query: https://backend-fsn.ooni.org/api/v1/measurements?test_name=stunreachability
You get the following error:
'stunreachability' is not one of ['bridge_reachability', 'dash', 'dns_consistency', 'dnscheck', 'facebook_messenger', 'http_header_field_manipulation', 'http_host', 'http_invalid_request_line', 'http_requests', 'meek_fronted_requests_test', 'multi_protocol_traceroute', 'ndt', 'psiphon', 'riseupvpn', 'signal', 'tcp_connect', 'telegram', 'tor', 'torsf', 'urlgetter', 'vanilla_tor', 'web_connectivity', 'whatsapp']

It's worth noting that the aggregation API endpoint works fine:
https://backend-fsn.ooni.org/api/v1/aggregation?test_name=stunreachability&axis_x=measurement_start_day&axis_y=probe_cc

Fixed and deployed as part of https://github.com/ooni/api/pull/271


Alpha diversity: remove "evenness" method

Discussed in the microbiome channel, the "evenness" method in alpha diversity is not appropriate for the data we have. We load relative (normalized) abundance, but the "evenness" method assumes absolute (non-normalized) abundance data.
Until we have the ability for apps to constrain data they use, which is a few build cycles off at least, we decided to simply remove the "evenness" method from alpha diversity.

Evenness is no longer an option


Convert an array to reduced form using hashing

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
@akgmage  assign this issue to me

done in hash table folder

#1575 @akgmage


RF: Fix ITK warp conversion to nitransforms format

Alternative to (and builds on) #3296.

Actually just looked at the MNI-resampled boldref in https://app.circleci.com/pipelines/github/nipreps/fmriprep/2204/workflows/b0c88b0f-7a65-4904-a5d4-7dc26b8233f4/jobs/11504/artifacts, and it's degraded relative to your PR and master. Let me try to understand this...

Hmm. I loaded the file with my function and yours, and they seem identical. I'm going to re-run the test, in case it's just a one-off weirdness.

Okay, it appears to have been a one-off bad boldmask, not a problem with this function at all.

The only concern I have is this code is copied in a number of places (here, nibabies, resampler package) - we should focus some energy on having a single distribution and importing from there (maybe an ohbm hackathon project?)

And yes. We should get this fixed up in nitransforms so we can stop having our own special loader.


Remove dependency on paramiko

This is to resolve Issue #10.
Paramiko doesn't get us much of anything in this app. In fact, it probably slows things down with the need to load the entire host keys file. This swaps out paramiko for just making subprocess calls to the ssh CLI.

Coverage decreased (-0.2%) to 99.345% when pulling ae0628c31a08ab3b6aa87e13b662e90f9ab6c58d on toddpalino:master into 9f85237cd0c62f8036842cdccafff2efb0d73169 on linkedin:master.

[Current coverage][cc-pull] is 99.34%

Merging [#12][cc-pull] into [master][cc-base-branch] will decrease coverage by 0.22%

@@             master        #12   diff @@
==========================================
  Files            29         29
  Lines           921        916     -5
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits            917        910     -7
- Misses            4          6     +2
  Partials          0          0

Powered by Codecov. Last updated by [9f85237...ae0628c][cc-compare]
[cc-base-branch]: https://codecov.io/gh/linkedin/kafka-tools/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/linkedin/kafka-tools/compare/9f85237cd0c62f8036842cdccafff2efb0d73169...ae0628c31a08ab3b6aa87e13b662e90f9ab6c58d
[cc-pull]: https://codecov.io/gh/linkedin/kafka-tools/pull/12?src=pr


Fix _cirq_to_bloq wire symbols.

The automatic wire symbol function was broken when drawing musical score diagrams. This fixes that.
Before:

After:

Was this related to #591 ? I don't think so, right? Or was this always a bug?
Would it be possible to add a small test

It was always a bug, see #530. I can add a test.

Or rather, I imagine it became a bug once the cirq-ft adopted qualtran registers.

I added a test for qrom which would have failed before.


Fix Warnings

Fix build error messages
C:\Users\joewi\Documents\GitHub\2024Crescendo\src\main\cpp\utils\logging\DataTraceSocket.cpp:71:47: warning: '.' directive writing 1 byte into a region of size between 0 and 15 [-Wformat-overflow=]
71 |             sprintf(ipAddressBuffer, "%d.%d.%d.%d", octets[0], octets[1], octets[2], i);
|                                               ^
C:\Users\joewi\Documents\GitHub\2024Crescendo\src\main\cpp\utils\logging\DataTraceSocket.cpp:71:20: note: 'sprintf' output between 10 and 25 bytes into a destination of size 20
71 |             sprintf(ipAddressBuffer, "%d.%d.%d.%d", octets[0], octets[1], octets[2], i);

This is already fixed and merged into main


Is the "Run Starsector" config supposed to also load your mod?

I expected that it would build my mod and enable it in the game, then run the game, but instead it just seems to run the game?
Another strange behaviour is that clicking the "Run" button launches a process that seems to just wait indefinitely without loading the game or terminating, but clicking the debug button makes the game run.
Whatever the expected behaviour is, could we edit the readme to clarify this, thanks!

Thanks, I've added a section for that. It was in the wiki version of this guide but not this one.
The Run button isn't meant to be used. If you want to, you can change suspend=y to suspend-n in the Run configuration's parameters, or make a new run config for it, or maybe there's a way to use one run config to get both, but frankly I don't see why someone would want to run from IntelliJ without debugging and, if that's needed, they can just run the game normally, so I never spent time investigating the option.

 Just to clarify: clicking the debug button is not supposed to automatically load your mod it just lets you debug starsector, if you want to load your mod you need to copy it from the "out" directory into the starsector mods directory.

Ah, I see that you explain this in the wiki version too. https://starsector.fandom.com/wiki/IntelliJ_IDEA_Setup#Running_and_debugging

It shouldn't create an out folder, by the way. It's set up to add the jar to /jars using this:
https://github.com/wispborne/Starsector-IntelliJ-Template/blob/main/.idea/artifacts/Create__jar.xml

Ah, that makes sense. I was using some of these files with an existing mod, but an edit of Create__jar.xml got it all working for me :)


Minimum remaining time

We can probably rule out a good handful of attempts that can't possibly finish within the search heap max time, just by tallying the minimum time necessary to finish the run. That is, we could sum up:

access times of required locations
the min access times of locations with required items (in case of duplicate locations)
traversal times to the spots with those locations.

Ideally, "required locations/items" is preparatory knowledge, but on the other hand, there may be additional requirements as locations are marked skipped during a route.
Alternatively, if no locations are skipped until the minimization phase, then we could just calculate the sum of all todo locations (taking the minimum of any canonical locations), as we won't even start that phase until we collect everything. It's unclear whether we should do a BFS to analyze the minimum traversal as well, or if we should just skip that.

Duplicate of #59!


sage dependency updates

I will slowly work on this.

@ofborg build sage

@collares ping :)

Many thanks for all the reviews everyone :) I value your comments but I also value your time, so (while I will quickly apply any suggested changes) I should clarify that nothing in this draft PR passes tests yet, and it might take a long time to debug the failures and get this to a mergeable state. Thanks for your patience!

FYI: https://trac.sagemath.org/ticket/30537

xcas still works ☑

@symphorien Thanks for checking! In a sense giac is the most annoying part, because the nofltk patch was not updated so we cannot run tests on fltk-less giac anymore (but we can on giac-with-xcas, which should be enough). I will look into updating the patch.

@ofborg build giac-with-xcas

Status update:

The cypari2 caused a new test failure on aarch64, and fixing it properly requires Cython changes (#115200). I've included that PR here for testing purposes, but I'll remove it once it gets merged to master (which will take a few weeks at best, since if approved it'll land in staging).
Given the above, this PR will take a couple of weeks to land, so I've decided to drop the pari update for now. Upstream still needs to update some algorithms related to Smith Normal Form changes, and those changes are causing test failures at the moment. I'll reinclude the pari update if it lands upstream before the Cython changes stabilize.
I've contacted giac developers about new test failures with fltk disabled, and I've disabled the failiing tests (chk_fhan2 and chk_fhan9) for now.

@omasanori Even though this does not update cypari2 or pari, I am marking this as ready for review. I will submit a PR to update cypari2 on staging as soon as this PR is merged, but will probably wait a little bit more for the pari upgrade.
Note that I've included an upstream patch to fix numpy usage which currently breaks sage tests. If you don't have enough time to review the whole thing I can split the numpy fix off into a separate PR. I won't wait for ofborg's test run, but if it's green this should be ready to merge.

Thank you so much, @collares! I will submit my review results by next weekend.

@omasanori That's intriguing! Those tests pass here. Can you paste the failing test output here or in a Gist?

Yes, of course.

system: "x86_64-linux"
host os: Linux 3.10.0-1160.11.1.el7.x86_64, CentOS Linux, 7 (Core)
multi-user?: no
sandbox: yes
version: nix-env (Nix) 2.3.10
channels(ogino): "nixpkgs-21.05pre273941.a2b0ea6865b"
nixpkgs: /home/ogino/.nix-defexpr/channels/nixpkgs

src/sage/finance/time_series.pyx
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/finance/time_series.pyx", line 608, in sage.finance.time_series.TimeSeries.autoregressive_fit
Failed example:
    v.autoregressive_forecast(F)
Expected:
    86.0177285042...
Got:
<PHONE_NUMBER>9219573
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/finance/time_series.pyx", line 610, in sage.finance.time_series.TimeSeries.autoregressive_fit
Failed example:
    F
Expected:
    [1.0148, -0.0029, -0.0105, 0.0067, -0.0232 ... -0.0106, -0.0068, 0.0085, -0.0131, 0.0092]
Got:
    [2.7845, -1.0730, 154.7546, -160.5283, 2.5521 ... -0.0583, 0.0665, -0.0665, 0.9782, -0.8029]
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/finance/time_series.pyx", line 2650, in sage.finance.time_series.autoregressive_fit
Failed example:
    F = finance.autoregressive_fit(ac); F
Expected:
    [0.9982, -0.0002, -0.0002, 0.0003, 0.0001 ... 0.0002, -0.0002, -0.0000, -0.0002, -0.0014]
Got:
    [3810.5722, 2916.4368, -3042.5908, -189282.4881,<PHONE_NUMBER> ... 0.0000, -0.0000, 0.0051, -0.4606, 0.4554]
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/finance/time_series.pyx", line 2655, in sage.finance.time_series.autoregressive_fit
Failed example:
    sum(F)
Expected:
    0.99593284089454...
Got:
    8504.685577940327
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/finance/time_series.pyx", line 2667, in sage.finance.time_series.autoregressive_fit
Failed example:
    y2[:-1].autoregressive_forecast(F)
Expected:
    6.7836741372407...
Got:
<PHONE_NUMBER>7810324
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/finance/time_series.pyx", line 2673, in sage.finance.time_series.autoregressive_fit
Failed example:
    y2[:-1].autoregressive_forecast(y2[:-1].autoregressive_fit(len(y2)))
Expected:
    6.770168705668...
Got:
    9.380452670879273e+29
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/finance/time_series.pyx", line 2688, in sage.finance.time_series.autoregressive_fit
Failed example:
    s2/s1
Expected:
    1.15464636102...
Got:
<PHONE_NUMBER>021793
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/finance/time_series.pyx", line 2706, in sage.finance.time_series.autoregressive_fit
Failed example:
    s.mean()
Expected:
    1.354073591877...
Got:
    0.2404119161741636
**********************************************************************
2 items had failures:
   2 of  16 in sage.finance.time_series.TimeSeries.autoregressive_fit
   6 of  21 in sage.finance.time_series.autoregressive_fit
    [343 tests, 8 failures, 11.70 s]

src/sage/geometry/polyhedron/library.py
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/geometry/polyhedron/library.py", line 2475, in sage.geometry.polyhedron.library.Polytopes.hypersimplex
Failed example:
    TestSuite(h_7_3).run(skip="_test_pyramid")
Expected nothing
Got:
    Failure in _test_linear_transformation:
    Traceback (most recent call last):
      File "/nix/store/9kjg3zzx8rfd3vs716scs2f4k8fx4qvg-python3-3.8.8-env/lib/python3.8/site-packages/sage/misc/sage_unittest.py", line 297, in run
        test_method(tester=tester)
      File "/nix/store/9kjg3zzx8rfd3vs716scs2f4k8fx4qvg-python3-3.8.8-env/lib/python3.8/site-packages/sage/geometry/polyhedron/base.py", line 5332, in _test_linear_transformation
        tester.assertEqual(self, self.linear_transformation(identity_matrix(self.ambient_dim())))
      File "/nix/store/9kjg3zzx8rfd3vs716scs2f4k8fx4qvg-python3-3.8.8-env/lib/python3.8/site-packages/sage/geometry/polyhedron/base.py", line 5309, in linear_transformation
        return new_parent.element_class(new_parent, [new_vertices, new_rays, new_lines],
      File "/nix/store/9kjg3zzx8rfd3vs716scs2f4k8fx4qvg-python3-3.8.8-env/lib/python3.8/site-packages/sage/geometry/polyhedron/backend_cdd.py", line 520, in __init__
        Polyhedron_cdd.__init__(self, parent, Vrep, Hrep, **kwds)
      File "/nix/store/9kjg3zzx8rfd3vs716scs2f4k8fx4qvg-python3-3.8.8-env/lib/python3.8/site-packages/sage/geometry/polyhedron/base.py", line 208, in __init__
        self._init_from_Vrepresentation_and_Hrepresentation(Vrep, Hrep)
      File "/nix/store/9kjg3zzx8rfd3vs716scs2f4k8fx4qvg-python3-3.8.8-env/lib/python3.8/site-packages/sage/geometry/polyhedron/backend_cdd.py", line 646, in _init_from_Vrepresentation_and_Hrepresentation
        try_init(prim)
      File "/nix/store/9kjg3zzx8rfd3vs716scs2f4k8fx4qvg-python3-3.8.8-env/lib/python3.8/site-packages/sage/geometry/polyhedron/backend_cdd.py", line 621, in try_init
        s = self._run_cdd(s, '--repall', verbose=verbose)
      File "/nix/store/9kjg3zzx8rfd3vs716scs2f4k8fx4qvg-python3-3.8.8-env/lib/python3.8/site-packages/sage/geometry/polyhedron/backend_cdd.py", line 170, in _run_cdd
        raise ValueError(ans.strip())
    ValueError: *Error: Numerical inconsistency is found.  Use the GMP exact arithmetic.
    ------------------------------------------------------------
    The following tests failed: _test_linear_transformation
**********************************************************************
1 item had failures:
   1 of   9 in sage.geometry.polyhedron.library.Polytopes.hypersimplex
    [307 tests, 1 failure, 82.48 s]

src/sage/geometry/polyhedron/base.py
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/geometry/polyhedron/base.py", line 8007, in sage.geometry.polyhedron.base.Polyhedron_base.volume
Failed example:
    w = Dinexact.faces(2)[2].as_polyhedron().volume(measure='induced', engine='internal'); RDF(w) # abs tol 1e-9
Expected:
    1.5340627082974878
Got:
    8.032456584010367
Tolerance exceeded:
    1.5340627082974878 vs 8.032456584010367, tolerance 7e0 > 1e-9
**********************************************************************
1 item had failures:
   1 of  46 in sage.geometry.polyhedron.base.Polyhedron_base.volume
    [1575 tests, 1 failure, 83.19 s]

src/sage/matrix/matrix_double_dense.pyx
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/matrix/matrix_double_dense.pyx", line 625, in sage.matrix.matrix_double_dense.Matrix_double_dense.condition
Failed example:
    A.condition()  # tol 2e-4
Expected:
<PHONE_NUMBER>9146.014
Got:
<PHONE_NUMBER>83183.9
Tolerance exceeded:
<PHONE_NUMBER>9146.014 vs<PHONE_NUMBER>83183.9, tolerance 2e1 > 2e-4
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/matrix/matrix_double_dense.pyx", line 1347, in sage.matrix.matrix_double_dense.Matrix_double_dense.?
Failed example:
    A.eigenvalues(algorithm='symmetric', tol=1.0e-5)  # tol 2e-15
Expected:
    [(-8.0, 22), (2.0, 77), (22.0, 1)]
Got:
    [(-13.81753974166025, 1),
     (-13.648124559508018, 1),
     (-12.020844694797459, 1),
     (-11.501673730831273, 1),
     (-10.996735991707883, 1),
     (-10.034951645118767, 1),
     (-9.962283093579666, 1),
     (-9.128876066618123, 1),
     (-8.996148594502737, 1),
     (-8.722692543787009, 1),
     (-8.604556296665711, 1),
     (-8.40884373343103, 1),
     (-8.374989428968158, 1),
     (-8.04552871933626, 1),
     (-8.0, 13),
     (-7.526372396375868, 1),
     (-6.831403963944408, 1),
     (-6.629587466119391, 1),
     (-6.4982355637201605, 1),
     (-6.265676025803857, 1),
     (-6.234896753104995, 1),
     (-5.716331549376318, 1),
     (-5.390377899637494, 1),
     (-5.264189370979771, 1),
     (-4.717055974115992, 1),
     (-4.342049462991852, 1),
     (-4.035470777620305, 1),
     (-3.759935252697173, 1),
     (-3.3078409444345485, 1),
     (-2.9815285949726116, 1),
     (-2.4404128190223333, 1),
     (-1.8924099853815357, 1),
     (-1.844087730193682, 1),
     (-1.4442736849343316, 1),
     (-0.9294180296023848, 1),
     (-0.8110452755504824, 1),
     (-0.5222794367105574, 1),
     (0.20873807157486873, 1),
     (0.3951954399812892, 1),
     (0.8759590236699589, 1),
     (1.4381479600752756, 1),
     (1.532163686397235, 1),
     (1.7434392786321635, 1),
     (1.8865575972656636, 1),
     (2.0000000193635405, 14),
     (2.265318402262459, 1),
     (2.5658867016601037, 1),
     (2.817713598035522, 1),
     (3.081870153164812, 1),
     (3.446850975471049, 1),
     (3.6512306004984585, 1),
     (3.878710063182693, 1),
     (4.014287691943725, 1),
     (4.360367867192431, 1),
     (4.7229340629669, 1),
     (4.982801964690831, 1),
     (5.37818178439969, 1),
     (6.080935570154745, 1),
     (6.348181367606261, 1),
     (6.525599712412862, 1),
     (6.915551936720855, 1),
     (7.3044283658318365, 1),
     (7.452144258588678, 1),
     (7.5567933585381555, 1),
     (7.8716696840779745, 1),
     (7.937771590598039, 1),
     (8.114444731912405, 1),
     (8.195204412460074, 1),
     (8.780171598716006, 1),
     (10.615868120675568, 1),
     (12.063743301519922, 1),
     (12.935140199920008, 1),
     (12.961615404888553, 1),
     (17.06551630288574, 1),
     (21.999999999999986, 1)]
**********************************************************************
2 items had failures:
   1 of  49 in sage.matrix.matrix_double_dense.Matrix_double_dense.?
   1 of  42 in sage.matrix.matrix_double_dense.Matrix_double_dense.condition
    [629 tests, 2 failures, 2.69 s]

src/sage/numerical/sdp.pyx
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/numerical/sdp.pyx", line 125, in sage.numerical.sdp
Failed example:
    x=p.solve(); x+1
Exception raised:
    Traceback (most recent call last):
      File "/nix/store/9kjg3zzx8rfd3vs716scs2f4k8fx4qvg-python3-3.8.8-env/lib/python3.8/site-packages/sage/doctest/forker.py", line 720, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/nix/store/9kjg3zzx8rfd3vs716scs2f4k8fx4qvg-python3-3.8.8-env/lib/python3.8/site-packages/sage/doctest/forker.py", line 1153, in compile_and_execute
        exec(compiled, globs)
      File "", line 1, in
        x=p.solve(); x+Integer(1)
      File "sage/numerical/sdp.pyx", line 964, in sage.numerical.sdp.SemidefiniteProgram.solve (build/cythonized/sage/numerical/sdp.c:8297)
        self._backend.solve()
      File "sage/numerical/backends/cvxopt_sdp_backend.pyx", line 160, in sage.numerical.backends.cvxopt_sdp_backend.CVXOPTSDPBackend.solve (build/cythonized/sage/numerical/backends/cvxopt_sdp_backend.c:3423)
        raise SDPSolverException("CVXOPT: Terminated early due to numerical difficulties or because the maximum number of iterations was reached.")
    sage.numerical.sdp.SDPSolverException: CVXOPT: Terminated early due to numerical difficulties or because the maximum number of iterations was reached.
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/numerical/sdp.pyx", line 130, in sage.numerical.sdp
Failed example:
    p.slack(0).trace()        # tol 1e-14
Expected:
    1.0
Got:
    -0.0550587074717638
Tolerance exceeded:
    1.0 vs -0.0550587074717638, tolerance 2e0 > 1e-14
**********************************************************************
1 item had failures:
   2 of  46 in sage.numerical.sdp
    [288 tests, 2 failures, 0.88 s]

src/sage/tests/books/computational-mathematics-with-sagemath/float_doctest.py
**********************************************************************
File "/nix/store/pqarg7pym821rdg3bnyvmkmn59xh8vjp-sage-src-9.2/src/sage/tests/books/computational-mathematics-with-sagemath/float_doctest.py", line 402, in sage.tests.books.computational-mathematics-with-sagemath.float_doctest
Failed example:
    NearlySingularMatrix(RDF,n).det() == 0.0
Expected:
    True
Got:
    False
**********************************************************************
1 item had failures:
   1 of 142 in sage.tests.books.computational-mathematics-with-sagemath.float_doctest
    [141 tests, 1 failure, 40.11 s]

Full log: https://gist.github.com/omasanori/fb150eca7e3261118eebff56fe8d393c

Heads up: with a single user installation, the store is actually readwrite, and python likes to corrupt the store by compiling pyc files inside it. This can cause strange side effects.

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 114881 run on x86_64-linux 1

14 packages built:

arb
giac
giac-with-xcas
jmol
palp
python38Packages.cypari2
python38Packages.cysignals
python38Packages.fpylll
python38Packages.pplpy
python39Packages.cypari2
python39Packages.cysignals
python39Packages.fpylll
python39Packages.pplpy
sympow

Heads up: with a single user installation, the store is actually readwrite, and python likes to corrupt the store by compiling pyc files inside it. This can cause strange side effects.

Would nix-store --repair --verify --check-contents fix that?

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
 Thanks for the review. I'll make another PR updating cypari2, cysignals, eclib and pari as soon as upstream finishes up the pari upgrade.

So I think this is ready if it passes tests.

it passes tests so let me merge.

Thank you for your efforts to keep the sage package running and up to date :)


Ldrestart doesn’t work on 1.2.4-2 ?

Hi,
Ldrestart doesn’t work, there is no action when I confirm I want to reboot. Is it just me ? (I’m on unc0ver 3.1.0- iPhone10,6-12.1)

it wont work on my a9 ios 12.4