CLOUDSTACK-8339: Allow non-root users to add KVM host

This allows non-root users to add KVM hosts, the user should be an admin or
added to sudoers to execute sudo cloudstack-setup-agent.

@bhaisaab Nice! The code LGTM, will give it a test-drive today or tomorrow

Note you still have to add a sudoers entry for the user to run cloudstack-setup-agent.
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Allow+non-root+user+to+add+KVM+hypervisor

Oh, I see the original post mentions that!

@bhaisaab, I tested your patch and ran to an issue where a TTY is required to run sudo, but not requested. The host cannot be added.
Logs:
2015-05-24 13:48:44,175 DEBUG [c.c.u.s.SSHCmdHelper] (ApiServer-8:ctx-0b7d379a ctx-5e04fa3a) sudo cloudstack-setup-agent  -m <IP_ADDRESS> -z 1 -p 1 -c 1 -g 57acd541-cd37-34ea-af6a-dc6ecd007325 -a --pubNic=cloudbr0 --prvNi
c=cloudbr0 --guestNic=cloudbr0 --hypervisor=kvm output:sudo: sorry, you must have a tty to run sudo

Because it always uses 'sudo' you cannot add a host as 'root' nor as other user. To test:
[root@cs1 ~]# ssh root@kvm1 sudo whoami
root@kvm1's password:
sudo: sorry, you must have a tty to run sudo

Two possible solutions:

Specify -t parameter or similar to request a TTY:

[root@cs1 ~]# ssh -t root@kvm1 sudo whoami
root@kvm1's password:
root
Connection to kvm1 closed.

I prefer this solution since we then solve it on the CloudStack side. Alternatively, one can remove the TTY requirement from /etc/sudoers file:
Defaults    requiretty

I've continued my tests with altering the /etc/sudoers file and then I can add the host as user 'root' again, and also as user 'cloud' that I added to the /etc/sudoers file to allow it root without a password.
Result:
[root@kvm2 ~]# ps aux | grep cloud
root     23044  0.7  0.0 133128  4748 ?        Ss   14:03   0:00 sshd: cloud [priv]
cloud    23048  0.0  0.0 133128  2216 ?        S    14:03   0:00 sshd: cloud@notty
root     23062  0.0  0.0 188720  2588 ?        Ss   14:03   0:00 sudo cloudstack-setup-agent -m <IP_ADDRESS> -z 2 -p 2 -c 3 -g 57acd541-cd37-34ea-af6a-dc6ecd007325 -a --pubNic=cloudbr0 --prvNic=cloudbr0 --guestNic=cloudbr0 --hypervisor=kvm
root     23073  0.8  0.1 199392  9740 ?        S    14:03   0:00 /usr/bin/python /bin/cloudstack-setup-agent -m <IP_ADDRESS> -z 2 -p 2 -c 3 -g 57acd541-cd37-34ea-af6a-dc6ecd007325 -a --pubNic=cloudbr0 --prvNic=cloudbr0 --guestNic=cloudbr0 --hypervisor=kvm

Could you see if you can add a TTY request to your 'sudo cloudstack-setup-agent' call?
Apart from this: works-like-a-sharm :-)

Hi @remibergsma
If you're running into the no tty issue, that will likely happen also for the other commands the management server tries to run.
I'm using this file in /etc/sudoers.d in all environments:
{code}
Cmnd_Alias CLOUDSTACK = /bin/mkdir, /bin/mount, /bin/umount, /bin/cp, /bin/chmod, /usr/bin/keytool, /bin/keytool
Defaults:cloud !requiretty
cloud   ALL=(root) NOPASSWD:CLOUDSTACK
{code}
I believe this is the best way to resolve it for all distros/configs, as long as we depend on sudo

Silly me.. this is running as supplied user, not as cloud. I agree remi, ssh -t is the better option :)

Why don't we make a dedicated user (be it 'cloud' or anything) during agent
install time?
Makes a lot more sense to me, than relying on the usage of 'root', albeit
this PR is a step in the right direction.
--
Erik
On Sun, May 24, 2015 at 11:25 PM, Rafael da Fonseca <
<EMAIL_ADDRESS>wrote:

Silly me.. this is running as supplied user, not as cloud. I agree remi,
ssh -t is the better option :)
—
Reply to this email directly or view it on GitHub
https://github.com/apache/cloudstack/pull/288#issuecomment-105063069.

Thanks for reviewing everyone. Regarding the tty/sudo issue, I'll test and see if we can fix something in the agent post installation or setup scripts similar to what we do with the management server.
While we can create a user 'cloud', it might be a case that sysadmins would want to use some other user they may have created manually or using ansible/chef/puppet etc and configure that user with suitable options in sudoers file. I like Remi's idea which I'll explore.
Comments, thoughts?

Creating a user for this use would be worse, because it would require some sort of 'standard' credential for ssh access. Or it would require that such account didn't have ssh access and supplied account would need to sudo su account to use It to run the script.. so no advantage.
I think that the best way to tackle this would be to use the actual agent for registration. Letting the agent startup without config and listen on tcp port for remote registration (no ssh or sudo) and just process the credentials sent and self configure.
I can work on this if you'd like :)

On Mon, May 25, 2015 at 12:50 AM, Rafael da Fonseca <
<EMAIL_ADDRESS>wrote:

Creating a user for this use would be worse, because it would require some
sort of 'standard' credential for ssh access. Or it would require that such
account didn't have ssh access and supplied account would need to sudo su
account to use It to run the script.. so no advantage.

We do that already, by requiring the usage of 'root'.
Wouldn't need to be anything standard, setting the password could be done
during agent install, by asking or generating something random and printing
it to stdout -- heck, the mgmt server could even change to something random
during the first setup.

I think that the best way to tackle this would be to use the actual agent
for registration. Letting the agent startup without config and listen on
tcp port for remote registration (no ssh or sudo) and just process the
credentials sent and self configure.

The mgmt server tries to ssh and start the agent if it's registered down
for some reason.
--
Erik

For now, I think the quick win is to make the proposed use of 'sudo' work, so that works what worked before (we require root credentials to add a host) and alternatively one can specify other credentials with proper sudo config pre-setup (which we should document properly). Then we can merge this PR. I actually think this is a great improvement!
Next, let's discuss a redesign of the agent. I've heard many people doing all sorts of suggestions and I can think of some myself as well. It might even be part of a bigger architecture change. Let's use a new thread for that.

The issue now is that root credentials for the host are stored in the db, and even echoed back if you ask to list hosts with details. It's a huge step forward to have a single user who only has access to run cloudstack-setup-agent. Perhaps you don't have the same root everywhere, but you don't have to have the same non-root user/password everywhere, either. You can even remove/disable this user after the host is added.
In addition, many places don't allow root to SSH as a basic infosec policy.
I'd like us to get out of the business of dictating  and conflicting with configuration management, where possible, including things suggested like changing passwords upon agent setup. These things will rub big shops who have their config management story together the wrong way, and we already do way too much (like touching iptables and libvirt settings) IMO. They also confuse newbies more than they help ("why can't I log in anymore?", and I've heard a lot of "why did my networking get screwed up and restart?" from the existing setups scripts).
Also re: agent listening, I don't think  we want the agent to listen. It has been a huge boon to have all ports closed on the hypervisor except SSH, everywhere I've gone, from an infosec standpoint. I'm also not sure there's a meaningful gain by having the mgmt server contact the host via a listening agent port vs contacting the host via SSH and a user who has no access other than to run setup. I'm also more confident that SSH would pass a penetration test.
An easy fix that I think would accommodate everyone at this point is to only use 'sudo' if the user passed in is not 'root'. Then everyone doing it the current way is unaffected by the sudo and requiretty, and people who want to only pass non-root credentials to cloudstack mgmt server can do that as well with a sudoers line for cloudstack-agent-setup and a -t for the sudo (or perhaps just document the requiretty along with the sudoers line).

I'm focussing on getting this sorted out so we can merge.
@mlsorensen I like the idea of not changing anything in case the username is 'root'.
Did some reading on 'require tty' security implications and I'm quite sure it doesn't matter much in our case. We could as well just document it then.
To wrap it up:

Let's make sure sudo is NOT prepended when the username equals 'root'. This way we ensure the current behaviour is unchanged and does not require sudo
Since we require a sudo config anyway, let's document it properly. The minimal version I tested (assuming a non-privileged user cloudstack, can be any user):

cloudstack ALL=NOPASSWD: /usr/bin/cloudstack-setup-agent
defaults:cloudstack !requiretty

This removes the 'requiretty' only for the cloudstack user so it does not change anything for existing sudo configs. Once we agree, I'll take care of documenting it.
@bhaisaab Could you implement sudo only if non-root user please? This would then be the only improvement before we can merge this PR. Let me know if I can help other than testing it again.
Do we all agree this is a great step we can take now?
Apart from this we can discuss other improvements. I just don't want that to stop this small improvement with great value.

Thanks for commenting @mlsorensen @remibergsma
In the new fix we're maintaining old behavior for root users by not adding sudo, I've updated the install docs with Remi's sudoers config https://github.com/apache/cloudstack-docs-install/commit/1bf8b0b261f4e80b64450b1b002804e45b61a24b

@bhaisaab Awesome, will give it a final test run soon!

@remibergsma thanks, I tested locally and it seems to be working for me

@bhaisaab Tested with non-privileged user (now uses sudo) and with root user (does not use sudo) and both just add the host as one would expect.
As user 'cloudstack':
May 25 07:29:00 kvm2 sshd[6938]: Accepted password for cloudstack from 192.168.x.x port 39542 ssh2
May 25 07:29:00 kvm2 sshd[6938]: pam_unix(sshd:session): session opened for user cloudstack by (uid=0)
May 25 07:29:03 kvm2 sudo: cloudstack : TTY=unknown ; PWD=/home/cloudstack ; USER=root ; COMMAND=/bin/cloudstack-setup-agent -m 192.168.x.x -z 2 -p 2 -c 3 -g 57acd541-cd37-34ea-af6a-dc6ecd007325 -a --pubNic=cloudbr0 --prvNic=cloudbr0 --guestNic=cloudbr0 --hypervisor=kvm
May 25 07:30:04 kvm2 sshd[6942]: Received disconnect from <IP_ADDRESS>: 11: Closed due to user request.
May 25 07:30:04 kvm2 sshd[6938]: pam_unix(sshd:session): session closed for user cloudstack

As user 'root':
May 25 07:33:16 kvm2 sshd[7376]: Accepted password for root from <IP_ADDRESS> port 39543 ssh2
May 25 07:33:16 kvm2 sshd[7376]: pam_unix(sshd:session): session opened for user root by (uid=0)
May 25 07:34:21 kvm2 sshd[7376]: Received disconnect from <IP_ADDRESS>: 11: Closed due to user request.
May 25 07:34:21 kvm2 sshd[7376]: pam_unix(sshd:session): session closed for user root

Since you also confirmed it works as expected I'm now merging it. Thanks!
Please note: the PR is against 4.5, we also need to get this to master..

For documentation purposes: the PR is merged and commit is here in 4.5: https://github.com/apache/cloudstack/commit/d2b0c1a32b2d8719eafd6d4574de9a824d46d290

Thanks @remibergsma I'll merge it on both 4.5 and master. On master, I'll need to fix the packaging.

Merged on 4.5, so closing.


[SPARK-20399][SQL][Follow-up] Add a config to fallback string literal parsing consistent with old sql parser behavior

What changes were proposed in this pull request?
As @srowen pointed in https://github.com/apache/spark/commit/609ba5f2b9fd89b1b9971d08f7cc680d202dbc7c#commitcomment-22221259, the previous tests are not proper.
This follow-up is going to fix the tests.
How was this patch tested?
Jenkins tests.
Please review http://spark.apache.org/contributing.html before opening a pull request.

cc @srowen @cloud-fan @gatorsmile

Test build #77147 has started for PR 18048 at commit b38aa71.

Test build #77148 has started for PR 18048 at commit 9af9caf.

ping @cloud-fan

retest this please

Test build #77218 has started for PR 18048 at commit 9af9caf.

Test build #77218 has finished for PR 18048 at commit 9af9caf.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77218/
Test PASSed.

Merged build finished. Test PASSed.

thanks, merging to master/2.2!

Thanks @cloud-fan and @srowen .


adding in prepare for the build

Please check if the PR fulfills these requirements

[x] Followed the Contributing guidelines.
[x] Tests for the changes have been added (for bug fixes/features) with 100% code coverage.
[x] Docs have been added / updated (for bug fixes / features)

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
adding in prepare for the build

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (d4f354b) 100.00% compared to head (1a37c2c) 100.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff            @@
##            master      #618   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            5         5
  Lines          295       295
  Branches        76        76
=========================================
  Hits           295       295

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


drive: Recommendation for creating own client ID

What is the purpose of this change?
Adding informational message to drive when creating remotes to suggest creation of their own client-id rather than using rclone's default one.
Was the change discussed in an issue or in the forum before?
https://forum.rclone.org/t/rclone-keeps-getting-error-reading-destination-directory-couldnt-list-directory-googleapi-error-403-rate-limit-exceeded-ratelimitexceeded/8885/8
Checklist

[x] I have read the contribution guidelines.
[ ] I have added tests for all changes in this PR if appropriate.
[ ] I have added documentation for the changes if appropriate.
[ ] All commit messages are in house style.
[ X] I'm done, this Pull Request is ready for review :-)

That looks great - thank you :-)


Provide a way of using Organist without flakes

Some people (see https://github.com/nickel-lang/organist/pull/134 by @BurNiinTRee for example) want to use Organist without flakes.
This PR:

documents a way for doing that and adds a test for it
cleans up lib.nix arguments so that pkgs.callPackage can subsitute all of them
cleans up importNcl interface so that it's easy to use with just a path to a directory

@BurNiinTRee I've changed importNcl interface here, so you'll have to change how you use it, but I think this way is cleaner. Please take a look.

It's great outside of flakes, but the nickel-res.json build fails when using it from a flake with a  nickel.lock.ncl present due to the imported path to lib/lib.ncl not being present in the sandbox.
see this gist

@BurNiinTRee It looks like making this work without flake-compat would require passing the root of the Organist sources to callPackage, because ${../.} that I'm using here creates a copy of sources in Nix store with different path, so lockfile will always be off. I've extracted the fix for running without nickel.lock.ncl to #137 so that we can merge it separately.
What do you think about using flake-compat?

Depends on #137.
I've added back flakeRoot with name organistSrc, so user will have to either use flakes or provide fetched tarball, which should match Organist flake and thus keep matching lockfile

Oh, for some reason shells without flakes don't really work now... All checks other than bash --version fail. Will investigate.

Forgot --command bash for new nix develop lines. Should be all better now! 🎉

This seems to work in all really well. Thank you :)
I'm not sure if we misunderstood, but what wasn't working before was using it from a flake with the lockfile, it worked well outside of the flake with and without the lockfile and from a flake without or with an outdated lockfile.
But now everything i threw at it worked as expected

@BurNiinTRee Before it wasn't working if lockfile on disk matched one that was calculated. For non-flake, it was calculating it wrong, so it wasn't failing if it existed. Now it should be calculating it in the same way for both flake and non-flake, and with #137 not fail what it matches.


feat: expose createServerModuleRunnerTransport

Description
Since we can't just provide a fetchModule function anymore, let's expose the helper.

I think we need to expose NormalizedServerHotChannel type. Otherwise, users would have to use as any.
https://github.com/vitejs/vite/blob/c45576ab18c42ce950ba76f0f389ad3dde2ab012/packages/vite/src/node/ssr/runtime/serverModuleRunner.ts#L131


Optimized function schema message for ToolMetadata

Description
Let's say we define a super simple schema
class MySchema(BaseModel):
    input: Optional[str]

Before
ToolMetadata.fn_schema_str returns:
{'title': 'FnSchema', 'description': 'Default tool function Schema.', 'type': 'object', 'properties': {'input': {'title': 'Input', 'type': 'string'}}}

Problem: It contain too many information such as title, and it's not a valid Json, because it simply produced by str(model.schema())
After
ToolMetadata.fn_schema_str returns:
{"type": "object", "properties": {"input": {"title": "Input", "type": "string"}}}

Usage Pattern
use it in query_engine
query_engine_tools = [
    RetrieverTool(
        retriever=vs_retriever,
        metadata=ToolMetadata(
            name="useless tool",
            description="super useless",
             fn_schema=MySchema
        ),
    ),
]

when we prompt tool to LLM by
def get_react_tool_descriptions(tools: Sequence[BaseTool]) -> List[str]:
    """Tool."""
    tool_descs = []
    for tool in tools:
        tool_desc = (
            f"> Tool Name: {tool.metadata.name}\n"
            f"Tool Description: {tool.metadata.description}\n"
            f"Tool Args: {tool.metadata.fn_schema_str}\n"
        )
        tool_descs.append(tool_desc)
    return tool_descs

tool.metadata.fn_schema_str is the description of schema that LLM use to understand how to call the tool
Type of Change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[x] Added new unit/integration tests
[ ] Added new notebook (that tests end-to-end)
[x] I stared at the code and made sure it makes sense

Suggested Checklist:

[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added Google Colab support for the newly added notebooks.
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] I ran make format; make lint to appease the lint gods

Does this break compatibility with OpenAI tool/function calling though?

Does this break compatibility with OpenAI tool/function calling though?

Reference:openai documentation
Yes, the three key I kept is based on above document in function.parameters
tools = [
  {
      "type": "function",
      "function": {
          "name": "get_current_weather",
          "description": "Get the current weather",
          "parameters": {
              "type": "object",
              "properties": {
                  "location": {
                      "type": "string",
                      "description": "The city and state, e.g. San Francisco, CA",
                  },
                  "format": {
                      "type": "string",
                      "enum": ["celsius", "fahrenheit"],
                      "description": "The temperature unit to use. Infer this from the users location.",
                  },
              },
              "required": ["location", "format"],
          },
      }
  }

@logan-markewich can I get merged?
otherwise phoenix callback gets error, this PR fits it
**************************************************
Traceback (most recent call last):
  File "/Users/azure/Documents/Workspace/Datasci/lib/python3.10/site-packages/phoenix/utilities/error_handling.py", line 34, in wrapper
    return func(*args, **kwargs)
  File "/Users/azure/Documents/Workspace/Datasci/lib/python3.10/site-packages/phoenix/trace/llama_index/callback.py", line 266, in on_event_start
    payload_to_semantic_attributes(event_type, payload),
  File "/Users/azure/Documents/Workspace/Datasci/lib/python3.10/site-packages/phoenix/trace/llama_index/callback.py", line 194, in payload_to_semantic_attributes
    attributes[TOOL_PARAMETERS] = tool_metadata.to_openai_function()["parameters"]
KeyError: 'parameters'

Hi, Is there any update on this PR? I think this looks good. THe current implementation is not producing the json format.

@azurewtl wow sorry, I let this PR slip. Thanks for the reminder @sandangel
Just tested this out a bit with a few various tools (LLMTextCompletionProgram, ReActAgent), seemed to work well with both gpt-3.5-turbo and gpt-3.5-turbo-instruct


Group common parameters for Architecture deployment

A series of parameters are mandatory in all cases. Let's avoid
duplication and group them in a single file.
That single file is then loaded if and only if we set the
cifmw_architecture_scenario parameter to a non-empty string.
As noted in the va-common.yml file, ANY parameter there can be
overridden by a parameter file, or even using -e foo=bar directly.
This should make "creating new VA or DT" process slightly easier for
everyone, while providing a unified way to fix all the scenarios in case
of new parameter/feature/others.
As a pull request owner and reviewers, I checked that:

[X] Appropriate testing is done and actually running

/approve
/lgtm


ImportError: numpy.core.multiarray failed to import / ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):

mbploreto:webservice loretoparisi$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.14.5
BuildVersion:	18F132

Modin version (modin.__version__):

ip-192-168-1-168:script loretoparisi$ python3 -c "import modin;print(modin.__version__);print(modin.__file__)";
0.7.3
/usr/local/lib/python3.7/site-packages/modin/__init__.py

Python version:

ip-192-168-1-168:script loretoparisi$ python3 --version
Python 3.7.7

Numpy version:

ip-192-168-1-168:script loretoparisi$ python3 -c "import numpy;print(numpy.__version__);print(numpy.__file__)";
1.15.4
/usr/local/lib/python3.7/site-packages/numpy/__init__.py

Code we can use to reproduce:

Describe the problem

Stacktrack:
E0613 12:29:02.072695 258487744 raylet_client.cc:74] Retrying to connect to socket for pathname /tmp/ray/session_2020-06-13_12-29-01_031250_17196/sockets/raylet (num_attempts = 1, num_retries = 5)
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
Traceback (most recent call last):
  File "dataset.stats.py", line 29, in <module>
    df = pd.read_csv(args.path, sep='\t')
  File "/usr/local/lib/python3.7/site-packages/modin/pandas/io.py", line 112, in parser_func
    return _read(**kwargs)
  File "/usr/local/lib/python3.7/site-packages/modin/pandas/io.py", line 125, in _read
    from modin.data_management.factories import BaseFactory
  File "/usr/local/lib/python3.7/site-packages/modin/data_management/factories.py", line 292, in <module>
    class ExperimentalPyarrowOnRayFactory(BaseFactory):  # pragma: no cover
  File "/usr/local/lib/python3.7/site-packages/modin/data_management/factories.py", line 294, in ExperimentalPyarrowOnRayFactory
    from modin.experimental.engines.pyarrow_on_ray.io import PyarrowOnRayIO
  File "/usr/local/lib/python3.7/site-packages/modin/experimental/engines/pyarrow_on_ray/io.py", line 16, in <module>
    from modin.experimental.engines.pyarrow_on_ray.frame.data import PyarrowOnRayFrame
  File "/usr/local/lib/python3.7/site-packages/modin/experimental/engines/pyarrow_on_ray/frame/data.py", line 17, in <module>
    from .partition_manager import PyarrowOnRayFrameManager
  File "/usr/local/lib/python3.7/site-packages/modin/experimental/engines/pyarrow_on_ray/frame/partition_manager.py", line 15, in <module>
    from .axis_partition import (
  File "/usr/local/lib/python3.7/site-packages/modin/experimental/engines/pyarrow_on_ray/frame/axis_partition.py", line 15, in <module>
    from .partition import PyarrowOnRayFramePartition
  File "/usr/local/lib/python3.7/site-packages/modin/experimental/engines/pyarrow_on_ray/frame/partition.py", line 20, in <module>
    import pyarrow
  File "/usr/local/lib/python3.7/site-packages/pyarrow/__init__.py", line 49, in <module>
    from pyarrow.lib import cpu_count, set_cpu_count
  File "pyarrow/lib.pyx", line 40, in init pyarrow.lib
ImportError: numpy.core.multiarray failed to import

Source code / logs

### built-in
import argparse
import json

### dependencies
# parallel pandas - https://github.com/modin-project/modin
import modin.pandas as pd

if __name__ == "__main__":

    parser = argparse.ArgumentParser()
    parser.add_argument('--path', '-p', action='store', type=str, help='Dataset file path', default='', required=True)
    args = parser.parse_args()

    stats = {} # holds df stats
    stats['keys'] = {}

    df = pd.read_csv(args.path, sep='\t')

    # prepare data holder
    for col in df.columns:
        if col not in stats['keys']:
            stats['keys'][col] = {}

    # calculate stats
    for col in df.columns:
        stats['keys'][col]['count'] = len(df[col].groupby([df[col]]))
        stats['keys'][col]['unique'] = len(df[col].groupby([df[col]]).nunique())

    print(json.dumps(stats, indent=4))

Sorry, I don't see how you run the test example, could you please provide one? I'm also wondering do you really need to use pyarrow_on_ray engine?

@Garra1980 Not specifically for this case actually. By the way, now it works on macOS after I forced re-installed numpy and bumped to 1.18.5
ip-192-168-1-168:script loretoparisi$ python3 -c "import numpy;print(numpy.__version__);print(numpy.__file__)";
1.18.5

This is the final code
    stats = {} # holds df stats
    stats['keys'] = {}

    df = pd.read_csv(args.path, sep='\t')

    # prepare data holder
    for col in df.columns:
        if col not in stats['keys']:
            stats['keys'][col] = {}
            stats['count'] = len(df)

    # calculate stats
    for col in df.columns:
        stats['keys'][col]['count'] = len(df[col].groupby(df[col]))
        stats['keys'][col]['unique'] = len(df[col].groupby(df[col]).nunique())
        stats['keys'][col]['null'] = df[col].isnull().sum()

        # breakdown stats
        if col in args.breakdown.split(','):
            stats['keys'][col]['freq'] = {}
            freq = df[col].value_counts()
            for index,key in enumerate(freq.index.tolist()):
                stats['keys'][col]['freq'][key] = freq[index]


Compiler plugin added

Purpose
Compiler plugin used for the connector generation is added through this PR.
Goals
For the POC.
Approach
I have added model package with this in-order to save the data types and the classes needed the for the wso2-mi-connector.
Current PR supports the class definition nodes only. Each method that declared in the ballerina class generates a component. This plugin analyze the code and create the Xmls for the connector and zip the whole package using the mediator and other icons as dependencies.
User stories

Summary of user stories addressed by this change>

Release note

Brief description of the new feature or bug fix as it will appear in the release notes

Documentation

Link(s) to product documentation that addresses the changes of this PR. If no doc impact, enter “N/A” plus brief explanation of why there’s no doc impact

Training

Link to the PR for changes to the training content in https://github.com/wso2/WSO2-Training, if applicable

Certification

Type “Sent” when you have provided new/updated certification questions, plus four answers for each question (correct answer highlighted in bold), based on this change. Certification questions/answers should be sent to<EMAIL_ADDRESS>and NOT pasted in this PR. If there is no impact on certification exams, type “N/A” and explain why.

Marketing

Link to drafts of marketing content that will describe and promote this feature, including product page changes, technical articles, blog posts, videos, etc., if applicable

Automation tests

Unit tests

Code coverage information

Integration tests

Details about the test cases and coverage

Security checks

Followed secure coding standards in http://wso2.com/technical-reports/wso2-secure-engineering-guidelines? yes
Ran FindSecurityBugs plugin and verified report? no
Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets? yes

Samples

Provide high-level details about the samples related to this feature

Related PRs

List any other related PRs

Migrations (if applicable)

Describe migration steps and platforms on which migration has been tested

Test environment

List all JDK versions, operating systems, databases, and browser/versions on which this feature/fix was tested

Learning

Describe the research phase and any blog posts, patterns, libraries, or add-ons you used to solve the problem.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.samithkavishke seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

These changes will be added with the PR - #4


Trying to install an extension, but am getting an error command not found

Trying to install a vscode extension, but OP doesn't have a readme I've tried putting the folder in the:
user/.vscode/extension

but when I run the command I get the error:

command 'extension.resolveNamespace' not found

The repo is here:
https://github.com/ivonunes/vscode-php-namespace

Any help would be appreciated.

If you are not using the marketplace, you need to compile the extension yourself before you are able to use it.
After the checkout from within the project directory, run npm install first for getting the build tools and required libraries. After that, you can either run it at least one time in the graphical debugger (this triggers the build); or build it on the command line with: npm run-script vscode:prepublish.
That should be enough to make it work.
BTW: The correct directory to put extensions in is ~/.vscode/extensions (notice the additional 's' at the end).
Extra tip: You don't even have to put the git repo in there, a simple symlink to the project will do too.

 Everything is working.


App Licode does not work with https protocol

I have an Ubuntu 14.04 server, in this I have  Licode linckia WebRTC is already installed and running in https on port 3004, I have installed the ssl certificate and everything is going well on the web part, I can connect with other users, enter a room and record the conversation of the person requesting it. My problem is that when I try to access the webRTC through the Licode client Android app this does not connect and I get the following error:

And on the screen of the licode app I can not see anything, only a black screen.

Could someone help me fix it?

@evilword666  Have you implemented licode in android client?


upgrade to karma 0.13

We'll have to wait for new release of karma-angular-filesort. https://github.com/wilsonjackson/karma-angular-filesort/issues/1

testing.js
var Server = require('karma').Server;

function runKarma (singleRun, done) {
  new Server({
    configFile: path.join(__dirname, '/../karma.conf.js'),
    singleRun: singleRun,
    autoWatch: !singleRun
  }, done).start();
}


Unable to read .meteor/versions

I installed montiapm:agent, which is dependent on this package.
Our Meteor project is not directly in the root folder and that may cause this error:
[Unable to read .meteor/versions - ENOENT: no such file or directory, open './.meteor/versions'](zodern:meteor-package-versions: Unable to read .meteor/versions - ENOENT: no such file or directory, open './.meteor/versions')
Is there any fix for this or is this needed to run MontiAPM properly? Maybe some root folder check.

Our Meteor project is not directly in the root folder

This should be fine. I have many projects that are structured similarly.
This package expects that the meteor command is run from the root of the app. I had thought that Meteor requires that. Are you running meteor in a different folder? What would be the relative path to .meteor/versions from where you run meteor?

Yeah, the problem was exactly what you mention. meteor is running on a different folder. We changed our Dockerfile and now it seems to work! Thank you!

@saksholm would you mind sharing what folder meteor was run in relative to the app? If you ran into this issue, other people probably have or will too so it would be nice if we could fix it.

@saksholm would you mind sharing what folder meteor was run in relative to the app? If you ran into this issue, other people probably have or will too so it would be nice if we could fix it.

Sorry for the delay. Basically our Dockerfile looked like this:
FROM zcloudws/meteor-build:2.12 as builder

WORKDIR /build/source
USER root

RUN chown zcloud:zcloud -R /build

USER zcloud:zcloud
COPY --chown=zcloud:zcloud . /build/source

ENV METEOR_DISABLE_OPTIMISTIC_CACHING=1

WORKDIR /build/source/meteor

RUN meteor npm ci
RUN meteor build --directory ../../app-build

FROM zcloudws/meteor-node-mongodb-runtime:2.12
COPY --from=builder /build/app-build/bundle /home/zcloud/app

RUN cd /home/zcloud/app/programs/server && npm --no-audit install

WORKDIR /home/zcloud/app

ENV NO_NPM_INSTALL=1

ENTRYPOINT ["/scripts/startup.sh"]

We had other issues from the beginning with our repo and decided to make a custom Dockerfile. We got issues resolved and we stayed using the custom version. After this issue was faced, we switched the custom Dockerfile to default presets and this issue magically disappeared.


feat: [RHTAP-1406]  Adjusting failed pipelineRun log storing function

Description
This PR adjusts function for storing failed pipelineRun logs so they can be uploaded to report portal and changes moving folders with logs from "artifcts" dir to "rp_preproc" dir to only those matching report portal dir structure.
Issue ticket number and link
[RHTAP-1406]
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Checklist:

[x] I have performed a self-review of my code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added meaningful description with JIRA/GitHub issue key(if applicable), for example HASSuiteDescribe("STONE-123456789 devfile source")
[ ] I have updated labels (if needed)

/retest

/retest

/retest

/retest

/retest

/retest

/retest


Asterisk UniqueID

Hi,
I have a question regarding Asterisk and AsterNET. I am writing application to control call flow. I have Asterisk 12.
In my application I have a List of current calls. I match my calls with AsterNET events by UniqueID attribute. For example in Originate response I have UniqueID =<PHONE_NUMBER>.134 then I am cutting .134 from this number and save to my Call object. The number of<PHONE_NUMBER> becomes my UniqieID of call. I know that UniqueID is identifier of channel not call in Asterisk so in my call object I have List of ChannelUniqueID.
The problem: Sometimes asterisk changes UniqueID (the number before "." without no reason).
Standard (correct log flow):

Wrong log flow:

As You can see in the second call flow asterisk send Originate Response with UniqueID=1460280786.134 and after that sends events with UniqueID="1460280787.136".
Of course I know that the UniqueID can be changed for example after transfer the call
I am surprised that UniqueID is changing during one simple outgoing call.
Plis help :)
Regards
Peter

Doesn't sound right. I will look at some of my Originate code and see how I expect it to flow and come back to you.

@skrusty I would be very gratefull.
I'm looking forward for you code samples.
Regards
Peter

@skrusty I renew my request for examples of event handling of calls.
My company expects results in two weeks and I'm stuck in this problem.
Plis help .It will be the perfect gift for my birthday, which I have today :)
Regards
Peter

The UniqueId should not change unless the channel changes. I think you need to that this issue up in the Asterisk forums, as I can't see why the behaviour your seeing should happen.
forums.asterisk.org
Sorry


analyzer, server: partial symbol, import, and semantic analysis

This PR improves analysis efficiency by only analyzing the affected AST nodes through its line number. Import analysis is now executed if the affected node is an import statement and semantic analysis is executed if its otherwise.
This will hopefully alleviate concerns surrounding #360.

Nice! Great job!

thanks @medvednikov ❤  😃


How to determine coordinate system for server

Hi,
I'm trying to proxy a public website so I can pull it into Google Earth as a WMS overlay.
The site is https://qpublic.schneidercorp.com/Application.aspx?AppID=1081&LayerID=26490&PageTypeID=1&PageID=10767&KeyValue=01225-002-000 and seems to use a tile server located at https://tilecache.blob.core.windows.net/cache-18749/%(z)s/%(x)s/%(y)s .
I've tried accessing it using webmercator/mercator/geodetic and can't seem to get the tiles.  The urls being accessed don't match up to what the website pulls and mapproxy gets 404 when trying to access them.
Can you please suggest how I can figure out what grid system the server is using so that I can use it with map proxy?
Thank you

I think the source is in epsg 102660 https://epsg.io/102660.  When I configure map proxy it fails (see below).  Please advise as to how to fix this.  Thank you.
Traceback (most recent call last):
  File "/usr/local/bin/mapproxy-util", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/script/util.py", line 403, in main
    commands[command]['func'](args)
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/script/util.py", line 86, in serve_develop_command
    app = make_wsgi_app(mapproxy_conf, debug=options.debug)
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/wsgiapp.py", line 58, in make_wsgi_app
    services = conf.configured_services()
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/config/loader.py", line 212, in configured_services
    return self.services.services()
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/config/loader.py", line 1936, in services
    new_services = creator(service_conf or {})
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/config/loader.py", line 2100, in demo_service
    lyr = layer_conf.wms_layer()
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/util/py.py", line 78, in wrapper
    cache[key] = func(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/config/loader.py", line 1718, in wms_layer
    map_layer = self.context.caches[source_name].map_layer()
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/util/py.py", line 78, in wrapper
    cache[key] = func(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/config/loader.py", line 1634, in map_layer
    for grid, extent, tile_manager in self.caches():
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/util/py.py", line 78, in wrapper
    cache[key] = func(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/config/loader.py", line 1535, in caches
    sources, source_image_opts = self._sources_for_grid(
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/config/loader.py", line 1416, in _sources_for_grid
    source = source_conf.source({'format': request_format})
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/config/loader.py", line 986, in source
    grid = self.context.grids[grid_name].tile_grid()
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/util/py.py", line 78, in wrapper
    cache[key] = func(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/config/loader.py", line 301, in tile_grid
    grid = tile_grid(
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/grid.py", line 101, in tile_grid
    srs = SRS(srs)
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/srs.py", line 109, in SRS
    srs = _srs_impl(srs_code)
  File "/usr/local/lib/python3.9/dist-packages/mapproxy/srs.py", line 337, in __init__
    self.proj = CRS.from_epsg(epsg_num)
  File "/usr/lib/python3/dist-packages/pyproj/crs/crs.py", line 331, in from_epsg
    return CRS(_prepare_from_epsg(code))
  File "/usr/lib/python3/dist-packages/pyproj/crs/crs.py", line 296, in __init__
    super().__init__(projstring)
  File "pyproj/_crs.pyx", line 2302, in pyproj._crs._CRS.__init__
pyproj.exceptions.CRSError: Invalid projection: epsg:102660: (Internal Proj Error: proj_create: crs not found)

Please ask questions like this on the mailing list. The source looks proprietary and the URLs are scrambled, no clue which projection is used. Your best bet may be to ask the author of that site.


[Feature Request] Resize TCP socket buffer with TCP window scaling (RFC 1323)

Currently, the buffer for TCP socket needs to be preallocated in socket::tcp::Socket::new() and I found nowhere to change its size. However, a configurable window size is necessary to achieve a balance between good bandwidth and low memory footprint. I notice that TCP window scaling (RFC 1323) is advertised to be implemented but it seems it's only properly implemented for the remote endpoint. Can this feature be implemented in local side?

I'm not sure what feature you're requesting:

You can already use buffers any size you want.
TCP window scaling is fully implemented. if you set a buffer bigger than 65536, window scaling will be used to advertise the full buffer size to the remote side. (Also note window scaling is just a TCP extension allowing advertising window sizes larger than 16 bits, which is the window field size in the header. Window scaling by itself won't make the buffer bigger.)

Thanks for your replying. What I want to achieve is the following:

I don't know if the connection needs a big bandwidth initially, so I create a TCP socket with tcp::Socket::new(vec![0;65536], vec![0; 65536]) to reduce memory usage.
After some data exchange, I notice that we need a bigger receive window to satisfy BDP. Then I want to make the receive window to be 1MB. (Always allocating 1MB buffer for every socket at the beginning will lead to much bigger memory footprint)

Is this already possible?

ah okay! you want to dynamically change the buffer size after the TCP socket has already been created. Makes sense.
Unfortunately that's not possible now, no. Doing it if the buffer is borrowed might be hard, but should be doable if it's an owned Vec.

Then can we add such an API for sockets with owned Vec or Socket<'static> such as
fn replace_recv_buf<T: Into<SocketBuffer<'static>>(&mut self, new_buffer: T)->Option<SocketBuffer<'static>>

? Enabling this feature will be helpful to memory-constraint devices with skewed bandwidth needs.


How to send variable in a sequence

Signed-off-by: Meg McRoberts<EMAIL_ADDRESS>Captures info from a #help exchange
https://github.com/keptn/keptn.github.io/issues/1522

@grabnerandi sent the following in Slack: "I think its not worth adding this because the example was very specific to the user and their webhook. we are referring to specific tools they are using and i think that confuses people
3:15
sorry that you put that much effort in this - but - i suggest we dont add this as it would probably add more confusion unless we really find time to create a more generic descripton of this"


Image is not uploaded or shown in rich text box in voyager

Version information

Laravel: v5.5
Voyager: v1.0
PHP: 7.0
Database: [MySQL] [8.0]

Description:
I have a rich text box named service details. There I normally uploaded information and related images of services. But image is not uploaded or shown there. while I uploaded it shows like below image
Additional context

Got the solution !
My image src link(path) is not right that's why the image is not shown.
Just change APP_URL=http://localhost:8000 from APP_URL=http://localhost in .env file


gfortran: many versions

This is a thread to gather feedback.  I see we have gfortran in five versions ATM, all being built on Hydra (for the two Linux platforms at least): 4.8, 4.9, 5, 6 and the default 7.  I see 8 is missing; I tried 9 would now build by the trivial copy&paste, so I expect we'll add that, but first I suggest revisiting the usefulness of the older versions – perhaps we should prune at least the set being built on Hydra?
/cc random people that might be interested: @markuskowa, @ttuegel, @dtzWill.

I do not know how it came along that we have all the gfortran versions in the standard package set. Looking at pkgs/top-level/all-packages.nix it seems that they are set up in lock step with the gcc versions. Unless someone provides a reason that they are really needed to compile legacy code I am in favor of removing old versions. All packages that are in nixpkgs use the default version of gfortran (and seem to work fine with it). I would recommend removing at least versions 4.8, 4.9, 5 from the build set.

Not really a problem AFAIK, but assuming the builds are very unlikely to be useful, it seems quite a waste of resources.  For gcc I guess we might also remove some, but C(++) are far more widespread than fortran nowadays, so It seems harder to determine they won't be useful (e.g. the relatively fresh RHEL/CentOS 7 uses gcc 4.8 by default).

@vcunat FWIW I am using gcc 4.8 from nixpkgs to mimic the RHEL7-like environment. I also did use older versions of gfortran when I was suspecting a compiler regression, I did not find any, but I did appreciate the ability to do the check.

Wow, the message ordering is still messed up. The first message from me appears to be written from the future...

Time machine discovered by accident?  :thinking: I'm sure I've heard of such a story already.

Five years later:

gfortran (last updated 338ed797fe4a2)
gfortran48 (last updated 48f63c2f2e375)
gfortran49 (last updated 784025c747561)
gfortran6 (last updated 58e0c4bcd9d6c)
gfortran7 (last updated f03ee2475406a)
gfortran8 (last updated 2bac9a5a01bd1)
gfortran9 (last updated 7dc03548d1d61)
gfortran10 (last updated da59984d367f4)
gfortran11 (last updated 6ba5ac89d255c)
gfortran12 (last updated c34a8024bf5f8)
gfortran13 (last updated 4b8b00f56c2c2)
gfortran14 (last updated 75104490c0f17)


Problem installing guzzle6-adapter

Dear diffbot community:
I'm trying to install diffbot using composer,
1- Downloaded the latest version (2.0.1)
2- Rechecked the lines
"prefer-stable": true,
"minimum-stability": "dev"
in the composer.json
3- Executed the following
composer require php-http/guzzle6-adapter
Every thing seems to be ok until I get the following at the end of the installation
Package guzzle/guzzle is abandoned, you should avoid using it. Use guzzlehttp/guzzle instead.
Warning: Version check failed: Could not determine Puli version. "puli -V" returned:
(Tried minimum-stability as dev, beta, stable >>> not resolved)
So that I can't install diffbot-php-client

Looking into it, will have more info soon.

Please try again now, it should be cleared up. Here's a sample composer.json for reference:
{
    "minimum-stability": "dev",
    "prefer-stable": true,
    "require": {
        "php-http/guzzle6-adapter": "~1.0",
        "php-http/message": "^1.2",
        "php-http/discovery": "~0.8.0",
        "puli/cli": "~1.0",
        "php-http/plugins": "~1.0",
        "puli/composer-plugin":"^1.0",
        "swader/diffbot-php-client": "^2.1@dev"
    }
}


Siad exits without internet connection

Siad seems to exit if there's no internet connection. This is a problem because it makes the UI unusable.

Probably due to the gateway.
Clearly Sia is pretty useless without internet, but we should at least support viewing the wallet balance, and maybe mining.

This is no longer the case.


dsl: Add functionality to intersect SubDimensions

Introduce the ability to form a new SubDimension as the intersection of multiple SubDimensions

This needs bunch of tests including with MPI. Do we wanna define union as well?
Tests to come. Union is more complex and less urgent so was going to leave for now

Dead, closing


Broken ack/nack with pubsub emulator

Environment details

OS: OS X 10.12.x
Node.js version: 8.1.4
npm version: 5.0.3
google-cloud-node version: @google-cloud/pubsub 0.14.2
emulator version: up to date as of 2017-9-27

Steps to reproduce
The test case that shows the issue is as follows, it is however more specific than the generic test case needs to be, it is my exact use case.

Use emulator.
Use pull subscription.on to receive messages.
nack first execution of message for reprocessing.
ack the message on second execution.

Expected
The nack will cause it to be reprocessed.
The ack will cause it to be removed from the queue.
What happens
The ack is never processed.
Cause
subscription.connectionPool is present, but isConnected always returns false (unknown cause, I assume that the streaming system is not available in the emulator).
setFlushTimeout_ sets up a timer to handle the queue flush. This works correctly for the ack. However on the second go (the nack) the flush timeout handle is still set as it is not null-ified at any point. setFlushTimeout assumes this means there is an awaiting timeout and bails early.
Possible solution
Much like renewLeases_ does flushQueues_ should also null-ify it's timeout handle. Adding this one-liner fixed my test case.
I'd like to provide a PR however signing the corporate CLA is tricky for me, so I've provided everything verbally.

Ah apparently this is a duplicate of https://github.com/GoogleCloudPlatform/google-cloud-node/issues/2621
Alas only realized this after already figuring out the issue.


Wrong RSSI is displayed

Using example BLE scanner RSSI is displayed wrong
most time it is just '0' but once you see '-85' which is a reasonable value.

But I've also already seen '2' as value, which makes no sense to me

Of course device (Raspberry PI4) is working correctly and reporting reasonable RSSI data

I'm seeing similar issues, though perhaps not quite as bad as this. Using both the example scanner and my own program I've written that makes use of the py-bluetooth-utils functions, I am occasionally seeing positive numbers for RSSI.

Thank you for reporting this (and sorry for the delay), I will try to figure out what is occurring ;)

Thanks for looking into this.
@MS3FGX Which device do you use?

I just pushed a fix on master: afb68229e28db4d784a25078bc0afb6b41ca9783
In my case the values now are like the dbm values reported for eg. when running the btmon tool.
Please tell me if it is OK now ;)

@colin-guyon Looks good to me with the new changes. Just logged a little over 100 packets from three different Xiaomi LYWSD03MMC sensors and all the RSSI values were in reasonable range.

Thanks for the fix. Also works for me now.


Add landing page to the site

Change routing to go to a new Landing.jsx component

New landing page:

Other UI tweaks

going to remove schema.rb..

@tfrcarvalho cool, made all the requested changes I believe


Full support for text-emphasis in Chrome 99

https://bugs.chromium.org/p/chromium/issues/detail?id=666433#c38

Status: Fixed (was: Available)
Labels: Target-99

Looking at https://tests.caniuse.com/text-emphasis in Chrome 96 already suggests it knows left prefixed as opposed to note 1:

Going for draft however for now since my Chrome Canary 99 keeps crashing and https://wpt.fyi/results/css?label=master&label=experimental&aligned&q=text-emphasis isn't at 99 yet - I would expect 14 / 148 to go way up or rather the test here being way more green (since they link to the bug above).
Looking good:

The tests from WPT seem good in 99 as well, at least a few I checked.

Thanks!


Animation: camera setting lost while rotating multiple mesh objects

Hi @marcomusy,
I am playing with the Animation class and I found that my camera setting got lost while I try to rotate multiple mesh objects.
The code I used is something like this:
#!/usr/bin/env python3

import numpy as np
from vedo import TetMesh, show, screenshot, settings, Picture, buildLUT, Box, \
    Plotter, Axes
from vedo.applications import Animation

import time as tm

# Do some settings
settings.useDepthPeeling=False  # Useful to show the axes grid
font_name = 'Theemim'
settings.defaultFont = font_name
settings.multiSamples=8
# settings.useParallelProjection = True # avoid perspective parallax

# Create a TetMesh object form the vtk file
tet = TetMesh('final_mesh.vtk')

# Clone the TetMesh object and threshold it to get the conducotrs
cond = tet.clone().threshold(name='cell_scalars', above=0, below=15)

# Clone the TetMesh object and threshold it to get southern part of BSMT1
bm1 = tet.clone().threshold(name='cell_scalars', above=22, below=22)
# The southern part of BSMT2
bm2 = tet.clone().threshold(name='cell_scalars', above=24, below=24)

# This will get rid of the background Earth unit and air unit in the model
# which leaves us with the central part of the model
tet.threshold(name='cell_scalars', above=0, below=21)

# Crop the entire mesh using a Box object (which is considered to be a mesh
# object in vedo)
# First build a Box object with its centers and dimensions
cent = [504700, 6416500, -615]
box = Box(pos=cent, size=(3000, 5000, 2430))
# So, we now cut the TetMesh object with a mesh (that Box object)
tet.cutWithMesh(box, wholeCells=True)
# We can also cut msh directly rather than cutting tet, but that gives us
# something uglier, like what you would get without click the 'crinkle clip'
# option in Paraview and much worse because it would not keep mesh cells intact
# bounds = [503500, 505000, 6414000, 6417000, -1830, 600]
# msh.crop(bounds=bounds, wholeCells=True)

# And we need to convert it to a mesh object for later plotting
msh = tet.tomesh().lineWidth(1).lineColor('w')

# Crop the conductor TetMesh object using a larger box
cent = [505500, 6416500, -615]
box_cond = Box(pos=cent, size=(3200, 5000, 2430))
cond.cutWithMesh(box_cond, wholeCells=True)
cond_msh = cond.tomesh().lineWidth(1).lineColor('w')

# Also need to crop bm1 and bm2, but with a smaller box than that used for the
# entire mesh
cent = [504500, 6416500, -615]
box = Box(pos=cent, size=(1500, 5000, 2430))
bm1.cutWithMesh(box, wholeCells=True)
bm1_msh = bm1.tomesh().lineWidth(1).lineColor('w')
bm2.cutWithMesh(box, wholeCells=True)
bm2_msh = bm2.tomesh().lineWidth(1).lineColor('w')

# We need to build a look up table for our color bar, and now it supports
# using category names as labels instead of the numerical values
# This was implemented upon my request
lut_table = [
    # Value, color, alpha, category
    (12.2, 'dodgerblue', 1, 'C1'),
    (15.2, 'skyblue', 1, 'C1-North'),
    (16.2, 'lightgray', 1, 'Overburden'),
    (18.2, 'yellow', 1, 'MFc'),
    (19.2, 'gold', 1, 'MFb'),
    (21.2, 'red', 1, 'PW'),
    (23.2, 'palegreen', 1, 'BSMT1'),
    (25.2, 'green', 1, 'BSMT2'),
]
lut = buildLUT(lut_table)

msh.cmap(lut, 'cell_scalars', on='cells')
cond_msh.cmap(lut, 'cell_scalars', on='cells')
bm1_msh.cmap(lut, 'cell_scalars', on='cells')
bm2_msh.cmap(lut, 'cell_scalars', on='cells')
bm_msh = bm1_msh + bm2_msh
# msh.cmap("coolwarm", 'cell_scalars', on='cells')
msh.addScalarBar3D(
    categories=lut_table,
    pos=(508000, 6416500, -1830),
    title='Units',
    titleSize=1.5,
    sx=100,
    sy=4000,
    titleXOffset=-2,
)

group = msh + cond_msh + bm_msh
zlabels = [(500, '500'), (0, '0'), (-500, '-500'), (-1000, '-1000'),
           (-1500, '-1500')]
axes = Axes(group,
            xtitle='Easting (m)',
            ytitle='Northing (m)',
            ztitle='Elevation (m)',
            xLabelSize=0.015,
            xTitlePosition=0.65,
            yTitlePosition=0.65,
            yTitleOffset=-1.18,
            yLabelRotation=90,
            yLabelOffset=-1.6,
            # yShiftAlongX=1,
            zTitlePosition=0.85,
            # zTitleOffset=0.04,
            zLabelRotation=90,
            zValuesAndLabels=zlabels,
            # zShiftAlongX=-1,
            axesLineWidth=3,
            yrange=msh.ybounds(),
            xTitleOffset=0.02,
            # yzShift=1,
            tipSize=0.,
            yzGrid=True,
            xyGrid=True,
            gridLineWidth=5,
            )

# Set the camera position
plt = Animation()
plt.camera.SetPosition( [512921.567, 6407793.637, 8217.335] )
plt.camera.SetFocalPoint( [505099.133, 6415752.321, -907.462] )
plt.camera.SetViewUp( [-0.494, 0.4, 0.772] )
plt.camera.SetDistance( 14415.028 )
plt.camera.SetClippingRange( [7367.387, 23203.319] )

size = [3940, 2160]

# Trying to play with the Animation class
plt.showProgressBar = True
plt.timeResolution = 0.025  # secs

# It would be okay if I only plot one mesh object
# plt.fadeIn(msh, t=0, duration=0.5)
# plt.rotate(msh, axis="y", angle=180, t=1, duration=2)

# If I try to plot multiple objects, then things quickly go wrong: the mesh objects become a small point
# in the middle of the mesh and then disappear
plt.fadeIn([cond_msh, msh, bm1_msh, bm2_msh], t=0, duration=0.5)
plt.rotate([cond_msh, msh, bm1_msh, bm2_msh], axis="z", angle=180, t=1, duration=3)
plt.totalDuration = 4 # can shrink/expand total duration

plt.play()
# plt.show(msh, cond_msh, bm_msh, axes, size=size, interactive=False, resetcam=0, zoom=1.0)
# screenshot('model_mesh_vedo.png')

Here is the data required for this script to work:
final_mesh.vtk.zip
While I can try to just plot with one mesh object, it would still be interesting to make the above script work to add some more flexibility. I am not entirely sure whether I am doing something wrong or this is just the default behaviour. When you get a chance, could you please take a look at this?
Here is what things look like before they disappear (very last frame generated in the call to faceIn function):

And here is what I mean by 'disappear' (can still be visible at the bottom-right coner):

You can see that they are still there if you zoom in to that area:

Well, I just discovered that it does not matter how many objects I rotate. This would happen even if I just rotate one object.
Also, there is no way for me to add Axes objects because of the following error: AttributeError: 'Assembly' object has no attribute 'alpha'".
The added 3D scalar bar also disappears.
The .mp4 file cannot be viewed using QuickTime Player on Mac. I've tried another video player and it does not work either.
Thanks very much,
Xushan

I will check, my first guess is that rotate() rotates around the origin (0,0,0)  that's why everything goes off scale (you have LARGE scales!)

no way for me to add Axes objects because of the following error: AttributeError

this is actually a bug, needs fixing..

The added 3D scalar bar also disappears.

see #505

The .mp4 file cannot be viewed using QuickTime Player on Mac. I've tried another video player and it does not work either.

this is probably the ffmpeg encoder.... you can still save screenshot series and then join them (i use a lot https://ezgif.com/)

Thanks very much for the comment.

this is probably the ffmpeg encoder.... you can still save screenshot series and then join them (i use a lot https://ezgif.com/)

I guess I will just use the command-line ffmpeg. I was thinking of generating a series of screenshots but I decided to just use the Animation class but it did not work. I've already created a rotation video using just a single mesh object and recorded the screen while the movie is playing in the GUI. I think I can just use that one for the current task. I am happy to learn new tricks using vedo. Thanks very much for your help.

Hi @marcomusy, I just updated to 2021.0.7 and I don't see the video is played inside the GUI anymore?

..what GUI, you mean the rendering window?

Yes. Previously I can watch it playing in that window but now nothing shows up!

what is the output of
vedo  --info

vedo version      : 2021.0.7   https://vedo.embl.es
vtk version       : 9.0.1
python version    : 3.8.11 (default, Aug  6 2021, 08:56:27) [Clang 10.0.0 ]
python interpreter: /Users/xsl/opt/anaconda3/bin/python
vedo installation : /Users/xsl/opt/anaconda3/lib/python3.8/site-packages/vedo
system            : Darwin 21.1.0 posix x86_64

Are you sure it is still woking with vedo.2021.0.6 ? I noticed that on mac Monterrey it behaves differently..
What if you set
from vedo import settings
settings.allowInteraction=True

That solved the problem!

That only solved the problem of showing things in the GUI but still the mp4 file cannot be viewed.

I guess this is a problem of ffmpeg... cannot do much on that, you may use screenshot(f"file{i}.png") in the loop and then join the series with external software

Hi @marcomusy,
Ok, I've managed to have the different meshes in different sub-windows buy using the Plotter instead of using the animation class:
vp = Plotter(shape=[3, 3], axes=0, interactive=0)

    # vp.camera.Zoom(1.2)
    vp.interactive = False

    for t in np.arange(0, 1, 0.005):
        vp.show(mesh_models[0].c('gray'), at=0)
        vp.show(mesh_models[1].c('gray'), at=1)
        vp.show(mesh_models[2].c('gray'), at=2, resetcam=0)
        vp.show(mesh_models[3].c('gray'), at=3)
        vp.show(mesh_models[4].c('gray'), at=4)
        vp.show(m.c('gray'), at=5)
        vp.camera.Azimuth(2)

    vp.show(interactive=1)

but I have two questions, 1) do you know why I am getting these kind of flashing artifact on some of the pieces and 2) how I can save this directly as a .gif or .mp4 file? I couldn't find something on the Plotter class.

Hi

because you are not resetting camera the flashing I guess is due to the camera depth clipping.. try adding
vp.camera.SetClippingRange( [dmin, dmax] )

You can either vp.screenshot(f'filename{t}.png') or use the Video()class (which depends on your system specs so it might not work..). Check outexamples/other/makeVideo.py`

Ok, apparently for the 1. most likely is a fault of my desktop renderer since on the final file (using the Video()) the flashing issues doesn't show up:

For 2. Video() works perfectly ;-).

@marcomusy I am trying to save directly to .gif format. However, do you know which parameter to pass to address this yellow glow:

According to this and this it is due to the transparency but passing the command:
video = Video("tombstone2.gif", backend='ffmpeg')  # backend='opencv/ffmpeg'
video.options = "-b:v 8000k -vf format=rgba"

doesn't really help much. Then I tried to set the bg2='white' in the Plotter() initialization but it didn't work as well. While using the opencv backend gives me:
OpenCV: FFMPEG: tag 0x7634706d/'mp4v' is not supported with codec id 12 and format 'gif / CompuServe Graphics Interchange Format (GIF)'
[gif @ 0x55c302d1b880] GIF muxer supports only a single video GIF stream.

Uhm sorry I'm not an expert on this... if nothing works you can still save the separate png screenshots and merge them.
An online toll which I used a lot is https://ezgif.com/

Uhm sorry I'm not an expert on this... if nothing works you can still save the separate png screenshots and merge them. An online toll which I used a lot is https://ezgif.com/

Fair enough, no worries. Actually after a lot of trial and error I managed to have the following working and which also gives a good quality .gif file:
video = Video("tombstone.gif", backend='ffmpeg')  # backend='opencv/ffmpeg'
video.options = "-b:v 8000k -filter_complex \"[0:v] split [a][b];[a] palettegen=stats_mode=single [p];[b][p] paletteuse=new=1\""

Sources: [1] and [2]
Also I would suggest to comment/remove this line https://github.com/marcomusy/vedo/blob/444b759b95381039b1b754d53a58b1d047391ba5/vedo/io.py#L1909 since it hard codes the .mp4 extension to the output file.
Moreover, another thing I want to ask you is whether it is possible to individually and/or automatically adjust the camera zoom per window depending on the object's size. For example in the following output the first object is bigger that the other two so it appears closer to the camera (thus you also have the clipping) while the other two appear fine. The vp.camera.SetClippingRange( [dmin, dmax] ) command you mentioned above affects all the sub-windows.

Fair enough, no worries. Actually after a lot of trial and error I managed to have the following working and which also gives a good quality .gif file:

Looks great thanks for that.

Also I would suggest to comment/remove this line

Yes - removed

Moreover, another thing I want to ask you is whether it is possible to individually and/or automatically adjust the camera zoom per window depending on the object's size. For example in the following output the first object is bigger that the other two so it appears closer to the camera (thus you also have the clipping) while the other two appear fine.

Either you share the same camera or have separate ones of each renderer. This is controlled by plt = Plotter(sharecam=True).
If you have separate cameras you should be able to access them via:
cam = plt.renderer.GetActiveCamera() # vtkCamera obj

Fair enough, no worries. Actually after a lot of trial and error I managed to have the following working and which also gives a good quality .gif file:

Looks great thanks for that.

:+1:  possibly it could be connected also with the issue https://github.com/marcomusy/vedo/issues/210

Also I would suggest to comment/remove this line

Yes - removed

:+1:

Moreover, another thing I want to ask you is whether it is possible to individually and/or automatically adjust the camera zoom per window depending on the object's size. For example in the following output the first object is bigger that the other two so it appears closer to the camera (thus you also have the clipping) while the other two appear fine.

Either you share the same camera or have separate ones of each renderer. This is controlled by plt = Plotter(sharecam=True). If you have separate cameras you should be able to access them via: cam = plt.renderer.GetActiveCamera() # vtkCamera obj

:heart: worked like a charm:


Seems the kubectl has build-in debug sub command, it conflicts with it

kubectl debug <pod-name>
error: you must specify --image when not using --copy-to.

https://kubernetes.io/docs/tasks/debug-application-cluster/debug-running-pod/#ephemeral-container

@aylei
I have the same problem. This is my version of kubectl:
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:52:14Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.8-aliyun.1", GitCommit:"2cbb16c", GitTreeState:"", BuildDate:"2021-01-27T02:20:04Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.21) and server (1.18) exceeds the supported minor version skew of +/-1

yes, to workaround this call the binary directly, for example:
# start the debug container in the same namespace, and cgroup etc as container 'CONTAINER_NAME' in pod 'POD_NAME' in namespace 'NAMESPACE' kubectl-debug --namespace NAMESPACE POD_NAME -c TARGET_CONTAINER_NAME


Feature: multiple balanced assets

In GitLab by @fahree on Mar 11, 2020, 14:52
Add support for interactions that involve multiple assets, wherein the fact that assets are balanced at all points is guaranteed by the language. Some assets may be fungible, others not.
Stretch goal: have a common abstraction for goods and services, so that transfer of goods is a service, and services can be sold.
In gerbil-ethereum assets.ss, a method for setting up the "expected-to-be-deposited-so-far" variable, updating that variable on deposit, and committing that on a transaction? And a similar thing for setting up withdrawls to each of the participants, and committing those at the end of a transaction
Subtasks:

[x] Move block-context back to glow
[x] Multiple assets in block context

[x] Tracking balances of multiple assets in participant-runtime
[x] Tracking balances to "variables" in the evm-runtime

[x] Asset type with methods for committing transfers at the end of a transaction
[x] Add assets to InteractionAgreement
[x] Use ethereum-config network to filter the asset options in CLI
[x] Pass assets through CLI into participant-runtime and consensus-code-generation
[x] In runtime, set up asset environment (including block contexts)

[x] In runtime, set up asset environment
[x] including block contexts

[x] For deposit/withdraw, use asset-environment/block-contexts to track balances
[x] On participant committing transactions, check balances are valid

[x] Check deposit native balance for native assets
[x] Check deposit erc20 balances for erc20 assets
[x] Check withdraw contract balances (balance variable, not necessarily contract balance, for multiple interactions per contract)

[x] On participant committing transactions, transfers

[x] Attach CALLVALUE for native assets
[x] Check for zero, approve transfer-from for erc20 assets

[x] On consensus committing transactions, use asset-environment to deposit/withdraw the asset amounts for the transaction

[x] Check CALLVALUE for native asset deposits
[x] Check for zero, invoke transfer-from for erc20 asset deposits
[x] Check for zero, invoke transfer for native asset withdraws
[x] Check for zero, invoke transfer for erc20 asset withdraws

[x] Test all of this

Related branches:

gerbil-ethereum: https://github.com/AlexKnauth/gerbil-ethereum/tree/assets4
glow: https://gitlab.com/mukn/glow/-/commits/assets5

In GitLab by @fahree on Mar 11, 2020, 14:54
changed the description

In GitLab by @fahree on Nov 7, 2020, 03:42
assigned to @fahree

In GitLab by @ApolloUnicorn on Feb 13, 2021, 01:35
marked this issue as related to #115

In GitLab by @AlexKnauth on Mar 2, 2021, 04:04
changed the description

In GitLab by @AlexKnauth on Mar 2, 2021, 04:05
changed the description

In GitLab by @ApolloUnicorn on Mar 4, 2021, 00:15
assigned to @AlexKnauth

In GitLab by @ApolloUnicorn on Mar 4, 2021, 00:15
removed the relation with #115

In GitLab by @ApolloUnicorn on Mar 4, 2021, 00:15
marked this issue as related to #115

In GitLab by @AlexKnauth on Mar 9, 2021, 03:31
marked this issue as related to #106

In GitLab by @ApolloUnicorn on Mar 17, 2021, 23:37
unassigned @fahree

In GitLab by @ApolloUnicorn on Mar 17, 2021, 23:41
removed the relation with #106

In GitLab by @ApolloUnicorn on Mar 26, 2021, 23:28
@AlexKnauth would you please add subtasks to this ticket that outline what is left? I am having a hard time understand the steps/progress.

In GitLab by @AlexKnauth on Mar 31, 2021, 23:49
changed the description

In GitLab by @AlexKnauth on Apr 6, 2021, 02:07
changed the description

In GitLab by @AlexKnauth on Apr 6, 2021, 02:08
changed the description

In GitLab by @AlexKnauth on Apr 7, 2021, 23:39
marked the task Add assets to InteractionAgreement as completed

In GitLab by @AlexKnauth on Apr 7, 2021, 23:45
changed the description

In GitLab by @AlexKnauth on Apr 20, 2021, 02:34
marked the task Pass assets through CLI into participant-runtime and consensus-code-generation as completed

In GitLab by @AlexKnauth on Apr 21, 2021, 23:37
marked the task Asset type with methods for committing transfers at the end of a transaction as completed

In GitLab by @AlexKnauth on Apr 27, 2021, 02:07
changed the description

In GitLab by @AlexKnauth on Apr 28, 2021, 23:21
marked the task including block contexts as completed

In GitLab by @AlexKnauth on Apr 28, 2021, 23:21
marked the task In runtime, set up asset environment (including block contexts) as completed

In GitLab by @AlexKnauth on Apr 28, 2021, 23:21
marked the task For deposit/withdraw, use asset-environment/block-contexts to track balances as completed

In GitLab by @AlexKnauth on May 6, 2021, 01:37
changed the description

In GitLab by @AlexKnauth on May 8, 2021, 04:53
marked the task Use ethereum-config network to filter the asset options in CLI as completed

In GitLab by @AlexKnauth on May 19, 2021, 01:26
changed the description

In GitLab by @AlexKnauth on May 19, 2021, 23:23
changed the description

In GitLab by @ApolloUnicorn on May 20, 2021, 18:59
@AlexKnauth are there any of the checklist items that are actually complete? I know there was some pairing last week. I am trying to identify if we are blocked on any particular piece of the plan specified.

In GitLab by @ApolloUnicorn on May 20, 2021, 19:01
@fahree I believe this was integral to the May delivery, but it's not clear what this blocks/is blocked by. Is there a subset of the open subtasks above that could be segmented out for post-May deadlines?

In GitLab by @AlexKnauth on Jun 15, 2021, 23:08
marked the task Tracking balances to "variables" in the evm-runtime as completed

In GitLab by @AlexKnauth on Jun 15, 2021, 23:08
marked the task Multiple assets in block context as completed

In GitLab by @AlexKnauth on Jun 15, 2021, 23:14
changed the description

In GitLab by @AlexKnauth on Aug 4, 2021, 21:59
changed the description

In GitLab by @alex413 on Aug 17, 2021, 06:11
assigned to @isd

In GitLab by @isd on Aug 25, 2021, 06:46
marked the task Check withdraw contract balances (balance variable, not necessarily contract balance, for multiple interactions per contract) as completed

In GitLab by @isd on Aug 25, 2021, 06:46
marked the task Check deposit native balance for native assets as completed

In GitLab by @isd on Aug 25, 2021, 06:46
marked the task Check deposit erc20 balances for erc20 assets as completed

In GitLab by @isd on Aug 25, 2021, 06:46
marked the task On participant committing transactions, check balances are valid as completed

In GitLab by @isd on Aug 25, 2021, 06:46
marked the task Check for zero, approve transfer-from for erc20 assets as completed

In GitLab by @isd on Aug 25, 2021, 06:46
marked the task On participant committing transactions, transfers as completed

In GitLab by @isd on Aug 25, 2021, 06:47
marked the task Check for zero, invoke transfer-from for erc20 asset deposits as completed

In GitLab by @isd on Aug 25, 2021, 06:47
marked the task Check for zero, invoke transfer for native asset withdraws as completed

In GitLab by @isd on Aug 25, 2021, 06:47
marked the task Check for zero, invoke transfer for erc20 asset withdraws as completed

In GitLab by @isd on Aug 25, 2021, 06:47
marked the task On consensus committing transactions, use asset-environment to deposit/withdraw the asset amounts for the transaction as completed

In GitLab by @isd on Aug 25, 2021, 06:47
marked the task Test all of this as completed

In GitLab by @isd on Aug 25, 2021, 06:51
mentioned in merge request !168


Tell changesets to ignore docs app

Noticed on https://github.com/CrowdStrike/ember-headless-form/pull/81
that the docs app wasn't omitted from changeset

Does that mean we cannot create explicit changeset entries for the docs-app anymore, that we want in the changelog? In most repos (like release-it based ones) we have documentation labelled entries also!

Oh! I hadn't considered that.
The changelog entries would be in docs-app/CHANGELOG.md
If that's what you want feel free to close this pr.
I thought incruding the docs app mas a mistake 😅


Add Item card heading font & style inconsistent

 Issue by rstrahan
Tuesday May 01, 2018 at 13:03 GMT
Originally opened as https://github.com/rstrahan/aws-ai-qna-bot-2/issues/24

Expected Behavior
Sections and fields in the Add/Update item card should have consistent heading style and font..
Actual Behavior
Some are bold, some are grey.. see screenshot.
Or maybe it's that some fields have a heading defined (bold) and some do not.

field order need to be consitant as well


Crashing: JS/TS language features

Issue Type: Bug
JS/TS language features keep re-initialising on their own. Still happening on latest update.

No I haven't tried it in "Insiders build"
For official build, it crashes even when all the extensions are disabled. After April update the crashes are few (once or twice every 20 minutes). Before April update they were once every 2 minutes.

Do *.dmp files created a long time after a crash still relevant? Found that *.dmp files took an hour to get created. If they are still relevant, I can attach them here.

@LohitakshTrehan  If it helps, I was getting this same issue on Stable, too (even with extensions disabled). Found two of my settings potentially caused it to happen; after removing these from my local settings.json, the issue was resolved:
"javascript.validate.enable": false,
"typescript.disableAutomaticTypeAcquisition": true,

Do you have either of these same settings? If so, try removing them.
Also, the latest Insiders doesn't suffer from the same bug when I tested, even with those settings in place. So, if it's an option for you, try Insiders.

@LohitakshTrehan Please test in insiders
Also the dmp file only gets generated if VS Code itself crashes, not when a language server restarts so it's expected that none is generated in this case


Update xquartz to 2.7.11

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

Apologies, deleted my cask-repair branches by mistake.

Travis has them queued, GitHub hasn't changed from 🔴  to 💛 .
I'll try restarting them again in a couple of minutes.

Thanks - typed cask-repair -d instead of cask-repair -e, and of course the d and e keys are neighbours.


Broken Link

There is a broken link - https://docs.microsoft.com/en-us/data-factory/data-flow-assert.md
It's a Hyperlink under Data Integration -  "Assert transformation in mapping data flow article"

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4f8972d2-1aa1-05fb-d9ea-8a3ef322cebf
Version Independent ID: d390905a-874f-884e-683b-f9c5acd764e5
Content: What's new? - Azure Synapse Analytics
Content Source: articles/synapse-analytics/whats-new.md
Service: synapse-analytics
Sub-service: overview
GitHub Login: @ryanmajidi
Microsoft Alias: rymajidi

@mariusraileanu Thank you for sharing feedback. We are actively validating it. We will share updates soon.

The PR for the fix has been created and it will be merged soon. Thanks again for pointing this out, I am closing the issue based on this update, please let us know if there is something else that we can help you with.


[Regression] very low fps on Android with batching gles2

Godot version:

godot 3.1.dev 238a78f13
OS/device including version:

Android 8.0 / Galaxy S8+
Issue description:

FPS is very low like 5~10 on android
it works fine with reverting f55039b194bbbd8d797b667d67e5677fb429d356 (#20965)
it seems fine on desktop though.
Steps to reproduce:
Minimal reproduction project:

cc @karroffel @dragmz

@volzhs Can you attach or describe a reproduction project, please?

The GLES2 implementation is hard to test since there are other things not complete yet.
Could you provide 'logcat' logs and is this reproducible with a simple 'cubescene' ?

I can confirm this works much better on eef9c1f5b57cab09f50d0580034b1cf52f4d96eb (~57 FPS) vs f55039b194bbbd8d797b667d67e5677fb429d356 (~40 FPS) on my device.


Add a test for cargo locate-project

There seems to be no test for cargo locate-project, so I add the simple test case for this command.

r? @alexcrichton
(rust_highfive has picked a reviewer for you, use r? to override)

@bors: r+

:pushpin: Commit ba74297c20f0048082f7ad888a8bc80358686ce5 has been approved by alexcrichton

:hourglass: Testing commit ba74297c20f0048082f7ad888a8bc80358686ce5 with merge a4cc2736ee8f2a2d0192e848770f91466dee8c16...

:sunny: Test successful - checks-azure
Approved by: alexcrichton
Pushing a4cc2736ee8f2a2d0192e848770f91466dee8c16 to master...


Remove ADPLAN

ADPLAN is no longer available.
https://www.adplan7.com/ (~ March 31, 2021)

Thanks!


Document Azure services used and cost running an Azure TRE instance

Description
As an Azure Administrator
I want to understand what is provisioned when setting up a TRE instance
So that I can estimate the Azure cost
Acceptance criteria

[ ] Description of Azure services used for a TRE instance (core)
[ ] Description of Azure services used for Vanilla workspace instance

Would be great to have this considered as this is a need emerging for some of our customers.
Many thanks
Phil


fix: Show extra line on space items

Summary
As discussed on call,
Display extra line on space items to occupy more space
(using line-clamp-4 is actually occupying full empty space)
How to test

Go to explore page

Before
After

May need some truncation on the about text, beside clamping, else we may end up with https://testnet.snapshot.box/#/explore?p=snapshotx&q=vivaldi

May need some truncation on the about text, beside clamping, else we may end up with https://testnet.snapshot.box/#/explore?p=snapshotx&q=vivaldi

Looks like line-clamp-3 already truncates text, what issue do you see?

May need some truncation on the about text, beside clamping, else we may end up with https://testnet.snapshot.box/#/explore?p=snapshotx&q=vivaldi

Looks like line-clamp-3 already truncates text, what issue do you see?

The issue is that it still showing the whole text in the code source, making the page unecessary heavier.
This hidden text is also messing up with you use "Find in page" to search for something in the page, as the text is hidden


[GHSA-p5pg-wm9q-8v6r] Improper Handling of Exceptional Conditions inn metadata-extractor

Updates

Affected products

Comments
https://mvnrepository.com/artifact/com.drewnoakes/metadata-extractor

@ZGorlock Thank you for the submission. According to the repo owner on the original issue, the fixes were included in version 2.18.0, so that is what I have set the fix version to.


frigate: fix path to onvif wsdl files

Frigate assumes, that all its dependencies are in the same site-packages directory as frigate itself, which is not the case with how we link together environments using the PYTHONPATH.
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/frigate-and-onvif-cameras/43193/6


manifest.yaml differs from MS documentation

https://learn.microsoft.com/en-gb/azure/deployment-environments/configure-catalog-item#add-a-new-catalog-item
Indicates that the manifest.yaml should have the structure:
name: WebApp
    version: 1.0.0
    description: Deploys an Azure Web App without a data store
    engine:
      type: ARM
      templatePath: azuredeploy.json

However the manifests in this sample project follow the structure:
name: FunctionApp
version: 1.0.0
summary: Azure Function App Environment
description: Deploys an Azure Function App, Storage Account, and Application Insights
runner: ARM
templatePath: azuredeploy.json

Can open a PR if required, just need confirmation which is correct

Hi @tkimmm - Thank you for bringing it to our attention, the discrepancy has been resolved now! 👍


Updated docstrings

_bifurcationfunc.py supplied by Liesbeth

Current coverage is 99.41% (diff: 100%)

Merging #576 into master will increase coverage by <.01%

@@             master       #576   diff @@
==========================================
  Files            42         42
  Lines          2046       2047     +1
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           2034       2035     +1
  Misses           12         12
  Partials          0          0

Powered by Codecov. Last update 5edae67...fd0680f

Mainly just spelling changes, formatting, etc.


Add definition file for AboutLibraries

Most apps feature a section that lists all used (open-source) libraries. A very simple way to do that is to use AboutLibraries, which tries to automatically pick up all used libraries and list them. To do that a simple resource file containing all the necessary information has to be added to the library, which is done in this PR.
If, for some reason, you don't want this in your lib then I can also make a PR to AboutLibraries to add it to it's internal database. But the preferred way is to add the definition to the upstream project.

IMHO we can include this :+1:
@Valodim


More user-friendly custom registry API

Currently you have to write some slightly obscure Julia code to add custom registries before the bot does its main thing. Maybe registry cloning could be done less directly via configuration? For example:
env:
  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  CUSTOM_REGISTRIES: A/B,C/D

It looks like we don't actually need the UUID and name of the registry, only the URL.

This sounds like a good feature to add. We will still support the old interface. That way this will be a non-breaking feature addition.

Yeah a previous run step that added the registries will still work fine.
I have some code for this here: https://github.com/christopher-dG/CompatHelper.jl/tree/cdg/registries
The tests are not quite finished though.
The API is just two environment variables:

CUSTOM_REGISTRIES: OwnerA/RegA,OwnerB/OwnerA (I think I'll probably add a check for a full URL in case someone wants to add a non-GitHub registry)
DISABLE_GENERAL: Set this to true if you want to not install General.

So I looked at your code. I think I didn’t do a good job of explaining how CompatHelper uses registries.
To tell CompatHelper about a registry, you don’t actually add the registry. You pass a vector of registries in the registries keyword argument to the CompatHelper.main() function.

Oops, I think I just assumed how it worked and didn't really look. Would something like this work, or do we actually need the name and UUID in the RegistrySpec?
using Pkg: RegistrySpec
using GitHub.HTTP: URI
const registries = map(split(get(ENV, "REGISTRIES", ""); keepempty=false)) do registry
    url = isempty(URI(registry).host) ? "https://github.com/$registry" : registry
    RegistrySpec(; url=url)
end


video uploading

Hi,
How can we implement video uploading instant  of photo. Kindly explain in details.
Thanking you

How about video uploading with photo both at the same time


Enhancement: Added Classname to Table Header Currently Getting Resized

PR Summary:

Added a class rt-active-resizing-header to the header which is getting resized.
Solves the problem of adding additional styles ⚡ to resizing header column in ease.
Added support only to v6 react-table.
Happy to take any suggestion or changes 👍

resolves #1916

@tannerlinsley i don't know whats wrong with deploy check the error states no doc:build found kind of.

Sorry for the delay, lots of work to do! I'm not currently accepting PRs for v6. I would suggest forking v6 to make your change.


feat: port Algebra.BigOperators.Finsupp

Some simpNF issues & some leaky classicals caused trouble. But done now!

bors r+


Monitor mode + Help = Display issues

After launching Dry and beginning to explore some of it's features, I noticed that when in Monitor Mode and then bringing up the help menu there is a small display glitch.
Steps to repeat:

Launch Dry
Enter Monitor mode
Press 'h' to enter the help menu

Thanks for reporting!


fuzzing: fix oss-fuzz build

Signed-off-by: AdamKorcz<EMAIL_ADDRESS>@howardjohn
Please provide a description of this PR:

@AdamKorcz: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

lint_istio
68256a27f9cb7017860fefe9e6a4bafefb84b400
link
true
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Alternative Hotspot Manufacturer

Thanks for the submission. This is an exciting development, but not relevant for the HIP process, which is meant for software upgrades to the Helium blockchain. Would love to see this news posted far & wide in the Discord though!

Per discussion in Discord #hips, I'm planning to help Georgica refactor this into a more general proposal around how the community would vet and approve new third-party manufacturers. This is similar to #66

This has been written and merged for discussion as #87


Cyrillic font issue

Was trying to translate a game which natively uses Liberation Sans font and it appeared that certain capital cyrillic letters are getting replaced with squares. I contacted developer to check if this is font-related issue, but it appeared that font is supporting all letters  fine, at least they appeared good in Unity editor. Also this issue doesn't appear in unmodified client when same phrases are put in a textbox. Another note is that only specific letters are affected, and those are not exactly the same letters that are unique for cyrillic languages, some of them appear to be fine so they are only partially broken.
I searched for same issues in bugtracker and found several results that suggested trying to replace fonts. It did fix the issue, although some UI elements disappeared due to resize and application always crashed if I tried to do this with Liberation Sans.
According the fact that only certain specific letters were broken and modification couldn't replace to one specific font makes me think that it may be this project issue.
Unity [2019.4.16f1]

TextMesh Pro (TMP) font replacement has always been a massive PITA.
Not sure what to make of it though, if the developer say they are built into the font.
As far as I am aware LiberationSans SDF is actually one of the default fonts that gets installed when you the install the various TMP packages in the Unity Editor. I am not confident that includes all characters, though. But it could also differ from version to version.
But I am somewhat confident that there must be something wrong with the said font if the translations appear properly in the translation file, so I am not sure what to do about this.


Silence rspec deprecation warnings

Run spawn_spec and store_spec through transpec to silence deprecation
warnings when running specs.

Coverage remained the same at 62.393% when pulling 395a3bb21839a3ccdd1d51afa5478a5789b43da1 on aclemons:rspec_deprecations into e0c2ef83890a795ec3ec252151a13b54c7ec36e8 on tra:master.

Much appreciated @aclemons


Support for Openshift

Describe the problem/challenge you have
I would like to use Antrea with Openshift 4.3
Describe the solution you'd like
I would like to see a certified Antrea Operator for use with Openshift 4.3
Anything else you would like to add?
Antrea should work as a CNI with Openshift running on vSphere and in AWS.

Closing in favor of https://github.com/vmware-tanzu/antrea/issues/680


Don't Merge! PR real time test for source branch full build

Docs Build status updates of commit 4f67168:
:clock10: Full build: the average full build time is 6 min(s) 7 sec(s), this is based on the last 275 build(s) for this repository.
:warning: This is a FULL build due to a configuration change (usually in .openpublishing.publish.config.json and/or docfx.json). It is strongly suggested to merge this configuration change after the business hours, as it could impact the overall build performance of other PRs on the 'pr-basebranch' branch. For more detailed info, please click here.

Docs Build status updates of commit 4f67168:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details
Dependent Files

docfx.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


command line options parsed incorrectly

Describe the bug

I'm trying to use following docker-compose file to run headless mode locust, but every time i ran i got error: /ERROR/locust.main: Unknown User(s). It used to work when i using docker-compose file in the docs(https://docs.locust.io/en/stable/running-in-docker.html?#docker-compose)
version: '3'

services:
  master:
    image: locustio/locust
    ports:
     - "8089:8089"
    volumes:
      - /home/load-test/locust:/mnt/locust
    command: -f /mnt/locust/${LOCUST_FILE} \
             -H ${LOAD_HOST} \
             --headless \
             --users 1000 \
             --spawn-rate 10 \
             --run-time 30m \
             --master \
             --html /mnt/locust/${NODE}.html \
             --stop-timeout 99 \
             ${NODE}

  worker:
    image: locustio/locust
    volumes:
      - /home/load-test/locust:/mnt/locust
    command: -f /mnt/locust/${LOCUST_FILE} --worker --master-host master

Expected behavior

Running correctly
Actual behavior

/ERROR/locust.main: Unknown User(s):  --run-time,  --headless,  --stop-timeout,  --spawn-rate,  Test_node,  -H, 30m, /mnt/locust/Test_node.html, 10,  --master,  --users,  --html, 99, 1000, http://localhost:7554

The order of options is different every time i ran:
/ERROR/locust.main: Unknown User(s): 99, http://localhost:7554, 30m,  --stop-timeout,  --html,  --spawn-rate,  Test_node,  --headless,  --users,  -H,  --master, 10, /mnt/locust/Test_node.html,  --run-time, 1000

Steps to reproduce

Using abovedocker-compose file with this cmd:
 LOCUST_FILE=locustfile.py LOAD_HOST=http://localhost:7554 NODE=Test_node docker-compose up
Environment

OS: Linux version 5.4.0-109-generic (buildd@ubuntu) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1))
Python version: Python 3.9.13
Locust version: locust 2.11.1
Locust command line that you ran:
Locust file contents (anonymized if necessary):

I was getting a similar error when (incorrectly) trying to pass in multiple locustfiles, separated by spaces instead of commas
Running: locust -f locustfiles/locustfile1.py locustfiles/more/locustfile2.py
Caused: ERROR/locust.main: Unknown User(s): locustfiles/more/locustfiles2.py
Have you tried keeping the master command to a single line, as a workaround?

I was getting a similar error when (incorrectly) trying to pass in multiple locustfiles, separated by spaces instead of commas
Running: locust -f locustfiles/locustfile1.py locustfiles/more/locustfile2.py
Caused: ERROR/locust.main: Unknown User(s): locustfiles/more/locustfiles2.py
Have you tried keeping the master command to a single line, as a workaround?

What do you mean by that? Can you show an example

What I'm thinking could be a fix here is, in docker-compose.yml, instead of this:
services:
  master:
    image: locustio/locust
    ports:
     - "8089:8089"
    volumes:
      - /home/load-test/locust:/mnt/locust
    command: -f /mnt/locust/${LOCUST_FILE} \
             -H ${LOAD_HOST} \
             --headless \
             --users 1000 \
             --spawn-rate 10 \
             --run-time 30m \
             --master \
             --html /mnt/locust/${NODE}.html \
             --stop-timeout 99 \
             ${NODE}

try:
services:
  master:
    image: locustio/locust
    ports:
     - "8089:8089"
    volumes:
      - /home/load-test/locust:/mnt/locust
    command: -f /mnt/locust/${LOCUST_FILE} -H ${LOAD_HOST} --headless --users 1000 --spawn-rate 10 --run-time 30m --master --html /mnt/locust/${NODE}.html --stop-timeout 99 ${NODE}

What I'm thinking could be a fix here is, in docker-compose.yml, instead of this:
services:
  master:
    image: locustio/locust
    ports:
     - "8089:8089"
    volumes:
      - /home/load-test/locust:/mnt/locust
    command: -f /mnt/locust/${LOCUST_FILE} \
             -H ${LOAD_HOST} \
             --headless \
             --users 1000 \
             --spawn-rate 10 \
             --run-time 30m \
             --master \
             --html /mnt/locust/${NODE}.html \
             --stop-timeout 99 \
             ${NODE}

try:
services:
  master:
    image: locustio/locust
    ports:
     - "8089:8089"
    volumes:
      - /home/load-test/locust:/mnt/locust
    command: -f /mnt/locust/${LOCUST_FILE} -H ${LOAD_HOST} --headless --users 1000 --spawn-rate 10 --run-time 30m --master --html /mnt/locust/${NODE}.html --stop-timeout 99 ${NODE}

I'm surprised this actually works, thanks for your help

this is most likely due to the yaml parser that docker-compose is using.
it might work with multiline and \ if used as either command: | or command: >, see https://yaml-multiline.info/
kind of strange behavior though, with the order being random.