09:23:31 [ERROR] invalid literal for int() with base 10: '$PORT'

cd tests && docker run -v $(pwd):/var/loadtest -v $HOME/.ssh:/root/.ssh --net host -it direvius/yandex-tank
09:23:31 [INFO] /etc/yandex-tank is not accessible to get configs list
/usr/local/lib/python2.7/dist-packages/yandextank/plugins/Phantom/reader.py:14: FutureWarning: The pandas.parser module is deprecated and will be removed in a future version. Please import from pandas.io.parser instead
  from pandas.parser import CParserError
09:23:31 [INFO] Loading plugins...
09:23:31 [INFO] Performing test
09:23:31 [INFO] Configuring plugins...
09:23:31 [INFO] Exception: Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/yandextank/core/consoleworker.py", line 421, in perform_test
    self.core.plugins_configure()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/core/tankcore.py", line 232, in plugins_configure
    plugin.configure()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/Phantom/plugin.py", line 70, in configure
    self.phantom_config = self.phantom.config_file
    self._phantom.read_config()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/Phantom/utils.py", line 95, in read_config
    stream.read_config()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/Phantom/utils.py", line 275, in read_config
    self.address, do_test_connect, explicit_port)
  File "/usr/local/lib/python2.7/dist-packages/yandextank/common/util.py", line 512, in resolve
    port = int(parts[1])
ValueError: invalid literal for int() with base 10: '$PORT'

09:23:31 [ERROR] invalid literal for int() with base 10: '$PORT'
09:23:31 [INFO] Trying to shutdown gracefully...
09:23:31 [INFO] Finishing test...
09:23:31 [INFO] Stopping load generator and aggregator
09:23:32 [INFO] Finishing monitoring
09:23:32 [ERROR] Failed finishing plugin <yandextank.plugins.DataUploader.plugin.Plugin object at 0x7fd6536966d0>
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/yandextank/core/tankcore.py", line 295, in plugins_end_test
    retcode = plugin.end_test(retcode)
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 305, in end_test
    self.__save_conf()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 467, in __save_conf
    self.lp_job.send_config_snapshot(self.core.cfg_snapshot)
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 563, in lp_job
    self._lp_job = self.__get_lp_job()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 575, in __get_lp_job
    api_client = self.__get_api_client()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 540, in __get_api_client
    self._api_token = self.read_token(self.get_option("token_file"))
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 629, in read_token
    "token length is %d bytes", filename, len(str(data)))
  File "/usr/local/lib/python2.7/dist-packages/future/types/newstr.py", line 102, in __new__
    return super(newstr, cls).__new__(cls, value)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)
09:23:32 [INFO] Post-processing test...
09:23:32 [ERROR] Failed post-processing plugin <yandextank.plugins.DataUploader.plugin.Plugin object at 0x7fd6536966d0>
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/yandextank/core/tankcore.py", line 313, in plugins_post_process
    retcode = plugin.post_process(retcode)
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 316, in post_process
    self.events.join()
  File "/usr/lib/python2.7/threading.py", line 929, in join
    raise RuntimeError("cannot join thread before it is started")
RuntimeError: cannot join thread before it is started
09:23:32 [INFO] Artifacts dir: /var/loadtest/logs/2019-03-04_09-23-31.yz7n8l
09:23:32 [INFO] Done graceful shutdown
09:23:32 [INFO] Close allocated resources...
09:23:32 [INFO] Done performing test with code 1

Okay, i manuaally replace   address: $SERVER:$PORT in load.yaml
But next error:
09:27:43 [INFO] Installing monitoring agent at<EMAIL_ADDRESS>09:27:43 [INFO] Creating temp dir on $server
09:27:43 [INFO] Execute on $server: /usr/bin/env python2 -c "import tempfile; print tempfile.mkdtemp();"
09:27:43 [ERROR] Failed to install monitoring agent to $server
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/Telegraf/client.py", line 209, in install
    out, errors, err_code = self.ssh.execute(cmd)
  File "/usr/local/lib/python2.7/dist-packages/yandextank/common/util.py", line 70, in execute
    with self.connect() as client:
  File "/usr/local/lib/python2.7/dist-packages/yandextank/common/util.py", line 40, in connect
    timeout=self.timeout, )
  File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 329, in connect
    to_try = list(self._families_and_addresses(hostname, port))
  File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 200, in _families_and_addresses
    hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM)
gaierror: [Errno -2] Name or service not known
09:27:59 [INFO] Checking tank resources...
09:27:59 [ERROR] There were errors:
df: no file systems processed

09:27:59 [INFO] Exception: Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/yandextank/core/consoleworker.py", line 422, in perform_test
    self.core.plugins_prepare_test()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/core/tankcore.py", line 240, in plugins_prepare_test
    plugin.prepare_test()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 229, in prepare_test
    lp_job = self.lp_job
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 563, in lp_job
    self._lp_job = self.__get_lp_job()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 575, in __get_lp_job
    api_client = self.__get_api_client()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 540, in __get_api_client
    self._api_token = self.read_token(self.get_option("token_file"))
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 629, in read_token
    "token length is %d bytes", filename, len(str(data)))
  File "/usr/local/lib/python2.7/dist-packages/future/types/newstr.py", line 102, in __new__
    return super(newstr, cls).__new__(cls, value)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)

09:27:59 [ERROR] 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)
09:27:59 [INFO] Trying to shutdown gracefully...
09:27:59 [INFO] Finishing test...
09:27:59 [INFO] Stopping load generator and aggregator
09:27:59 [INFO] Finishing monitoring
09:27:59 [ERROR] Failed finishing plugin <yandextank.plugins.DataUploader.plugin.Plugin object at 0x7f1b4d5c46d0>
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/yandextank/core/tankcore.py", line 295, in plugins_end_test
    retcode = plugin.end_test(retcode)
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 305, in end_test
    self.__save_conf()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 467, in __save_conf
    self.lp_job.send_config_snapshot(self.core.cfg_snapshot)
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 563, in lp_job
    self._lp_job = self.__get_lp_job()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 575, in __get_lp_job
    api_client = self.__get_api_client()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 540, in __get_api_client
    self._api_token = self.read_token(self.get_option("token_file"))
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 629, in read_token
    "token length is %d bytes", filename, len(str(data)))
  File "/usr/local/lib/python2.7/dist-packages/future/types/newstr.py", line 102, in __new__
    return super(newstr, cls).__new__(cls, value)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)
09:27:59 [INFO] Post-processing test...
09:27:59 [ERROR] Failed post-processing plugin <yandextank.plugins.DataUploader.plugin.Plugin object at 0x7f1b4d5c46d0>
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/yandextank/core/tankcore.py", line 313, in plugins_post_process
    retcode = plugin.post_process(retcode)
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 316, in post_process
    self.events.join()
  File "/usr/lib/python2.7/threading.py", line 929, in join
    raise RuntimeError("cannot join thread before it is started")
RuntimeError: cannot join thread before it is started
09:27:59 [INFO] Artifacts dir: /var/loadtest/logs/2019-03-04_09-27-38.fJ8XJZ
09:27:59 [INFO] Done graceful shutdown
09:27:59 [INFO] Close allocated resources...
09:27:59 [INFO] Done performing test with code 1

How to fill monitor.xml instead  <Host address="$SERVER" interval="1" username="$USERNAME">  ?

Okay, I remove from load.yaml section telegraf:. But next error:
09:41:07 [INFO] Installing monitoring agent at<EMAIL_ADDRESS>09:41:07 [INFO] Creating temp dir on <IP_ADDRESS>
09:41:07 [INFO] Execute on <IP_ADDRESS>: /usr/bin/env python2 -c "import tempfile; print tempfile.mkdtemp();"
09:41:08 [ERROR] Failed to install monitoring agent to <IP_ADDRESS>
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/Telegraf/client.py", line 209, in install
    out, errors, err_code = self.ssh.execute(cmd)
  File "/usr/local/lib/python2.7/dist-packages/yandextank/common/util.py", line 70, in execute
    with self.connect() as client:
  File "/usr/local/lib/python2.7/dist-packages/yandextank/common/util.py", line 40, in connect
    timeout=self.timeout, )
  File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 357, in connect
    raise NoValidConnectionsError(errors)
NoValidConnectionsError: [Errno None] Unable to connect to port 22 on <IP_ADDRESS>
09:41:23 [INFO] Checking tank resources...
09:41:23 [ERROR] There were errors:
df: no file systems processed

09:41:23 [INFO] Exception: Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/yandextank/core/consoleworker.py", line 422, in perform_test
    self.core.plugins_prepare_test()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/core/tankcore.py", line 240, in plugins_prepare_test
    plugin.prepare_test()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 229, in prepare_test
    lp_job = self.lp_job
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 563, in lp_job
    self._lp_job = self.__get_lp_job()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 575, in __get_lp_job
    api_client = self.__get_api_client()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 540, in __get_api_client
    self._api_token = self.read_token(self.get_option("token_file"))
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 629, in read_token
    "token length is %d bytes", filename, len(str(data)))
  File "/usr/local/lib/python2.7/dist-packages/future/types/newstr.py", line 102, in __new__
    return super(newstr, cls).__new__(cls, value)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)

09:41:23 [ERROR] 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)
09:41:23 [INFO] Trying to shutdown gracefully...
09:41:23 [INFO] Finishing test...
09:41:23 [INFO] Stopping load generator and aggregator
09:41:23 [INFO] Finishing monitoring
09:41:23 [ERROR] Failed finishing plugin <yandextank.plugins.DataUploader.plugin.Plugin object at 0x7f557aceb6d0>
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/yandextank/core/tankcore.py", line 295, in plugins_end_test
    retcode = plugin.end_test(retcode)
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 305, in end_test
    self.__save_conf()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 467, in __save_conf
    self.lp_job.send_config_snapshot(self.core.cfg_snapshot)
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 563, in lp_job
    self._lp_job = self.__get_lp_job()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 575, in __get_lp_job
    api_client = self.__get_api_client()
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 540, in __get_api_client
    self._api_token = self.read_token(self.get_option("token_file"))
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 629, in read_token
    "token length is %d bytes", filename, len(str(data)))
  File "/usr/local/lib/python2.7/dist-packages/future/types/newstr.py", line 102, in __new__
    return super(newstr, cls).__new__(cls, value)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)
09:41:23 [INFO] Post-processing test...
09:41:23 [ERROR] Failed post-processing plugin <yandextank.plugins.DataUploader.plugin.Plugin object at 0x7f557aceb6d0>
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/yandextank/core/tankcore.py", line 313, in plugins_post_process
    retcode = plugin.post_process(retcode)
  File "/usr/local/lib/python2.7/dist-packages/yandextank/plugins/DataUploader/plugin.py", line 316, in post_process
    self.events.join()
  File "/usr/lib/python2.7/threading.py", line 929, in join
    raise RuntimeError("cannot join thread before it is started")
RuntimeError: cannot join thread before it is started
09:41:23 [INFO] Artifacts dir: /var/loadtest/logs/2019-03-04_09-41-06.knfy6z
09:41:23 [INFO] Done graceful shutdown
09:41:23 [INFO] Close allocated resources...
09:41:23 [INFO] Done performing test with code 1

To tests you need to:

copy tests/load.yaml from tests/load.yaml.example
and replace $SERVER and $PORT variables in tests/load.yaml

copy tests/monitoring.xml from tests/monitoring.xml.example
and replace $SERVER and $USERNAME variables in tests/monitoring.xml

get token from https://overload.yandex.net and put it to file tests/overload_token.txt

put ssh public  key to target server.

run yandex-tank:
cd tests && docker run -v $(pwd):/var/loadtest -v $HOME/.ssh:/root/.ssh --net host -it direvius/yandex-tank

If you have errors with Yandex tank - you need to describe the problem there https://github.com/yandex/yandex-tank


position_reference with panos_security_rule_group usage

Is it possible to be able to reference an entire rule group instead of individual rules when using position_reference?. I am looking to ensure 3 independent groups of rules stay in a particular order and that the rules stay within those blocks/groups.

If this isn't possible, is it possible to be able to maintain a certain order of rules within a rule group? I've seen that it doesn't always maintain the rule order in the rule group definition.

This isn't specific to the panos provider, you just use HCL to do this.  Here's an example:
resource "panos_security_rule_group" "first" {
    rule {
        name = "first"
        source_zones = ["any"]
        source_addresses = ["any"]
        source_users = ["any"]
        destination_zones = ["any"]
        destination_addresses = ["any"]
        applications = ["any"]
        services = ["any"]
        categories = ["any"]
        log_end = true
        action = "allow"
    }

    lifecycle {
        create_before_destroy = true
    }
}

resource "panos_security_rule_group" "second" {
    position_keyword = "directly after"
    position_reference = (
        panos_security_rule_group.first.rule[
            length(panos_security_rule_group.first.rule) - 1
        ].name
    )
    rule {
        name = "second"
        source_zones = ["any"]
        source_addresses = ["any"]
        source_users = ["any"]
        destination_zones = ["any"]
        destination_addresses = ["any"]
        applications = ["any"]
        services = ["any"]
        categories = ["any"]
        log_end = true
        action = "allow"
    }
    rule {
        name = "third"
        source_zones = ["any"]
        source_addresses = ["any"]
        source_users = ["any"]
        destination_zones = ["any"]
        destination_addresses = ["any"]
        applications = ["any"]
        services = ["any"]
        categories = ["any"]
        log_end = true
        action = "allow"
    }

    lifecycle {
        create_before_destroy = true
    }
}

resource "panos_security_rule_group" "third" {
    position_keyword = "directly after"
    position_reference = (
        panos_security_rule_group.second.rule[
            length(panos_security_rule_group.second.rule) - 1
        ].name
    )
    rule {
        name = "fourth"
        source_zones = ["any"]
        source_addresses = ["any"]
        source_users = ["any"]
        destination_zones = ["any"]
        destination_addresses = ["any"]
        applications = ["any"]
        services = ["any"]
        categories = ["any"]
        log_end = true
        action = "allow"
    }

    lifecycle {
        create_before_destroy = true
    }
}


Import favicon stuff from existing site

The icon for our site is blank currently. We should have stuff you can steal now from kubernetes.io. The wheel icon is fine. Higher-res is always better if you got it.
cc @CreaturesInUnitards

yup done


UI fixes

Fixed the Navbar component

Fixed the Contact Us component

LGTM will be merging it ASAP

Resolve up the conflicts!!


[INTERNAL] ui.support: suggest mvc instead of core

For consistency and proper resource loading, the root node of an XML view must be configured with the namespace mvc.
The resolution text of the support rule "xmlViewWrongNamespace", however, was still suggesting to define the root node as <core:View ...>.
The resolution text is now aligned with the description text.

Fixes: https://github.com/SAP/openui5/issues/2155

Hello,
The pull request was merged with 947a8ed in master branch.
The change will be in the following UI5 version(s): 1.58.2
Thank you for the contribution


Add transferable-memory create to fix regressions

Fixes regressions from #174
For example we had now a max. byte length for vertices and indices.

Closing in favor of #221


HTML

HTML
USER STORY
As a boot camp student,
I want the prework noters to be structured on a webpage
So that I can easily find and read the information
ACCEPTANCE CRITERIA
GIVEN a prework study guide website
WHEN I visit the website in my browser
THEN I see four boxes titled HTML, CSS, Git, and JavaScript with associated notes listed.

All done meow :3


Make Calico nodename overridable on bare metal

Fixes #2753
Signed-off-by: Daniel Mohr<EMAIL_ADDRESS>

ci check this

Thanx so much!
This workaround saved a lot of time for me when I tried install via 2.5.0 version on Baremetal (DO) from scratch (not upgrading).
So I think this PR or like this one exactly must be added to fix the problem.

I can't see an error in the CI check. Why did it fail?

@Atoms Thanks. So, can we merge it?


Feature-Example-Promethusrule

Information on the feature
Considering Prometheus can scrape metrics from hue pods the next step is to set up some alerts (For example: If pods are down etc). I couldn't find this feature/option enabled in the current code repo. I am aware of this could be handled differently based on k8's monitoring setup but keeping in mind already the awesome features that the code has wanted felt alertrule example could be added.
Hence, Wanted to add this feature in place in case someone wants to set up alert rules on Prometheus.
(This could also be done on Grafana, etc but many people use this approach too).
Please let me know if I have to change any naming for the options/variables used in the commit.
Reference used : PrometheusRule
I hope this feature shall be accepted and does add a small upgrade towards monitoring hue :)

Another nit, renaming the file to follow the convention, e.g. prometheus-rule-hue.yml?
And :+1: indeed this is one step in easier monitoring of the service!

Another nit, renaming the file to follow the convention, e.g. prometheus-rule-hue.yml?
And :+1: indeed this is one step in easier monitoring of the service!

Thanks a lot @romainr for encouraging me on this PR . I shall address a few name changes right away. Variable declaration will test in a few mins and push a follow up commits.

Thanks a lot @romainr for encouraging me on this PR . I shall address a few name changes right away. Variable declaration will test in a few mins and push a follow up commits.

Thanks a tun @zorro1992, let's go with it and we can always iterate!

Thanks a tun @zorro1992, let's go with it and we can always iterate!

And for the record some of potential other metrics to use:
https://gethue.com/collecting-hue-metrics-with-prometheus-in-kubernetes/
https://gethue.com/monitoring-hue-activity-with-grafana-dashboards/

And for the record some of potential other metrics to use:
https://gethue.com/collecting-hue-metrics-with-prometheus-in-kubernetes/
https://gethue.com/monitoring-hue-activity-with-grafana-dashboards/


[Deepcast] Formality support

Extension
https://www.raycast.com/mooxl/deepcast
Description
DeepL allows specifying formality in the API request. It would be great to add support for this parameter

Who will benefit from this feature?
Users that use languages that have formal and informal translations, like Spanish for example
Anything else?
https://www.deepl.com/docs-api/translate-text

keep open. I'll hopefully get to it at some point

@raycastbot keep this issue open

@raycastbot assign me

Forgot to tag, fixed in #15511

@raycastbot close this issue


Can RP2040 build environment be supported?

Is it possible to support the RP2040 build environment?
Would be appreciated
PR - https://github.com/RobTillaart/MCP_DAC/pull/16

Found how to do it.
.arduino-ci.yml
platforms:
  pico:
    board: rp2040:rp2040:rpipico
    package: rp2040:rp2040
    gcc:
      features:
      defines:
        - ARDUINO_ARCH_RP2040
      warnings:
      flags:

packages:
  rp2040:rp2040:
    url: https://github.com/earlephilhower/arduino-pico/releases/download/global/package_rp2040_index.json

compile:
  # Choosing to run compilation tests on 2 different Arduino platforms
  platforms:
    - uno
    # - due
    # - zero
    # - leonardo
    # - m4
    # - esp32
    # - esp8266
    # - mega2560
    - pico

Tested and it works

If this is a popular board, I wouldn't mind adding it to https://github.com/Arduino-CI/arduino_ci/blob/master/misc/default.yml

Think it is pretty popular as there are several devboards and variations.
For me the yml file is also a 'guide' how other boards can be added, so no need to add it to default.yml imho.
It would be interesting to know what are the top 15 boards used.

My high level goal isn't necessarily to represent the most popular board, just to have enough boards defined that any given library developer will find at least one working arduino_ci platform for their library, out of the box.

Imho you have enough boards supported. The ones I could think of

AT TINY series (most interesting as these have very little RAM etc)
NANO BLE
RP2040
Teensy

That said, when compiling the examples, is it possible to get info about sketch size and RAM usage?
When developing libraries I often want to know if code increases or decreases and by how much.
This is mostly an AVR thingy as these are the most limited wrt RAM.
( no this is not a request for a new feature, as I investigate this locally before pushing code )

Hi, I will test with STM32.
We had lot of librairies.

Hi, I will test with STM32. We had lot of librairies.

STM32  +1

@RobTillaart

when compiling the examples, is it possible to get info about sketch size and RAM usage?

This is my idea with the ARDUINO_CI_POST_UNIT_TEST_RUN_SCRIPT hook script that enables you to for instance capture build results by running avr-size and append the result to a csv file/send an email/push a chat message/etc (or maybe even just print to the screen).
This is present on my scripts branch pending a merge of #320.

@hlovdal on my local branch I've started reporting the "free bytes" (in the message at the end of each compilation step) to the test output, but I can see that we're getting to the point where machine-readable output is necessary.
Can you open an issue to track this idea? It sounds like you'd like a report of things that are more informational than pass/fail. Sizes and compilation times are 2 things that jump out at me there.
Regarding your issue about the hooks, I think it's a great idea!  My first step for it is figuring out how to do it in a generic way. Happy to continue the discussion on that other issue


⚠ Adacta-fintech.com has degraded performance

In 3341d68, Adacta-fintech.com (https://www.adacta-fintech.com) experienced degraded performance:

HTTP code: 200
Response time: 1441 ms

Resolved: Adacta-fintech.com performance has improved in 7f739db after 14 minutes.


Azure Playbook [BlockAADUser-Alert]

Describe the bug
I clone the BlockAADUser-Alert template and attached it to the automation rules of one of my analytics rule.
I expected the playbook to block user every time the alert is triggered, however, I received a comment from the playbook saying:

Expected behavior
It's expected to block the user but it is not and the connection is a global admin account
Screenshots
Is this connection right:

or this:

Hi @DevMickeal
did you authorize the AAD connection?  if so, the user that you used does it have permissions to update the user?

Thanks for checking @dicolanl, the user is a GA and I did authorize the AAD connection.
As shared in the screenshot, which is the right connection, I believe it should be Azure AD

the template will use the azuread-Block-AADUser.  but in reality, it doesnt matter which connection you use as long as the connection is authorized and the user that authorized has the needed permissions.

The connection was authorized by a GA, so it definitely has the right permission and how do I know if the connection has been authorized.

could you look at previous runs and screen shot the failed run?

@dicolanl and see below forbidden on the successful run

Hi @DevMickeal, so the user authenticated to the connector is a global administrator on the tenant? I'm asking since the actual user which the connector was authenticated with must have
Group.ReadWrite.All
User.ReadWrite.All
Directory.ReadWrite.All
Can you please share the error message as well?

to update - API needs special permission to be able to block admin user (reference - https://docs.microsoft.com/en-us/graph/api/user-update?view=graph-rest-beta&tabs=http#request-body). We are working on updating current readme to state that, and creating additional playbook with higher permissions to be able to block admin users.

Readme for playbook Block-AADUser is updated with info that it doesn't support  block of admin users. Template updated to have notification to manager, if manager available.
Additional playbook created that supports block of admin users as well - Block-AADUserOrAdmin.

That's great...thank you Benjamin..thank you SecuritJedi😊
On Mon, Nov 29, 2021, 7:30 PM Benjamin Kovacevic @.***>
wrote:

Readme for playbook Block-AADUser is updated with info that it doesn't
support block of admin users. Template updated to have notification to
manager, if manager available.
Additional playbook created that supports block of admin users as well -
Block-AADUserOrAdmin.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Azure/Azure-Sentinel/issues/3522#issuecomment-981902470,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AO3INGRCVHXYFAGFPBQEYTLUOPBF3ANCNFSM5ITCDMPQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.


Added transport of pcapng data and unicast to Server/Client protocol

Server/Client protocol transports pcapng data via Multi- or Unicast

modified output option -o +  server sends pcapng data to client
-o <file.pcapng> on client side dumps packets received to file.pcapng
server starts transmission with sending the pcapng header
starting unicast client receiving running server messages requests pcapng header from server
starting multicast client requests pcapng header on multicast group
running client receiving new pcapng header from (re-)started server creates new dump file
default destination is multicast ip <IP_ADDRESS>
use --ip to set either Multicast address, localhost or unicast client ip on both sides
use --enable_status to activate client (256) or server (128) as usual

Note: protocol uses UDP without further provisions, packets may get lost or be received and stored in wrong sequence

Nice improvement. Thanks. Merged.


Flattened maps with numeric keys do not work correctly.

#[derive(serde::Deserialize)]
struct A {
    pub map: std::collections::BTreeMap<u64, String>,
}

#[derive(serde::Deserialize)]
struct B {
    #[serde(flatten)]
    pub map: std::collections::BTreeMap<u64, String>,
}

fn main() {
    // works
    let a: A = serde_json::from_str(r#"{"map": {"1": "a", "2": "b"}}"#).unwrap();
    // fails
    let b: B = serde_json::from_str(r#"{"1": "a", "2": "b"}"#).unwrap();
}

The second call fails with Error("invalid type: string \"1\", expected u64", line: 1, column: 20)'.
Surprisingly (to me) deserializing below struct will fail as well, so it seems it's just enough if the map is nested anywhere in the flattened struct.
#[derive(serde::Deserialize)]
struct C {
    #[serde(flatten)]
    pub map: A,
}

The map key conversion from string to number is a serde_json feature so this looks like a special case of https://github.com/serde-rs/serde/issues/1183. I'll close in favor of that issue.
For a workaround it would be reasonable for a crate such as https://github.com/jonasbb/serde_with to provide the conversion from string keys to numeric keys. Please file a feature request over there if it isn't already supported.
#[derive(Deserialize)]
pub struct B {
    #[serde(flatten, with = "???")]
    map: BTreeMap<u64, String>,
}


Autoformatting moves object to the next line

Expected Behavior
Autoformatting should not reformat below line to incorrect state
private val deferredMessageQueueSubscription = DeferredMessageQueueSubscription(uri = ServiceBusUri("deferredQueueTestUri"))

Observed Behavior
Autoformatting moves DeferredMessageQueueSubscription(uri = ServiceBusUri("deferredQueueTestUri")) to the next line and as the results it looks like this:
    private val deferredMessageQueueSubscription =
    DeferredMessageQueueSubscription(uri = ServiceBusUri("deferredQueueTestUri"))

I think it may be related to max line length limit but still I think it should not touch this but throw a warning instead.
Related to PropertyWrapping.
Your Environment

Version of detekt used: 1.23.0
Operating System and version: Kotlin 1.8.21

Hi, @nacyolsa in my case autoformatting is working correctly like below
private val deferredMessageQueueSubscription =
    DeferredMessageQueueSubscription(uri = ServiceBusUri("deferredQueueTestUri"))

with an indentation in the second line.
Is wrong formatting happening consistently in your case?

For me in both cases like below:
private val deferredMessageQueueSubscription =
DeferredMessageQueueSubscription(uri = ServiceBusUri("deferredQueueTestUri"))

class TestC {
    private val deferredMessageQueueSubscription =
    DeferredMessageQueueSubscription(uri = ServiceBusUri("deferredQueueTestUri"))
}

Formatting is incorrect.

Is wrong formatting happening consistently in your case?

For this case always when I run Detekt.
I checked this also in smaller project and it didn't occur so it is connected to something else.

Hi, @nacyolsa can you also share your detekt.yml?

@atulgpt Here you go:
formatting:
    excludes: [ '**/build/**' ]
    MaximumLineLength:
        active: false
    NoBlankLineBeforeRbrace:
        active: false
    Indentation:
        active: false
    ArgumentListWrapping:
        active: false
    MultiLineIfElse:
        active: false
style:
    excludes: [ '**/build/**' ]
    MagicNumber:
        active: true
        excludes: [ '**/build.gradle.kts', '**/test/**', '**/build/**' ]
    MaxLineLength:
        maxLineLength: 180
        active: true
    ForbiddenComment:
        active: true
    RedundantHigherOrderMapUsage:
        active: false
    VarCouldBeVal:
        active: false
naming:
    excludes: [ '**/build/**' ]
    MemberNameEqualsClassName:
        active: true
        excludes: [ '**/buildSrc/**' ]
    InvalidPackageDeclaration:
        active: false
    NoNameShadowing:
        active: false
    TopLevelPropertyNaming:
        active: false
complexity:
    excludes: [ '**/build/**' ]
    LongParameterList:
        functionThreshold: 15
        excludes: ['**/test/**']
    TooManyFunctions:
        thresholdInFiles: 15
        thresholdInClasses: 15
coroutines:
    InjectDispatcher:
        active: false

ArrowRuleSet:
    NoEffectScopeBindableValueAsStatement:
        active: true

Hi, @nacyolsa actually it is an issue(or intended behaviour) with ktlint. Actually when you run PropertyWrapping rule(PropertyWrapping is a stable rule hence it is enabled by default) without Indentation then PropertyWrapping simply inserts the new line and doesn't indent it.
Either you have to enable the Indentation or disable the PropertyWrapping rule.
Can you also create an issue at the ktlint repo? In my opinion, this can be fixed. @paul-dingemans might be able to answer there

Issue in ktlint https://github.com/pinterest/ktlint/issues/2095

@nacyolsa I've updated ktlint to 0.50.0. Can you please try a detekt snapshot and check that the upstream fix has resolved the issue?

@3flex I can check it but I can't find snapshot name in this build https://github.com/detekt/detekt/actions/runs/5450089240
What is its name?

Please follow these instructions: https://detekt.dev/docs/introduction/snapshots

I tried with main-SNAPSHOT  it downloaded detekt-gradle-plugin-main-20230704.022131-1376 version.
Looks like issue is fixed. Results:
    private val deferredMessageQueueSubscription =
        DeferredMessageQueueSubscription(uri = ServiceBusUri("deferredQueueTestUri"))

Instead of
    private val deferredMessageQueueSubscription =
    DeferredMessageQueueSubscription(uri = ServiceBusUri("deferredQueueTestUri"))


Pptxgen es6 poyline

Can create Polyline and Polygone

The library is slated to support IE11 for a while longer (until the end of 2017 probably), so i'm closing this for now.
ES6 features will be implemented once IE11 support is dropped.
Thanks!


Default SSL credentials don't work on Win32/OsX.

I am trying to build and run the code snippet from https://grpc.github.io/docs/guides/auth.html
`// Create a default SSL ChannelCredentials object.
auto channel_creds = grpc::SslCredentials(grpc::SslCredentialsOptions());
// Create a channel using the credentials created in the previous step.
auto channel = grpc::CreateChannel("some.endpoint.com:443", channel_creds);
// Create a stub on the channel.
std::unique_ptrGreeter::Stub stub(Greeter::NewStub(channel));
// Make actual RPC calls on the stub.
grpc::Status s = stub->sayHello(&context, *request, response);`
The endpoint I'm trying to reach is a TLS secured gRPC service with no gRPC server certificates.
What version of gRPC and what language are you using?
Tried 1.27, 1.32 and 1.34 with C++, none seems to be working, only Linux works.
What operating system (Linux, Windows,...) and version?
Windows, OsX and Linux, only Linux works, Windows and OsX fails.
What runtime / compiler are you using (e.g. python version or version of gcc)
MSDEV 2017 and 2019, Clang 11.0 on OsX. GCC 7.5 on Linux which works.
What did you do?
I was trying to run the example from the documentation page, trying to create a secure channel with server side certificate only.
What did you expect to see?
I expected the channel to establish a connection and connect. Tried to same code in Java and .NET and it works without issues:
Channel channel = new Channel("some.endpoint.com:443", new SslCredentials()); var client = new MyGrpcClient(channel);
What did you see instead?
Received an error and the connection was not established:
E0114 10:45:32.025000000  2336 ssl_utils.cc:565] load_file: {"created":"@1610649932.024000000","description":"Failed to load file","file":"c:\.conan\46f65c\1\source_subfolder\src\core\lib\iomgr\load_file.cc","file_line":72,"filename":"/usr/share/grpc/roots.pem","referenced_errors":[{"created":"@1610649932.024000000","description":"No such file or directory","errno":2,"file":"c:\.conan\46f65c\1\source_subfolder\src\core\lib\iomgr\load_file.cc","file_line":45,"os_error":"No such file or directory","syscall":"fopen"}]} E0114 10:45:32.052000000  2336 ssl_security_connector.cc:413] Could not get default pem root certs. E0114 10:45:32.057000000  2336 secure_channel_create.cc:107] Failed to create secure subchannel for secure name 'some.endpoint.com:443' E0114 10:45:32.065000000  2336 secure_channel_create.cc:49] Failed to create channel args during subchannel creation.
Anything else we should know about your project / environment?
Tried both Conan packages and hand-built gRPC. I made sure the grpc++_unsecure & grpc_unsecure libraries are NOT added to the link dependencies.
I also tried to add self signed certificates (not the server's) and it still failed to authenticate and connect. I am not sure why it works on Linux and not on Windows or OsX.

Is this commit fixing your problem? Try 1.35.0 release

Is this commit fixing your problem? Try 1.35.0 release

@nicolasnoble where is default root.pem installed in the system for Mac and windows?

@nicolasnoble where is default root.pem installed in the system for Mac and windows?

I am not sure about Mac but on Windows you need to go through the specific API. You need to open the "ROOT" certificate store and I assume the first certificate will be the "root" one.
example

I am not sure about Mac but on Windows you need to go through the specific API. You need to open the "ROOT" certificate store and I assume the first certificate will be the "root" one.
example

From the error message, it seems on Windows, the client is not able to read the root certificate from the default root path. Would you be able to manually read the certificate byte data from that "specific API", and then put it into grpc client? You can set the root certificates from grpc::SslCredentialsOptions on the client side as well.

From the error message, it seems on Windows, the client is not able to read the root certificate from the default root path. Would you be able to manually read the certificate byte data from that "specific API", and then put it into grpc client? You can set the root certificates from grpc::SslCredentialsOptions on the client side as well.

@zackgalbreath did you install grpc to your system? That is grpc/etc/roots.pem will be installed to /usr/share/grpc/roots.pem.
If /usr/share/grpc/roots.pem does not exist, it is a good signal that you did not install it.

@zackgalbreath did you install grpc to your system? That is grpc/etc/roots.pem will be installed to /usr/share/grpc/roots.pem.
If /usr/share/grpc/roots.pem does not exist, it is a good signal that you did not install it.

@ZhenLian, I appreciate you looking into this issue. Yes, I can definitely read the root certificate on the application level or I can even generate a certificate on the fly. I see that there are experimental features in the latest version that allow me to plug in my own certificate provider but I simply did not have enough time to look into it yet.

@ZhenLian, I appreciate you looking into this issue. Yes, I can definitely read the root certificate on the application level or I can even generate a certificate on the fly. I see that there are experimental features in the latest version that allow me to plug in my own certificate provider but I simply did not have enough time to look into it yet.

Sounds good! I will close this for now, but feel free to reopen it if you have anything else that you want to bring up :)

@zbordas-kollective Is is possible to share the code snippet to populate SslCredentialsOptions on Windows from the root certificate? I tried but failed getting it to work so far...

@zbordas-kollective Is is possible to share the code snippet to populate SslCredentialsOptions on Windows from the root certificate? I tried but failed getting it to work so far...

The root certificate we currently use is Cloudflare Inc ECC CA-3 Certificate
`        grpc::SslCredentialsOptions sslCrendentialOptions;
    sslCrendentialOptions.pem_root_certs = <-----BEGIN CERTIFICATE----- ...>;

    auto channel = grpc::CreateChannel(mGrpcEndpoint,
                                   grpcEndpointSslEnabled ? grpc::SslCredentials(sslCrendentialOptions) : grpc::InsecureChannelCredentials());

    channel->WaitForConnected(chrono::system_clock::now() + chrono::milliseconds(grpcConnectionTimeout));

`

Alright, I solved it, I thought I only had to provide the first root certificate as someone mentioned above, but I had to iterate through the whole root certificate store. Here a complete example for anyone needing this:
#include <wincrypt.h>
#include <Windows.h>

string utf8Encode(const wstring& wstr)
{
    if (wstr.empty())
        return string();

    int sizeNeeded = WideCharToMultiByte(CP_UTF8, 0, &wstr[0], (int)wstr.size(), NULL, 0, NULL, NULL);
    string strTo(sizeNeeded, 0);
    WideCharToMultiByte(CP_UTF8, 0, &wstr[0], (int)wstr.size(), &strTo[0], sizeNeeded, NULL, NULL);
    return strTo;
}

SslCredentialsOptions getSslOptions()
{
	// Fetch root certificate as required on Windows (s. issue 25533).
	SslCredentialsOptions result;

	// Open root certificate store.
	HANDLE hRootCertStore = CertOpenSystemStoreW(NULL, L"ROOT");
	if (!hRootCertStore)
		return result;

	// Get all root certificates.
	PCCERT_CONTEXT pCert = NULL;
	while ((pCert = CertEnumCertificatesInStore(hRootCertStore, pCert)) != NULL)
	{
		// Append this certificate in PEM formatted data.
		DWORD size = 0;
		CryptBinaryToStringW(pCert->pbCertEncoded, pCert->cbCertEncoded, CRYPT_STRING_BASE64HEADER, NULL, &size);
		vector<WCHAR> pem(size);
		CryptBinaryToStringW(pCert->pbCertEncoded, pCert->cbCertEncoded, CRYPT_STRING_BASE64HEADER, pem.data(), &size);

		result.pem_root_certs += utf8Encode(pem.data());
	}
	CertCloseStore(hRootCertStore, 0);

	return result;
}

Alright, I solved it, I thought I only had to provide the first root certificate as someone mentioned above, but I had to iterate through the whole root certificate store. Here a complete example for anyone needing this:
#include <wincrypt.h>
#include <Windows.h>

string utf8Encode(const wstring& wstr)
{
	if (wstr.empty())
		return string();

	int sizeNeeded = WideCharToMultiByte(CP_UTF8, 0, &wstr[0], (int)wstr.size(), NULL, 0, NULL, NULL);
	string strTo(sizeNeeded, 0);
	WideCharToMultiByte(CP_UTF8, 0, &wstr[0], (int)wstr.size(), &strTo[0], sizeNeeded, NULL, NULL);
	return strTo;
}

SslCredentialsOptions getSslOptions()
{
	// Fetch root certificate as required on Windows (s. issue 25533).
	SslCredentialsOptions result;

	// Open root certificate store.
	HANDLE hRootCertStore = CertOpenSystemStoreW(NULL, L"ROOT");
	if (!hRootCertStore)
		return result;

	// Get all root certificates.
	PCCERT_CONTEXT pCert = NULL;
	while ((pCert = CertEnumCertificatesInStore(hRootCertStore, pCert)) != NULL)
	{
		// Append this certificate in PEM formatted data.
		DWORD size = 0;
		CryptBinaryToStringW(pCert->pbCertEncoded, pCert->cbCertEncoded, CRYPT_STRING_BASE64HEADER, NULL, &size);
		vector<WCHAR> pem(size);
		CryptBinaryToStringW(pCert->pbCertEncoded, pCert->cbCertEncoded, CRYPT_STRING_BASE64HEADER, pem.data(), &size);

		result.pem_root_certs += utf8Encode(pem.data());
	}
	CertCloseStore(hRootCertStore, 0);

	return result;
}

Thanks for your solution!
Btw, <Windows.h> should be included before <wincrypt.h>, otherwise it fails to build

Thanks for your solution! Btw, <Windows.h> should be included before <wincrypt.h>, otherwise it fails to build

Updated in my comment, may wanna remove the code in your reply so people don't see the old thing twice

nvm im still getting:
Received http2 header with status: 526 (526 == invalid SSL) ?


Update CI Python versions for Ubuntu

updating base py version + gh action version for pyenv

This can be merged to master now. Dropped python 3.6 support. At some point we should add 3.10 and 3.11 too.
A Squash and merge using the dropdown on the green button is the way to go I'd say.


Fix for #191

Scribble doesn't correctly instrument contracts wtih contract-level invariants and try/catch statements. Specifically given the following code:
/// #invariant true;
contract Foo {
        function main(bool flag, Bar b) internal {
                if (flag)
                        try b.bar() {
                        } catch Error(string memory reason) {
                                revert(reason);
                        } catch Panic(uint code) {
                                revert("panic");
                        } catch {
                                revert("foo");
                        }
        }
}

Scribble will attempt to replace the b.bar() inside the try with a call to an internal wrapper. However calls in try statements must be external. So the instrumented code doesn't compile.
The fix is to introduce special-case instrumentation for external callsites in try/catch statements.
Instead of wrapping the callsites themselves, we insert the neccessary instrumentation around the try/catch statement.

Codecov Report

Merging #196 (ccc68aa) into develop (8ab0ac8) will increase coverage by 0.01%.
The diff coverage is 92.85%.

@@             Coverage Diff             @@
##           develop     #196      +/-   ##
===========================================
+ Coverage    85.84%   85.86%   +0.01%
===========================================
  Files           74       74
  Lines         5397     5411      +14
  Branches      1211     1214       +3
===========================================
+ Hits          4633     4646      +13
  Misses         492      492
- Partials       272      273       +1

Impacted Files
Coverage Δ

src/instrumenter/instrument.ts
94.67% <92.85%> (-0.08%)
:arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Filtering fixes

This PR fixes some filtering issues on Safari.
I confirm I have signed the Contributor License Agreement.

Decided to remove the commented code - this is public code after all!


Configurable/Custom Mediator Service

Description of feature / user story
Allows the user to scan/setup a mediator service of their choice
Acceptance Criteria
Wireframes or relevant image assets / links

This would be a really important feature, Ledger and Meditator should be configurable.


Feature/gradle

Goal of this PR: add Gradle build similar to Maven build so users can select their prefered way of working.
See changes in README.

@eskatos thanks a lot for all your feedback!!!!! will look into it asap

@eskatos thanks for your thorough review! I followed all your remarks, only one change needed which was not working:
tasks.register("copyDistribution", Copy) {
instead of proposed
tasks.register("copyDistribution", CopySpec) {

Ah right, wrong muscle memory ;)
Glad I could help!


f32 and Option<f32> are decoded as LargeUtf8 instead of f32

Hi, I have a struct with several f32 and Option<f32> fields and serialize_into_fields  generate LargeUtf8 fields instead of the expected f32.  Is there any interaction with serde_as?  I understand that the json file itself is using strings for these numbers (I have no control over it hence my use of the serde_with crate) but the official parquet crate works.  And serialize_into_fields is using the type with f32, not the text version which has strings.
#[serde_as]
#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct Asd {
    #[serde(skip_deserializing)]
    pub time: i64,
    pub journey: u32,
    pub ident: String,
    pub model: Option<String>,
    pub source: String,
    pub location: u32,
    pub timestamp: String,
    #[serde_as(as = "DisplayFromStr")]
    pub latitude: f32,
    #[serde_as(as = "DisplayFromStr")]
    pub longitude: f32,
    pub altitude: Option<i16>,
    pub elevation: Option<i32>,
    pub gps: Option<u32>,
    pub rssi: Option<i32>,
    #[serde_as(as = "Option<DisplayFromStr>")]
    pub home_lat: Option<f32>,
    #[serde_as(as = "Option<DisplayFromStr>")]
    pub home_lon: Option<f32>,
    pub home_height: Option<f32>,
    pub speed: f32,
    pub heading: f32,
    pub station_name: Option<String>,
    #[serde_as(as = "Option<DisplayFromStr>")]
    pub station_latitude: Option<f32>,
    #[serde_as(as = "Option<DisplayFromStr>")]
    pub station_longitude: Option<f32>,
}

DEBUG parquet2 json=[Asd { time: 0, journey: 36354, ident: "08RDE92001041T", model: Some("MavicPro"), source: "as", location: 1674998, timestamp: "2023-10-03 01:00:52", latitude: 44.862114, longitude: -0.523638, altitude: Some(120), elevation: Some(63), gps: None, rssi: None, home_lat: Some(44.862034), home_lon: Some(-0.523769), home_height: Some(59.0), speed: 2.0, heading: 50.0, station_name: Some("0QRDKC2R038370"), station_latitude: Some(44.831463), station_longitude: Some(-0.702713) }, Asd { time: 0, journey: 36351, ident: "08RDE92001041T", model: Some("MavicPro"), source: "as", location: 1674936, timestamp: "2023-10-03 00:29:19", latitude: 44.86231, longitude: -0.523483, altitude: Some(166), elevation: Some(63), gps: None, rssi: None, home_lat: Some(44.862083), home_lon: Some(-0.523792), home_height: Some(105.0), speed: 16.0, heading: 18.0, station_name: Some("0QRDKC2R038370"), station_latitude: Some(44.83142), station_longitude: Some(-0.702685) }]
 parquet2::read_json{base="small"}
 DEBUG parquet2 data=[Asd { time:<PHONE_NUMBER>, journey: 36354, ident: "08RDE92001041T", model: Some("MavicPro"), source: "as", location: 1674998, timestamp: "2023-10-03 01:00:52", latitude: 44.862114, longitude: -0.523638, altitude: Some(120), elevation: Some(63), gps: None, rssi: None, home_lat: Some(44.862034), home_lon: Some(-0.523769), home_height: Some(59.0), speed: 2.0, heading: 50.0, station_name: Some("0QRDKC2R038370"), station_latitude: Some(44.831463), station_longitude: Some(-0.702713) }, Asd { time:<PHONE_NUMBER>, journey: 36351, ident: "08RDE92001041T", model: Some("MavicPro"), source: "as", location: 1674936, timestamp: "2023-10-03 00:29:19", latitude: 44.86231, longitude: -0.523483, altitude: Some(166), elevation: Some(63), gps: None, rssi: None, home_lat: Some(44.862083), home_lon: Some(-0.523792), home_height: Some(105.0), speed: 16.0, heading: 18.0, station_name: Some("0QRDKC2R038370"), station_latitude: Some(44.83142), station_longitude: Some(-0.702685) }]
 parquet2::write_chunk{base="small"}
    2.00ms DEBUG parquet2 schema=Schema { fields: [Field { name: "time", data_type: Int64, is_nullable: false, metadata: {} }, Field { name: "journey", data_type: UInt32, is_nullable: false, metadata: {} }, Field { name: "ident", data_type: LargeUtf8, is_nullable: false, metadata: {} }, Field { name: "model", data_type: LargeUtf8, is_nullable: true, metadata: {} }, Field { name: "source", data_type: LargeUtf8, is_nullable: false, metadata: {} }, Field { name: "location", data_type: UInt32, is_nullable: false, metadata: {} }, Field { name: "timestamp", data_type: LargeUtf8, is_nullable: false, metadata: {} }, Field { name: "latitude", data_type: LargeUtf8, is_nullable: false, metadata: {} }, Field { name: "longitude", data_type: LargeUtf8, is_nullable: false, metadata: {} }, Field { name: "altitude", data_type: Int16, is_nullable: true, metadata: {} }, Field { name: "elevation", data_type: Int32, is_nullable: true, metadata: {} }, Field { name: "gps", data_type: Null, is_nullable: true, metadata: {} }, Field { name: "rssi", data_type: Null, is_nullable: true, metadata: {} }, Field { name: "home_lat", data_type: LargeUtf8, is_nullable:true, metadata: {} }, Field { name: "home_lon", data_type: LargeUtf8, is_nullable: true, metadata: {} }, Field { name: "home_height", data_type: Float32, is_nullable: true, metadata: {} }, Field { name: "speed", data_type: Float32, is_nullable: false, metadata: {} }, Field { name: "heading", data_type: Float32, is_nullable: false, metadata: {} }, Field { name: "station_name", data_type: LargeUtf8, is_nullable: true, metadata: {} }, Field { name: "station_latitude", data_type: LargeUtf8, is_nullable: true, metadata: {} }, Field { name: "station_longitude", data_type: LargeUtf8, is_nullable: true, metadata: {} }], metadata: {} }

cf. https://github.com/keltia/fetiche-rs/blob/develop/formats/examples/parquet2.rs

Hey.
TBH. I am not 100 % sure, what serde_as is doing exactly, but I assume it serializes these values as strings. Therefore serde_arrow picks them up as strings as wells. To test what's going on, you could try to serialize this struct to JSON. If these fields are written as strings, they also appear as strings to serde_arrow.

The original JSON is using strings and not the real types. serde_as enables me to have the real type in the struct and converts on the fly.  What I need is serde_arrow to do the same in fact, or to integrate with serde_as (from the serde_with crate BTW).

Is there any reason you require the additional annotations? serde_json will observe the float values with their full precision also without annotations.

No, if I do not use serde_as annotations, serde_json will takes them as strings, not f32.

Okay. That's tricky. I always planned to allow further customization of how values are handled. At the moment this is not possible. You would need to make sure that the relevant values are serialized as floats by serde. Probably the easiest way would be to create a second struct with the same fields, but without the serde_as annotations.
Another option would be to skip serde_as and roll your own impl, similar to (Rust Playground) :

fn deserialize_f32_from_str<'de, D: Deserializer<'de>>(deserializer: D) -> Result<f32, D::Error> {
  let s = String::deserialize(deserializer)?;
  let val = f32::from_str(&s).map_err(|e| D::Error::custom(e.to_string()))?;
  Ok(val)
}

fn deserialize_optional_f32_from_str<'de, D: Deserializer<'de>>(deserializer: D) -> Result<Option<f32>, D::Error> {
  let Some(s) = Option::<String>::deserialize(deserializer)? else {
    return Ok(None);
  };
  let val = f32::from_str(&s).map_err(|e| D::Error::custom(e.to_string()))?;
  Ok(Some(val))
}

#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct Asd {
    #[serde(skip_deserializing)]
    pub time: i64,
    pub journey: u32,
    pub ident: String,
    pub model: Option<String>,
    pub source: String,
    pub location: u32,
    pub timestamp: String,
    #[serde(deserialize_with = "deserialize_f32_from_str")]
    pub latitude: f32,
    #[serde(deserialize_with = "deserialize_f32_from_str")]
    pub longitude: f32,
    pub altitude: Option<i16>,
    pub elevation: Option<i32>,
    pub gps: Option<u32>,
    pub rssi: Option<i32>,
    #[serde(deserialize_with = "deserialize_optional_f32_from_str")]
    pub home_lat: Option<f32>,
    #[serde(deserialize_with = "deserialize_optional_f32_from_str")]
    pub home_lon: Option<f32>,
    pub home_height: Option<f32>,
    pub speed: f32,
    pub heading: f32,
    pub station_name: Option<String>,
    #[serde(deserialize_with = "deserialize_optional_f32_from_str")]
    pub station_latitude: Option<f32>,
    #[serde(deserialize_with = "deserialize_optional_f32_from_str")]
    pub station_longitude: Option<f32>,
}

Thanks. That's actually a really cool solution. Thanks for sharing. Great to hear you found a solution.


Add plot_ prefix back to plotting methods

Is your feature request related to a problem?
Now that the plotting methods are a method of either Table or Column, it would be nice to have them grouped together.
Desired solution
Add the plot_ prefix back to those methods.
Possible alternatives (optional)
No response
Screenshots (optional)
No response
Additional Context (optional)
No response

:tada: This issue has been resolved in version 0.11.0 :tada:
The release is available on:

v0.11.0
GitHub release

Your semantic-release bot :package::rocket:


Minified version is missing

I feel like there should be a minified version of css file as well. There is one for different frames. For example bootstrap comes with one as well.

@jossef is this repo still supported by you or no plans for the future?

fixed in https://github.com/jossef/material-design-icons-iconfont/releases/tag/v6.3.0


"Location" might not give understandable information

Situation:
I have a package with five METS-files: A root METS and  four representation METS.
I get a lot of errors.
Problem:
How do I know which METS-file(s) cause(s) the problems?

Maybe it is the location that I am not able to interpret:

Reported in the wrong repository...


Problems with Authorize and AuthenticationSchemes property in ASP.NET Core 2.0

Hi,
I hace and ASP.NET Core 2.0 Api and today I've configured the security. When I've configured the controllers with Authorize and set AuthenticationSchemes property with "Bearer", NSwag throws an exception:
System.Reflection.CustomAttributeFormatException: 'AuthenticationSchemes' property specified was not found. ---> System.Reflection.CustomAttributeFormatException: 'AuthenticationSchemes' property specified was not found.
    [Route("api/[controller]")]
    [ApiVersion("1.0")]
    [Authorize(AuthenticationSchemes = JwtBearerDefaults.AuthenticationScheme)]
    public class CategoriesController : ControllerBase

Any ideas???
Regards!!!

Hi,
I've tried to encapsulate AuthorizeAttribute in my own
    public class JwtAuthorizeAttribute : JwtAuthorizeAttribute
    {
        public JwtAuthorizeAttribute()
        {
            AuthenticationSchemes = JwtBearerDefaults.AuthenticationScheme;
        }
    }

But...
System.MissingMethodException: Method not found: 'Void Microsoft.AspNetCore.Authorization.AuthorizeAttribute.set_AuthenticationSchemes(System.String)'.
Regards!

Works with ActiveAuthenticationSchemes but is obsolete and you have to use AuthenticationSchemes:

Regards!

Do you have a stacktrace?

Hi @RSuter
Changing the order in referencepaths works:
Initial
 <Exec Command="$(NSwagExe_Core20) webapi2swagger /assembly:$(OutDir)$(ProjectName).dll /DefaultPropertyNameHandling:CamelCase /output:swagger.json /referencepaths:"%USERPROFILE%\\.nuget\\packages,C:\\Program Files\\dotnet\\store\\x64\\netcoreapp2.0"" />

Final
 <Exec Command="$(NSwagExe_Core20) webapi2swagger /assembly:$(OutDir)$(ProjectName).dll /DefaultPropertyNameHandling:CamelCase /output:swagger.json /referencepaths:"C:\\Program Files\\dotnet\\store\\x64\\netcoreapp2.0,%USERPROFILE%\\.nuget\\packages"" />

I don't know why if I use %programfiles% msbuild try to look for in Program Files x86 for this reason I'm using full path :(
Do you want to close the issue or do you need aditional info?
Regards!

Assembly loading works best if the correct runtime is used... so i think it's fixed...

I think if you first "dotnet publish" and use the correct runtime Core20 then you dont need referencepaths...


Error using Blazor Server with Proxy not allowing WebSocket-Connection

After upgrading to the RTM-Version of AspNetCore, the connection-fallback to LongPolling doesn't work again. (this was fixed in Preview7)
Console output is (nothing happens on the screen):
WebSocket connection to 'wss://xxx.domain.ext/_blazor?id=zqmUOV0jdiG8Tdz5FhuHCA' failed: Error in connection establishment: net::ERR_SOCKS_CONNECTION_FAILED

Error: Failed to start the transport 'WebSockets': Error: There was an error with the transport.

Warning: Call to HttpConnection.stopConnection(Error: WebSocket closed with status code: 1006 ().) was ignored because the connection hasn't yet left the in the connecting state.

Error: Failed to start the transport 'LongPolling': TypeError: Cannot set property 'onreceive' of undefined

Error: Failed to start the connection: Error: Unable to connect to the server with any of the available transports. WebSockets failed: Error: There was an error with the transport. ServerSentEvents failed: Error: 'ServerSentEvents' does not support Binary. LongPolling failed: TypeError: Cannot set property 'onreceive' of undefined

Error: Error: Unable to connect to the server with any of the available transports. WebSockets failed: Error: There was an error with the transport. ServerSentEvents failed: Error: 'ServerSentEvents' does not support Binary. LongPolling failed: TypeError: Cannot set property 'onreceive' of undefined

Uncaught (in promise) Error: Cannot send data if the connection is not in the 'Connected' State.
    at e.send (blazor.server.js:formatted:2228)
    at e.sendMessage (blazor.server.js:formatted:855)
    at e.sendWithProtocol (blazor.server.js:formatted:859)
    at blazor.server.js:formatted:885
    at new Promise (<anonymous>)
    at e.invoke (blazor.server.js:formatted:880)
    at e.<anonymous> (blazor.server.js:formatted:10537)
    at blazor.server.js:formatted:10491
    at Object.next (blazor.server.js:formatted:10504)
    at blazor.server.js:formatted:10410

I wish I could give more information.

Error: Failed to start the transport 'LongPolling': TypeError: Cannot set property 'onreceive' of undefined

/cc @BrennanConroy

Dupe of https://github.com/aspnet/AspNetCore/issues/14075
We're fixing this in 3.0.X


1.1.4 crashing when pause/resume charging

[12/03/2022, 22:51:18] TypeError: Converting circular structure to JSON
--> starting at object with constructor 'ClientRequest'
|     property 'socket' -> object with constructor 'TLSSocket'
--- property '_httpMessage' closes the circle
at JSON.stringify ()
at easeeAPI.command (/usr/local/lib/node_modules/homebridge-easee/easeeapi.js:274:100)
at runMicrotasks ()
at processTicksAndRejections (node:internal/process/task_queues:96:5)
[12/03/2022, 22:51:18] Got SIGTERM, shutting down Homebridge...

reverted back to 1.1.3 and all is working as expected. maybe a typo in your new code?

Sorry about that. corrected in 1.1.5.


Attribute Error in the Austrian tutorial regarding Vintage and Active Year

Hi. I am trying to run the Austrian RES example in Jupyter notebook (I tried in Spyder as well) and when I execute the command lines
'year_df = scenario.vintage_and_active_years()
vintage_years, act_years = year_df['year_vtg'], year_df['year_act']'
I get these errors.

It is apparently a problem with Java conversions. Does it have anything to do with my JDK version?
Help is much appreciated.

Can you please run pip list on a command-line and tell us which versions of ixmp, message-ix, and JPype1 you have installed?

I have the following version installed.
ixmp 2.0.0
message ix 2.0.0
JPype1 0.7.2

JPype1 0.7.2

We have some known issues with JPype 0.7.2 (eg. iiasa/ixmp#279) although the symptoms are not the same as this one. Can you please try pip install JPype1==0.7.1?
Also CC @zikolach since the Java code to be returning an string where some kind if list is expected.

@khaeru java.lang.String[] - array of strings, getCatEle returns array of strings from Java. I would check what kind of exception is triggered in try clause as according to documentation arrays should be converted to JArray:
def to_pylist(jlist):
    """Convert Java list types to :class:`list`."""
    try:
        return list(jlist[:])
    except Exception:
        # java.LinkedList
        return list(jlist.toArray()[:])

I installed Jpype 0.71 and it showed an error 'graphviz not installed'. I later installed graphviz as well.

However when I tried to run messageix again this error popped up.

Changing string conversion to False does not solve the problem.

@aaronliuqy not sure (as not use Windows), but maybe https://github.com/iiasa/message_data/issues/108#issuecomment-597663689 can help you:

Note this iiasa/ixmp#278 (comment): if conda installs JPype1 version 0.7.2 and then pip install JPype1==0.7.1 downgrades it, the tests seem to fail. OTOH if JPype1 0.7.1 is installed into a clean environment, the tests pass.

@zikolach when I install Jpype 0.7.1 in a clean environment (i.e reinstalled anaconda) I still need to install messageix package which comes with Jpype1 version 0.7.2, then it comes back to the original error. Could you maybe clarify what is a 'clean environment' please. Thank you

I don't know what was meant in the original reference, but can you try pip uninstall --yes JPype1 && pip install --no-cache-dir JPype1==0.7.1

I have tried the above command and this is the error I get. Jpype1 is now in version 0.71.

@aaronliuqy just to make sure - did you restart kernel in jupyter after re-installing JPype1 package?

@zikolach Yes I did. I have tried it in Spyder as well and the same error pops up.

Hi. Installing Jpype version 0.70 solves all the mentioned issues above. Thank you very much for your help.


conan fails when running under haiku

I'm trying to build xdispatch under haiku, so cloned xdispatch repository and installed conan 2.2.2 using pip, then also mz-cmaketools (into build/ ) but this is the fail:

It seems conan cannot detect the environment (it doesn't know anything about haiku) and stops.
Any hints?
(I know here you don't develop/fix conan, but maybe I can workaround this issue and do the build avoiding conan, or even fooling it)

Yeah you would need to manually define a matching conan profile in build/Conan and update build/conan.cmake to pick it accordingly. I opted to not use Conan's auto detection but manually maintain the profiles.
Alternatively you can try disabling the inclusion of conan.cmake in CMakeLists.txt:30

Resolved by cba7f72:

Will fall back to using the system default profile if unavailable
conan is only enabled when configuring via ./build/generator.sh compared to using CMake directly and can ultimately be controlled using cache variable MZ_USE_CONAN


⚠ Cioos Atlantic CKAN has degraded performance

In e2f5199, Cioos Atlantic CKAN (https://catalogue.cioosatlantic.ca/) experienced degraded performance:

HTTP code: 200
Response time: 12838 ms

Resolved: Cioos Atlantic CKAN performance has improved in 30f4efe.


CesiumSunSky will cause crash when no viewport activated in Editor

When I switch the Editor layout between viewport tab and CeiumPanel tab, CesiumSunSky causes a crash if no viewport activated in the Editor. BTW, I'm using CesiumForUnreal v2.6.0 and UE 5.4.

pViewport will be nullptr if no viewport activated.

Hi @yuziqii,
Thanks for the catch! We should definitely check if pViewport is valid before dereferencing it. Would you be able to open a PR with the change?

Of course! Here is the PR. Let me know if you need any further adjustments.

Closed by #1460 !


lumary-us-sd4e YAML Creation Issue

I have lumary-us-sd4e but the 6in version. I was able to flash successfully, however not getting all the option there should be.
Says "Module 'cmod': rgbcw doesn't match platform cwww"
Not sure where to go from here.

This is a bit beyond the scope of CloudCutter, as you're now dealing with 3rd party firmware.  This device is "RGB AI", which is another name for RGBIC, which I'm not sure is supported by LibreTiny/ESPHome yet, as it needs support added to a 3rd party component fastled.

This is a bit beyond the scope of CloudCutter, as you're now dealing with 3rd party firmware. This device is "RGB AI", which is another name for RGBIC, which I'm not sure is supported by LibreTiny/ESPHome yet, as it needs support added to a 3rd party component ~fastled~ neopixel.
You will need the data in Device configuration to configure the rgbic once it is supported.

This device has a profile on this repository, so pretty misleading if it’s not supported. That profile should be removed…

It is supported by Cloudcutter - you were even able to flash it successfully.
It is also supported for local control ("detach" mode instead of flashing).
Keep in mind that Cloudcutter and UPK2ESPHome or LibreTiny are completely different things. UPK is just a program that generates suggested config files, based on Cloudcutter profiles - they're not tied in any way.

This is a bit beyond the scope of CloudCutter, as you're now dealing with 3rd party firmware. This device is "RGB AI", which is another name for RGBIC, which I'm not sure is supported by LibreTiny/ESPHome yet, as it needs support added to a 3rd party component ~fastled~ neopixel.
You will need the data in Device configuration to configure the rgbic once it is supported.

This device has a profile on this repository, so pretty misleading if it’s not supported. That profile should be removed…

CloudCutter supports either cutting or flashing of the device, but it does not support any 3rd party firmware you decide to install, or the configuration afterwards.  You should use the support vehicles of those respective firmwares instead.  This issue section is just for issues related to the cutting process itself.  We can't test the functionality of all features of all firmwares.  This is why the cutting option exists for where 3rd party firmware doesn't have full functionality.  Unfortunately, there is no going back once you've flashed 3rd party firmware however.

This is a bit beyond the scope of CloudCutter, as you're now dealing with 3rd party firmware. This device is "RGB AI", which is another name for RGBIC, which I'm not sure is supported by LibreTiny/ESPHome yet, as it needs support added to a 3rd party component ~fastled~ neopixel.
You will need the data in Device configuration to configure the rgbic once it is supported.

This device has a profile on this repository, so pretty misleading if it’s not supported. That profile should be removed…

CloudCutter supports either cutting or flashing of the device, but it does not support any 3rd party firmware you decide to install, or the configuration afterwards. You should use the support vehicles of those respective firmwares instead. This issue section is just for issues related to the cutting process itself. We can't test the functionality of all features of all firmwares. This is why the cutting option exists for where 3rd party firmware doesn't have full functionality. Unfortunately, there is no going back once you've flashed 3rd party firmware however.

That makes sense


reporting operator not connecting to hive backed by pgsql

Per the documentation, I am configuring a new install of metering to use a pgsql pod deployed to the same OpenShift project:
$ oc get svc | grep postgres
postgresql-95-centos7                      ClusterIP   <IP_ADDRESS>    <none>        5432/TCP              5d

I then created this config:
apiVersion: metering.openshift.io/v1alpha1
kind: Metering
metadata:
  name: "operator-metering"
spec:
  presto:
    spec:
      hive:
        config:
          dbConnectionURL: "jdbc:postgresql://postgresql-95-centos7:5432/metering"
          dbConnectionDriver: "org.postgresql.Driver"
          dbConnectionUsername: "metering"
          dbConnectionPassword: "my-password-string"

Pointed to it with the METERING_CR_FILE environment variable, and ran hack/openshift-install.sh.
After the install runs, the reporting-operator starts up but never shows ready. The logs for that pod show:
time="2019-01-16T15:49:45Z" level=info msg="adding ReportGenerationQuery pod-memory-request" app=metering
time="2019-01-16T15:49:45Z" level=info msg="adding ReportGenerationQuery pod-memory-request-raw" app=metering
time="2019-01-16T15:49:45Z" level=info msg="adding ReportGenerationQuery pod-memory-request-vs-node-memory-allocatable" app=metering
time="2019-01-16T15:49:45Z" level=info msg="adding ReportGenerationQuery pod-memory-usage" app=metering
time="2019-01-16T15:49:45Z" level=info msg="adding ReportGenerationQuery pod-memory-usage-raw" app=metering
time="2019-01-16T15:49:46Z" level=info msg="starting HTTP server" app=metering
time="2019-01-16T15:49:46Z" level=info msg="testing ability to write to Presto" app=metering
time="2019-01-16T15:49:46Z" level=info msg="HTTP API server listening with TLS on <IP_ADDRESS>:8080" app=metering
time="2019-01-16T15:50:34Z" level=error msg="cannot create Presto table operator_health_check" app=metering component=testWriteToPresto error="timed out waiting for the condition"
time="2019-01-16T15:51:41Z" level=info msg="\"GET https://<IP_ADDRESS>:8080/ready HTTP/2.0\" from <IP_ADDRESS>:56882 - 500 50B in 594.958µs" app=metering component=api
time="2019-01-16T15:52:41Z" level=info msg="\"GET https://<IP_ADDRESS>:8080/ready HTTP/2.0\" from <IP_ADDRESS>:57172 - 500 50B in 49.564µs" app=metering component=api
time="2019-01-16T15:53:41Z" level=info msg="\"GET https://<IP_ADDRESS>:8080/ready HTTP/2.0\" from <IP_ADDRESS>:57456 - 500 50B in 43.356µs" app=metering component=api
time="2019-01-16T15:54:41Z" level=info msg="\"GET https://<IP_ADDRESS>:8080/ready HTTP/2.0\" from <IP_ADDRESS>:57754 - 500 50B in 33.625µs" app=metering component=api

I encountered this with release-0.11, then noticed .13 was released, and I replicated the problem with 0.13 after deleting and re-creating my project.  The only difference is I don't see the timeout message in .13:
time="2019-01-16T16:13:06Z" level=info msg="testing ability to write to Presto" app=metering
time="2019-01-16T16:13:06Z" level=info msg="HTTP API server listening with TLS on <IP_ADDRESS>:8080" app=metering
time="2019-01-16T16:14:26Z" level=info msg="\"GET https://<IP_ADDRESS>:8080/ready HTTP/2.0\" from <IP_ADDRESS>:33732 - 500 50B in 373.63µs" app=metering component=api
time="2019-01-16T16:15:26Z" level=info msg="\"GET https://<IP_ADDRESS>:8080/ready HTTP/2.0\" from <IP_ADDRESS>:34030 - 500 50B in 49.415µs" app=metering component=api
time="2019-01-16T16:16:26Z" level=info msg="\"GET https://<IP_ADDRESS>:8080/ready HTTP/2.0\" from <IP_ADDRESS>:34332 - 500 50B in 38.505µs" app=metering component=api
time="2019-01-16T16:17:26Z" level=info msg="\"GET https://<IP_ADDRESS>:8080/ready HTTP/2.0\" from <IP_ADDRESS>:34626 - 500 50B in 40.387µs" app=metering component=api
time="2019-01-16T16:18:26Z" level=info msg="\"GET https://<IP_ADDRESS>:8080/ready HTTP/2.0\" from <IP_ADDRESS>:34922 - 500 50B in 42.805µs" app=metering component=api
time="2019-01-16T16:19:26Z" level=info msg="\"GET https://<IP_ADDRESS>:8080/ready HTTP/2.0\" from <IP_ADDRESS>:35224 - 500 50B in 43.796µs" app=metering component=api
time="2019-01-16T16:20:26Z" level=info msg="\"GET https://<IP_ADDRESS>:8080/ready HTTP/2.0\" from <IP_ADDRESS>:35530 - 500 50B in 43.522µs" app=metering component=api
time="2019-01-16T16:20:49Z" level=info msg="got signal terminated, performing shutdown"

If I shell into the postgres pod, I do see that a schema was populated:
postgres=# \c metering
You are now connected to database "metering" as user "postgres".
metering=# \dt
                   List of relations
 Schema |           Name            | Type  |  Owner
--------+---------------------------+-------+----------
 public | BUCKETING_COLS            | table | metering
 public | CDS                       | table | metering
 public | COLUMNS_V2                | table | metering
 public | DATABASE_PARAMS           | table | metering
 [snip]

So I at least got my DB connection parameters right.

reporting-logs-debug.txt
If I'm reading this right, reporting-operator is trying to connect to service presto:8080, which fails. If I rsh to the reporting-operator pod and curl presto:8080 I get a http 301 response, which sounds like it's working. That may have been transient, while the presto pod came up.
There are several failures in the hive-server-0 log as it tries and fails to connect to hive-metastore, but, likely after the metastore pod comes up, those messages stop and the last message looks like a successful connection.
I note this in the hive-metastore-0 log:
19/02/04 15:14:12 [pool-7-thread-7]: WARN metastore.HiveMetaStore: Location: hdfs://hdfs-namenode-proxy:9820/operator_metering/storage/metering_health_check specified for non-external table:operator_health_check
that seems related to the test that's failing from the reporting-operator.
It also looks like metastore is successfylly connected to postgres:
19/02/04 15:19:49 [pool-7-thread-25]: INFO metastore.ObjectStore: ObjectStore, initialize called
19/02/04 15:19:49 [pool-7-thread-25]: INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is POSTGRES
19/02/04 15:19:49 [pool-7-thread-25]: INFO metastore.ObjectStore: Initialized ObjectStore
This makes it sound like an issue with my config? But if that's the case, I'm not sure how to correct it.

$ oc get pods NAME                                  READY     STATUS    RESTARTS   AGE hdfs-datanode-0                       1/1       Running   0          30m hdfs-namenode-0                       1/1       Running   0          30m hive-metastore-0                      1/1       Running   0          30m hive-server-0                         1/1       Running   0          30m metering-operator-558d78b4cf-7jq29    2/2       Running   0          31m postgresql-95-centos7-2-zjmsb         1/1       Running   0          1h presto-coordinator-66b7fc44fc-xh9mg   1/1       Running   0          30m reporting-operator-655889659d-979mn   0/1       Running   3          30m $ oc get services NAME                                                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)               AGE glusterfs-dynamic-hdfs-datanode-data-hdfs-datanode-0   ClusterIP   <IP_ADDRESS>   <none>        1/TCP                 59m glusterfs-dynamic-hdfs-namenode-data-hdfs-namenode-0   ClusterIP   <IP_ADDRESS>   <none>        1/TCP                 59m glusterfs-dynamic-hive-metastore-db-data               ClusterIP   <IP_ADDRESS>     <none>        1/TCP                 59m glusterfs-dynamic-pvc-t8q8l                            ClusterIP   <IP_ADDRESS>   <none>        1/TCP                 1h hdfs-datanode                                          ClusterIP   None             <none>        9866/TCP              31m hdfs-datanode-web                                      ClusterIP   <IP_ADDRESS>     <none>        9864/TCP              31m hdfs-namenode                                          ClusterIP   None             <none>        9820/TCP              31m hdfs-namenode-proxy                                    ClusterIP   <IP_ADDRESS>   <none>        9820/TCP              31m hdfs-namenode-web                                      ClusterIP   <IP_ADDRESS>   <none>        9870/TCP              31m hive-metastore                                         ClusterIP   <IP_ADDRESS>   <none>        9083/TCP              31m hive-server                                            ClusterIP   <IP_ADDRESS>     <none>        10000/TCP,10002/TCP   31m postgresql-95-centos7                                  ClusterIP   <IP_ADDRESS>   <none>        5432/TCP              1h presto                                                 ClusterIP   <IP_ADDRESS>   <none>        8080/TCP              31m presto-metrics                                         ClusterIP   <IP_ADDRESS>     <none>        8082/TCP              31m reporting-operator                                     ClusterIP   <IP_ADDRESS>    <none>        8080/TCP              31m reporting-operator-metrics                             ClusterIP   <IP_ADDRESS>      <none>        8082/TCP              31m
in case that helps line up pod/ port/ip/service

According to the logs this is hitting a known issue where hive-server is getting a "No route to host" error when talking to hive-metastore. Try deleting the hive-server-0 pod, and it will automatically be re-created and that typically allows it to connect. We're still investigating the underlying cause for this.

So now it's getting read timeouts. You can't currently adjust the timeout for hive-server talking to metastore (not configurable yet), but you can likely increase the amount of resources you give to hive-metastore.
https://github.com/operator-framework/operator-metering/blob/master/Documentation/tuning.md covers increasing resource limits.

I had applied my own tunings:
apiVersion: metering.openshift.io/v1alpha1
kind: Metering
metadata:
  name: "operator-metering"
spec:
  reporting-operator:
    spec:
      resources:
        requests:
          memory: "300Mi"
          cpu: "50m"
        limits:
          memory: "300Mi"
          cpu: "100m"
  presto:
    spec:
      hive:
        config:
          dbConnectionURL: "jdbc:postgresql://postgresql-95-centos7:5432/metering"
          dbConnectionDriver: "org.postgresql.Driver"
          dbConnectionUsername: "metering"
          dbConnectionPassword: "my-password-string"
      presto:
        coordinator:
          resources:
            requests:
              memory: "2Gi"
              cpu: "2"
            limits:
              memory: "2Gi"
              cpu: "2"
        worker:
          resources:
            requests:
              memory: "2Gi"
              cpu: "2"
            limits:
              memory: "2Gi"
              cpu: "2"

Now I'm adding my postgres config parameters to the recommended-resource-limits, and setting datanode replicas to 1 since we only have one compute node available in our lab, but otherwise I'm using it verbatim:
https://raw.githubusercontent.com/operator-framework/operator-metering/master/manifests/metering-config/recommended-resource-limits.yaml
I tore down my metering project and re-created it with that config. I'm still getting timeouts from hive-metastore:
hive-server.log
Is it worth it to embiggen hive-metastore even more?

You're currently not increasing hive limits at all, only presto.

sorry, my last comment wasn't clear. I'm now using all the tuning values here:
https://raw.githubusercontent.com/operator-framework/operator-metering/master/manifests/metering-config/recommended-resource-limits.yaml
except hdfs replicas is only 1 not 3, since I don't have the compute nodes needed to run 3.
At 1.5Gi for hive metastore, I'm still getting the read errors in the log above.

As I suggested before, I would try to increase the value further to see if it helps. It could also be a networking issue, since the default timeout seems to be 5 minutes I believe (trying to verify) so it seems strange to be getting a timeout like that between hive-server and hive-metastore. You may also want to increase hive-server memory to see if it helps.


use init service provider for Amazon Linux

release 0.9.6 breaks (https://github.com/elastic/puppet-elasticsearch/commit/066f70c99d7010ec9ddfe07ec87f4671765c68c4) service management on Amazon Linux since it does not have systemd installed
this change creates separate case for Amazon Linux to allow to define "init" as service provider
facter output from AWS linux
# facter  operatingsystem
Amazon
# facter  operatingsystemmajrelease
2015

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run; then  say 'jenkins, test it'.

Thank you for re-doing the commit.
I'll let the test system do its thing and if everything is green i'll merge it.
I'm planning to do a release next week.
Thanks!

jenkins, test it

Thank you again for the patch.


UNSCOPED: Caps are layering on top of roads, fix roads layer order

Let's @bcamper, @zerebubuth, @sensescape and @nvkelso brainstorm about this one. I suspect it will require combination of server and client work.

I think the issue here is not the entire cap, but just the outline stroke
of the cap. Those are drawn directly under the line fill by default, but we
can control the order of it separately in the scene file if we have the
right data to filter on.
On Friday, September 25, 2015, Nathaniel V. KELSO<EMAIL_ADDRESS>wrote:

Let's @bcamper https://github.com/bcamper, @zerebubuth
https://github.com/zerebubuth, @sensescape
https://github.com/sensescape and @nvkelso https://github.com/nvkelso
brainstorm about this one. I suspect it will require combination of server
and client work.
—
Reply to this email directly or view it on GitHub
https://github.com/tangrams/eraser-map/issues/2#issuecomment-143150628.

@sensescape do you have good examples where I can see this?

Here's one example:
http://localhost:8000/#17/37.80199/-122.45861

Another:
http://localhost:8000/#17/40.72369/-74.00750

I would like the round cap here to smoothen the transition for the different road types.
I'm not sure if this is working:
https://github.com/tangrams/eraser-map/blob/gh-pages/eraser-map.yaml#L1104

http://localhost:8000/#17/37.76917/-122.40942

http://localhost:8000/#17/40.75633/-73.99643

@sensescape I believe that round rule is working based on most of your screenshots having round caps :)  But it may be overridden in some cases like this, where it's explicitly set to butt? https://github.com/tangrams/eraser-map/blob/gh-pages/eraser-map.yaml#L1123

@bcamper I was thinking about the transition between roads, and wondering how flexible we can be with the end caps? For example, if we have the topology (hi, TopoJSON!) then we could find the max(line_width) at any shared point and have the incoming segments flare out to that width. It would be even more awesome if we could also do avg(fill_color) and blend to that :smile:
It looks like we might have the information in the (TopoJSON) tiles to do this, and it seems like the kind of thing that might be possible to do at some point in the future. What do you think? Something we might have a chat about in November?

@zerebubuth yes I wonder if something cool could be done there. Thinking about a generic Tangram feature, the most straightforward would be to specify some kind of tapering options when drawing lines, like:

start and end widths, with continuous tapering along the whole line
tapering caps, e.g. taper from the normal line width up/down to a target taper width, but only at the end of the line segment (how short/long the taper is could be configurable)

This simplest form of this would have the start/end data encoded in the tile already, so it could just be plugged into the renderer. I can also see a more advanced feature where Tangram can use topology to derive that (not trivial but plausible...), though I think we would still have problems at places like tile borders. Let's talk more!

@zerebubuth I guess what I'm asking for is adjacency data in the tiles, like road types that connect at the start and/or end of a road line, so that we could filter and style on that. Like denormalizing the topology into into the geometry properties. Using actual topology could be cool too, though it also implies a format like TopoJSON that can encode it? We'd be going offroad with the MVT format at that point if there's no current way to encode topology (it's protobuf though so we could probably additional optional topology info).

@bcamper We can try recovering adjacency data in the same way we do for TopoJSON tiles. The difficulty might be if we have any joins which don't occur at the first or last point in the linestring. Keeping the topology through the tile generation process is something that I'm really interested in doing for other reasons (e.g: simplifying boundaries properly, generalising networks without introducing gaps, etc...), so perhaps this has to be hitched to that.
Nice idea! Having TopoMVT would be really interesting and might be a big win in file size for some cases - in the same way that TopoJSON can be over GeoJSON.

I need to draw some pictures.

The scene file now uses a different approach. Fixing the root issue of this original is not a 1.0 launch feature.

Here's the same area today:


[FEATURE] It seems that there is currently only i18n support for English?

Is your feature request related to a problem?
A clear and concise description of what the problem is, e.g. I'm always frustrated when [...]

I saw https://github.com/opensearch-project/dashboards-i18n/blob/main/DEVELOPER_GUIDE.md which claims to implement i18n for multiple languages via files under the translations folder.

Currently, we support en/en-US (English), es/es-LA (Spanish), fr/fr-FR (French), de/de-DE (German), ja/ja-JP (Japanese), ko/ko-KR (Korean), ru/ru-RU (Russian), zh/zh-CN (Chinese). If your locale is not in it, please open an issue in OpenSearch Dashboards.

But I looked at https://github.com/opensearch-project/dashboards-i18n/tree/main/translations and it seems there is only text in English?

Am I to understand that i18n work for languages other than English hasn't actually started yet?
Or what is the relationship between the current git and https://github.com/opensearch-project/OpenSearch-Dashboards/tree/main/packages/osd-i18n? Changing the i18n language through the URL, but there is no corresponding documentation, really confuses me.

https://github.com/user-attachments/assets/7f078b92-4e15-4e17-b703-f1e23085b646
What solution would you like?
A clear and concise description of what you want to happen.

No sure.

What alternatives have you considered?
A clear and concise description of any alternative solutions or features you've considered.

No sure.

Do you have any additional context?
Add any other context or screenshots about the feature request here.

See https://github.com/opensearch-project/OpenSearch-Dashboards/pull/7686 .
See https://github.com/opensearch-project/OpenSearch-Dashboards/issues/8075 .

[Catch All Triage - 1, 2, 3, 4]