  Keep up the great work.  ok, back to the issue..
Version: 1.31
Type: Bug
Severity: low-medium
Issue:
When a table's cell has too many words the entire table no longer stretches the width of the available space and shrinks to minimum size.
Reproduce:
The 3rd row, 2nd cell has just enough words in it to uncover the defect. When a few words are removed the entire table's width returns to normal.
| Branch | Description |
|--------|-------------|
| `master`  | Main branch ready for public release at anytime; always builds. |
| `develop` | All development work is pulled from here and is the only one allowed to merge up to `master`; always builds.|
| `feature-` | Some text here... |

This is a known issue with Pandoc.

Thank you for the heads up.


Pausing job

Description

Expose methods in Job_manager to pause and resume a job from its id or the entity.

_JobManager.pause(job.id)
_JobManager.resume(job.id)
_JobManager.pause(job)
_JobManager.resume(job)

Expose method in Taipy to pause and resume a job from its id or the entity.

tp.pause(job.id)
tp.resume(job.id)
tp.pause(job)
tp.resumue(job)

Expose method in Job to pause and resume itself.

job.pause()
job.resume()

Acceptance Criteria

[ ] Ensure new code is unit tested, and check code coverage is at least 90%
[ ] Propagate any change in the demos and run all of them to ensure there is no breaking change
[ ] Propagate any change in taipy-rest
[ ] Ensure any change is well documented

The added value is not clear to me.


error messsage not useful when image not found on RHEL

# buildah from busybox
unknown: Not Found

# buildah version
Version:       0.12
Go Version:    go1.8.3
Image Spec:    1.0.0
Runtime Spec:  1.0.0
Git Commit:    e073df1
Built:         Tue Feb 13 15:51:59 2018
OS/Arch:       linux/amd64

@gouyang can't reproduce it.

@ripcurld0 are you reproducing it on RHEL?

No, Fedora 27

The issue s reported on RHEL, it cannot see this error on Fedora.

It's not happened on centos.  it happened on RHEL because of it only search registry.access.redhat.com by default.
# cat /etc/containers/registries.conf
....
[registries.search]
registries = ['registry.access.redhat.com']

If add docker.io to the registries.search or remove registry.access.redhat.com, the image can be pulled and no error occurs.
The issue is complaining the error message unknown: Not Found not very clean to user, not why it happened.

@gouyang this info was very helpful.
I could successfully reproduce the problem.

@gouyang Did the default registries.conf on RHEL not have docker.io, or did you modify it?
Are you just reporting that you think we should have a better error message to tell the user the image is not found at any of the specified registries?

On Fedora I see this error when I have a bogus image
 buildah from foobar
no such image "foobar": image not known

If I remove all registies except for the redhat registry, I get the same error.
[registries.search]
#registries = ['docker.io', 'registry.fedoraproject.org', 'registry.access.redhat.com']
registries = ['registry.access.redhat.com']

The buildah is built from master.
# buildah version
Version:       0.12
Go Version:    go1.8.3
Image Spec:    1.0.0
Runtime Spec:  1.0.0
Git Commit:    d59af12
Built:         Wed Feb 14 06:45:04 2018
OS/Arch:       linux/amd64

Test on RHEL7.4, with registries = ['registry.access.redhat.com']  in /etc/containers/registries.conf:
# ./buildah from gouyang/foobar
unknown: Not Found
# ./buildah from busybox
unknown: Not Found
# ./buildah from foobar
unknown: Not Found
# ./buildah from gouyang/httpd
unknown: Not Found

with registries = ['docker.io'] or registries = []:
# ./buildah from gouyang/httpd
Getting image source signatures
Copying blob sha256:e7bb522d92ff6d4e5b2087409b0fc783c2e3b06acf87bee739ee47d90bf02e96
^C
# ./buildah from busybox
Getting image source signatures
Copying blob sha256:57310166fe88e0dc63a80ca5c219283a932db0f3969712e2f8a86ada143bf566
^C
# ./buildah from foobar
denied: requested access to the resource is denied
unauthorized: authentication required
# ./buildah from gouyang/foobar
manifest unknown: manifest unknown

Please also note there is an different message for buildah from gouyang/foobar.

@nalind @mitrmac any ideas?

@runcom PTAL

@mtrmac PTAL

@mtrmac Any idea why we would see hugely different messages between RHEL and Fedora?

@mtrmac Any idea why we would see hugely different messages between RHEL and Fedora?

If this is referring to
 buildah from foobar
no such image "foobar": image not known

I believe that comes from something like https://github.com/projectatomic/buildah/blob/815cedfc71fff7460b683b5f9e8fe053b31a31ef/new.go#L223 .

@mtrmac yes, that's right.
We are checking if the errors.Cause of the error is type of errcode and assumed the errorcode will contain a good error message (like in the unauthorized case).
Are you saying we shouldn't relay on that?

I am overall a bit confused by the conversation in this bug. It shows quite a few different environments, cases and outputs, and it’s not overall clear to me what is the specific issue being solved. Is it that the human-readable error messages, i.e. the .Error() values, from c/image lack context? Is it something about how buildah is adding/not adding its own context?
Is it unknown/still changing just because no-one has looked into the code in sufficient detail yet? Or are we having distinct parallel conversations here?

We are checking if the errors.Cause of the error is type of errcode and assumed the errorcode will contain a good error message (like in the unauthorized case).
Are you saying we shouldn't relay on that?

I’m not sure that’s a good use of errors.Cause. The idea of errors.Wrapf is to add context to the underlying “cause”; so, errors.Cause necessarily removes context. I’d lean towards using errors.Cause for machine decisions (“what kind of failure is this”), where the added human-readable and non-API strings are unusable, but preserving full text (not just the “cause”/”errcode” for human-readable error messages (unless the machine specifically decides using based no the Cause that the context is irrelevant). Or, in some cases, go all in and
if errors.Cause(err) == someExpectedValue {
  report(text_entirely_independent_of_err).
}

I am overall a bit confused by the conversation in this bug.

Also, if every reported “bad” case of missing context could be accompanied with the --debug output from the same situation, that would probably be helpful in quickly tracking down the error flow.

I’m not sure that’s a good use of errors.Cause

To be more specific, this issue will probably be the last push for me to add at least the repo name as a context to the unknown: not found messages. If buildah then uses errors.Cause to strip it away, it can’t benefit. (OTOH, sure, the chain of wrapped context annotations can get horribly long, and if buildah know exactly what is going on, it can be quite justified in stripping it away.)

I agree with you @mtrmac
I guess I shouldn't have stripped the error drastically.

Closing via #485. Thanks all for the investigation on this.

To be more specific, this issue will probably be the last push for me to add at least the repo name as a context to the unknown: not found messages.

See https://github.com/containers/image/pull/420 .


slugify parent lockname in generator

Signed-off-by: Felix Kaechele<EMAIL_ADDRESS>Proposed change
Slugify the parent lockname as well, as the package generation creates references to the parent lock which had it's name slugified when it was set up initially.
Type of change

[ ] Dependency upgrade
[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (which adds functionality)
[ ] Breaking change (fix/feature causing existing functionality to break)
[ ] Code quality improvements to existing code or addition of tests

Additional information

This PR fixes or closes issue: fixes #231

Thanks!


June 2019 update breaks CQUI

Describe the bug
After getting the June update, I opened a save game I had started with CQUI. The mini map is broken and shows the whole world as uncharted territory. Cannot use the buttons on minimap (no menu pops up when clicking). Attempted to disable CQUI, reopen save game, and minimap itself still broken. Re-enabled CQUI, reopened save game, still broken.
I am still able to open the CQUI settings menu, however.
To Reproduce

Get June 2019 Civ VI update, with gathering storm
Open save game started with CQUI before update
Mini Map is blank, everything is uncharted territory. Cannot click on mini map to go to that part of the world. and none of the buttons above it work.

Expected behavior
To work as usual after the June 2019 update
Screenshots
If applicable, add screenshots to help explain your problem.
General informations

OS: Win 10
Version Current, isntalled 10 days ago
Expansion Gathering Storm
Other mods Radial Measuring tool (tried disabling it)

Save game and logs
Attached two save games - from before and after the update (DidoEarly and DidoMedieval
Attached all logs
before_and_after.zip
AllLogs.zip

same failed minimap behavior if you started a new game

I wish Firaxis would allow us to use the latest version before the patch when using the Steam "Legacy" option... right now if reverts like 3/4 versions if you don't want the patch...

City names mixup when negotiating peace. New game, CQUI only mod activated.
W10 x64
v <IP_ADDRESS>8(426563)
https://i.imgur.com/cZQe5ru.jpg

Odd, because my mini-map still works just fine.
W10 x64
<IP_ADDRESS>8(426563)
I only have the expansion Rise&Fall though?

Minimap issues are an issue with More Lenses. Removing the Integration resolves the issue.
For more context sdgabai on steam has a good traceback
I renamed LateInitialize and the map showed up just fine, but it seems there are still some other issues.

So the patch didn't really cause any major issue with CQUI? Totally playable if it's the only MOD?

Hi @scrubdaddytm - forgive my ignorance on the technical workings of civ 6 mods, but if it is not a bother, could you give some steps on how to do that. I am understanding you mean not to remove CQUI, but instead disable its integration with More Lenses?

@scrubdaddytm just saw your pull request - thanks for stepping up on a fix!

I fixed this issue by deleting the ML folder in \Mods\CQUI\integrations, and installing "More Lenses 4.2" (link provided by scrubdaddytm a few comments above) directly into the \Mods folder
Now my saved games seems to be working like pre-june2019 patch
Hope this helps!

just trying to find that folder lol

After my excitement was done I noticed that this fix makes the CQUI settings button to disappear. Worth mentioning!

Until #405 is integrated, feel free to download the files from Integrations/ML in this branch on my fork: https://github.com/scrubdaddytm/CQUI_Community-Edition/tree/update-more-lenses It will fix the minimap and also provide the CQUI settings button :)

thanks @scrubdaddytm ! Sadly, I tried with no success, but then again that Mods folder is empty for me - I have steam and wound up finding the appropriate folder hidden under a folder with a numeric name in the mods part of the program files/steam folder.
I will try to just place it directly on the My Documents/Civ/Mods even though it is currently empty and see what happens when I get a chance

@ffcruz85
Try to log off from steam and follow @scrubdaddytm instructions (dowload appropriate files here and ...)

 Here are my steps:

Disabled Mod in civ menu
Logged out of steam
copied files from Integrations/ML from @scrubdaddytm branch of CQUI
pasted them into this folder, which is the only one where I found the equivalent files:
C:\Program Files (x86)\Steam\steamapps\workshop\content\289070\1125169247\Integrations\ML
The "My Documents/Civ/Mods" folder is empty
Logged into steam, started civ
Re-enabled CQUI, but unsubscribed just in case it might be overwritten by main branch
still the same.

Is there any updated status on this? Any word from Azurency? Have not heard from him in a while.
Has anyone been successful using the branch mentioned above? I only see one reply saying it didn't work at all and another saying they had some weird issue with missing filters.
Can we get a merge and a Steam update if the branch is indeed correct?

You can use scrubdaddytm's solution and it works well enough to forget on original UI.
And that small flaw with (sometimes) missing lenses isn't big issue at all... ;)

@ffcruz85 I had the same problem you did. Try unsubscribing from the mod on Steam, ensuring the mod is deleted in the appropriate Steam mod storage folder, and then copying @scrubdaddytm 's forked version into your mod folder.
Nevertheless, the Lua.log log file will tell you what is going on. If there are still problems, that's where your answer for "why" will likely be.

Using only copied files from Integrations/ML from @scrubdaddytm branch of CQUI didn't work for me either, but using the whole branch solved the issue.
I think I have found a bug within this branch, how can I report it? Not sure if it is present in the original CQUI.
When I have Secret or Top Secret level visibility on a foreign civilization I should be able to click on one of that civilization's city banners and view this City Details for that foreign city.
However:

With Secret level, the banner still says "Increase Access Level to view this city's details",
With Top Secret level, the banner says "Click to view the city details", but nothing happens when clicking on the small round "button" at the left of the banner.

I play with R&F and GS. The mod is probably causing the second behaviour, but probably not the first one (base game bug?).

Any word on when the mini map is going to be fixed? I have never used a mod until CQUI, but with Gathering Storm and all expansions available from Steam the mini map does not work at all. There are dozens of comments on the steam community page asking if it's going to be fixed. It's now 3 months after the June update. If this is not going to be fixed because it's technically playable, just say so.

Using scrubdaddy's version the minimap is working fine but there is a bit of a lense issue. Sometimes the resource lense isn't there and sometimes the builder lense stops working.

Is there anything I can do to help, I've never done any sort of Civ modding, but I am passingly familiar with lua.  If there is a getting started guide for debugging, I can poke around at it.


Logic flaw in Min and Max temperature compares

I really really appreciate your efforts with this plugin and if there is anything I can do to help make it better, or support you I will do it please reach out.  I did submit this issue previously, but it was closed.  I believe there is a logic flaw, please take a second look.
I am not a developer, but in testing multiples of values there appears to be a logic flaw in the Min vs Max temperature settings.  I should state that I have my temperature set to 125 F within the Control-R app prior to enabling this plugin and never change it.  Also, the Celsius setting appears to be ignored, or at least handled incorrectly.  It's somewhat difficult to discern as the output uses Celsius values even though the code is using Fahrenheit.  Notice this log output when using 125.1 min and 125.1 max:
[3/8/2022, 4:00:39 PM] [homebridge-rinnai-controlr] This plugin generated a warning from the characteristic 'Current Temperature': characteristic was supplied illegal value: number 51.666666666666664 exceeded minimum of 51.722222222222214.
When using 120.1 min and 120.2 max:
[3/8/2022, 4:07:35 PM] [homebridge-rinnai-controlr] This plugin generated a warning from the characteristic 'Current Temperature': characteristic was supplied illegal value: number 51.666666666666664 exceeded maximum of 49.
Now notice when I change from F to C, and use 51.6 min and 51.6 max:
[3/8/2022, 4:04:43 PM] [homebridge-rinnai-controlr] This plugin generated a warning from the characteristic 'Target Temperature': characteristic was supplied illegal value: number 125 exceeded maximum of 51.6.
Plugin Config:
This is my desired config, I don't want to change temperature with this plugin, I only want the recirculation.
        {
            "username": "xxx",
            "password": "XXX",
            "recirculationDuration": 15,
            "temperatureUnits": "F",
            "minimumTemperature": 125,
            "maximumTemperature": 125,
            "platform": "RinnaiControlR"
        }

Environment:

Plugin Version: v1.0.14
Homebridge Version: v1.4.0
Node.js Version: v16.13.2
NPM Version: v8.1.2
Operating System: Docker running Alpine Linux (3.12.9)

I am getting the same error on my heater too. Would appreciate a fix if possible!

More info: I see this message spammed constantly:
[homebridge-rinnai-controlr] This plugin generated a warning from the characteristic 'Current Temperature': characteristic was supplied illegal value: number 36.666666666666664 exceeded minimum of 48.888888888888886. See https://homebridge.io/w/JtMGR for more info.

I'm not sure why this resolved this particular issue to me, but setting the heater's "max temp" mode from 120F to 140F (mode 1b instead of 1A on my 199en) made these messages go away.

Min/max should not be configured with the same values. Closing this out as invalid config. If you'd like to disable temperature control, please raise a new feature request, but you could also just not change the temperature in HomeKit. The feature is there if you need it, but you don't have to use it...just like a physical push button controller.


scroll-button

Issue solved
Closes: #77
Changes Proposed
Added scroll-button

@yash982000 please review this.


gy91 mpu9255 and new ak8963

Using a gy-91 which contains a mpu9255 and a different model AK8963
Changed in AK8963.init
if 0x48 != self.whoami:
to
if self.whoami not in (0x48,0x5F):
in AK8963.magnetic (was getting the same y an z values, appears you can no longer read six bytes at a time but must read 2 bytes for each of hxl, hyl, hzl)
So changed
xyz = list(self._register_three_shorts(_HXL)
to
x = self._register_short(_HXL)
y = self._register_short(_HYL)
z = self._register_short(_HZL)
xyz=[x,y,z]

and in mpu6500.init
if 0x71 != self.whoami
to
if self.whoami not in (0x71,0x73)
Still working away trying to get this working ...

You mean this? Looks quite interesting board. Let me know how it goes.

Cant seem to get this to work properly .. when I plot all the graphs for calibration the data doesnt look correct. Heres the MPU-9255 register map
Sample output and the code follows
6.53933,-1.681542,-6.165654
-42.7859,-6.165654,-12.14447
-42.41222,-12.14447,-12.14447
-6.165654,-12.14447,-12.14447
-6.165654,-24.84945,-12.14447
-3.176246,-36.05973,-12.14447
-1.681542,-39.04914,-12.14447
6.913006,-41.2912,-15.13388
-37.55444,-39.04914,-12.14447
-30.82827,-24.47578,-12.14447
-36.43341,-6.165654,-12.14447
-37.74128,-5.231464,-15.13388
-42.41222,-46.89634,-12.14447
-42.7859,-46.89634,-12.14447
-0.560514,-42.22539,-12.14447
0.0,-36.43341,-12.33131
-45.77531,-30.08092,-12.14447
3.923598,17.74961,-12.14447
-42.7859,-25.59681,-6.165654
-42.7859,-27.09151,-6.165654
-42.22539,-27.09151,-6.165654
-43.90693,-30.26776,-6.165654
-43.53325,-30.08092,-6.165654
-1.681542,-30.08092,-6.165654
-3.176246,-30.82827,-6.165654
-3.176246,-30.08092,-6.165654
-3.176246,-34.9387,-6.165654
-1.681542,-30.08092,-6.165654
-3.176246,-30.08092,-6.165654
-3.176246,-36.43341,-6.165654
-6.165654,-37.55444,-6.165654
-6.165654,-40.73068,-6.165654
-6.165654,-42.7859,-6.165654
-6.165654,-0.186838,-12.14447
-6.165654,-1.681542,-6.165654
-6.165654,-6.165654,-6.165654
-12.14447,-6.165654,-6.165654
-12.14447,-12.89182,-6.165654
-12.14447,-12.14447,-6.165654
-6.165654,-12.14447,-12.14447
-6.165654,-12.14447,-12.14447
-6.165654,-24.1021,-12.14447
-3.176246,-30.08092,-12.14447
-3.176246,-30.08092,-12.14447
-0.186838,-30.08092,-12.14447
-1.681542,-30.08092,-12.14447
-45.40164,-39.04914,-12.14447
-43.72009,-39.04914,-12.14447
-37.55444,-43.53325,-12.14447
-37.55444,-45.96215,-6.165654
-37.55444,-43.90693,-12.14447
-30.08092,-42.22539,-12.14447
-30.08092,-0.560514,-12.14447
-30.82827,-45.40164,-12.14447
-27.09151,-0.93419,-6.165654
-12.14447,-0.93419,-6.165654
-12.14447,-0.93419,-6.165654
-12.14447,-0.93419,-12.14447
-12.14447,-1.681542,-12.14447
-12.14447,-0.560514,-12.14447
-6.165654,-46.52266,-12.14447
-3.176246,-46.52266,-6.165654
-1.681542,-42.7859,-12.14447
50.81994,-39.79649,-12.14447
53.062,-39.04914,-12.14447
55.11721,-33.81768,-202.7192
-30.08092,-30.08092,-250.5497
-34.9387,-24.47578,-11.02344
62.96441,-12.14447,-6.165654
-30.82827,-12.14447,-6.165654
-30.08092,-12.14447,-12.14447
-30.08092,-25.59681,-6.165654
-39.42282,-36.05973,-6.165654
48.39104,-41.2912,-6.165654
-13.63917,-1.681542,-6.165654
-12.14447,-0.93419,-6.165654
15.32072,-47.08318,-12.14447
7.47352,-44.84112,-12.14447

# Copyright (c) 2018-2019 Mika Tuupola
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of  this software and associated documentation files (the "Software"), to
# deal in  the Software without restriction, including without limitation the
# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
# sell copied of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

# https://github.com/tuupola/micropython-mpu9250
# https://www.akm.com/akm/en/file/datasheet/AK8963C.pdf

"""
MicroPython I2C driver for AK8963 magnetometer
"""

__version__ = "0.2.1"

# pylint: disable=import-error
import ustruct
import utime
from machine import I2C, Pin
from micropython import const
# pylint: enable=import-error

_WIA = const(0x00)
_INFO = const(0x01)
_ST1 = const(0x02)
_HXL = const(0x03)
_HXH = const(0x04)
_HYL = const(0x05)
_HYH = const(0x06)
_HZL = const(0x07)
_HZH = const(0x08)
_ST2 = const(0x09)
_CNTL1 = const(0x0a)
_ASAX = const(0x10)
_ASAY = const(0x11)
_ASAZ = const(0x12)

_MODE_POWER_DOWN = 0b00000000
MODE_SINGLE_MEASURE = 0b00000001
MODE_CONTINOUS_MEASURE_1 = 0b00000010 # 8Hz
MODE_CONTINOUS_MEASURE_2 = 0b00000110 # 100Hz
MODE_EXTERNAL_TRIGGER_MEASURE = 0b00000100
_MODE_SELF_TEST = 0b00001000
_MODE_FUSE_ROM_ACCESS = 0b00001111

OUTPUT_14_BIT = 0b00000000
OUTPUT_16_BIT = 0b00010000

_SO_14BIT = 0.6 # μT per digit when 14bit mode
_SO_16BIT = 4912/32760 # μT per digit when 16bit mode

DRDY = 0b00000001
DOR = 0b00000010
HOFL = 0b00001000

class AK8963:
    """Class which provides interface to AK8963 magnetometer."""
    def __init__(
        self, i2c, address=0x0c,
        mode=MODE_CONTINOUS_MEASURE_1, output=OUTPUT_16_BIT,
        offset=(0, 0, 0), scale=(1, 1, 1)
    ):
        self.i2c = i2c
        self.address = address
        self._offset = offset
        self._scale = scale

	self._raw_buf_x = bytearray(2)
	self._raw_buf_y = bytearray(2)
	self._raw_buf_z = bytearray(2)
	self._raw_xyz = [0.0,0.0,0.0]

        #if 0x48 != self.whoami:
	if self.whoami not in (0x48,0x5F):
            raise RuntimeError("AK8963 not found in I2C bus.")

        # Sensitivity adjustement values
        self._register_char(_CNTL1, _MODE_FUSE_ROM_ACCESS)
        asax = self._register_char(_ASAX)
        asay = self._register_char(_ASAY)
        asaz = self._register_char(_ASAZ)
        self._register_char(_CNTL1, _MODE_POWER_DOWN)

        # Should wait atleast 100us before next mode
        self._adjustement = (
            (0.5 * (asax - 128)) / 128 + 1,
            (0.5 * (asay - 128)) / 128 + 1,
            (0.5 * (asaz - 128)) / 128 + 1
        )
        utime.sleep_us(100)

        # Power on
        self._register_char(_CNTL1, (mode | output))

        if output is OUTPUT_16_BIT:
            self._so = _SO_16BIT
        else:
            self._so = _SO_14BIT

    @property
    def magnetic(self):
        """
        X, Y, Z axis micro-Tesla (uT) as floats.
        """

	if self.data_is_ready:
	    self.i2c.readfrom_mem_into(self.address, _HXL, self._raw_buf_x)
	    self.i2c.readfrom_mem_into(self.address, _HYL, self._raw_buf_y)
	    self.i2c.readfrom_mem_into(self.address, _HZL, self._raw_buf_z)

	    if not self.magnetic_sensor_overflow:
		self._raw_xyz[0] = self._read_int(self._raw_buf_x)
		self._raw_xyz[1] = self._read_int(self._raw_buf_y)
		self._raw_xyz[2] = self._read_int(self._raw_buf_z)

	xyz=self._raw_xyz

        self._register_char(_ST2) # Enable updating readings again

        # Apply factory axial sensitivy adjustements
        xyz[0] *= self._adjustement[0]
        xyz[1] *= self._adjustement[1]
        xyz[2] *= self._adjustement[2]

        # Apply output scale determined in constructor
        so = self._so
        xyz[0] *= so
        xyz[1] *= so
        xyz[2] *= so

        # Apply hard iron ie. offset bias from calibration
        xyz[0] -= self._offset[0]
        xyz[1] -= self._offset[1]
        xyz[2] -= self._offset[2]

        # Apply soft iron ie. scale bias from calibration
        xyz[0] *= self._scale[0]
        xyz[1] *= self._scale[1]
        xyz[2] *= self._scale[2]

        return tuple(xyz)

    @property
    def adjustement(self):
        return self._adjustement

    @property
    def whoami(self):
        """ Value of the whoami register. """
        return self._register_char(_WIA)

    @property
    def data_is_ready(self):
        return bool(self._register_char(_ST1) & DRDY)

    @property
    def data_overrun(self):
        return bool(self._register_char(_ST1) & DOR)

    @property
    def magnetic_sensor_overflow(self):
	return bool(self._register_char(_ST2) & HOFL)

    def calibrate(self, count=256, delay=200):
        self._offset = (0, 0, 0)
        self._scale = (1, 1, 1)

        reading = self.magnetic
        minx = maxx = reading[0]
        miny = maxy = reading[1]
        minz = maxz = reading[2]

        while count:
            utime.sleep_ms(delay)
            reading = self.magnetic
            minx = min(minx, reading[0])
            maxx = max(maxx, reading[0])
            miny = min(miny, reading[1])
            maxy = max(maxy, reading[1])
            minz = min(minz, reading[2])
            maxz = max(maxz, reading[2])
            count -= 1

        # Hard iron correction
        offset_x = (maxx + minx) / 2
        offset_y = (maxy + miny) / 2
        offset_z = (maxz + minz) / 2

        self._offset = (offset_x, offset_y, offset_z)

        # Soft iron correction
        avg_delta_x = (maxx - minx) / 2
        avg_delta_y = (maxy - miny) / 2
        avg_delta_z = (maxz - minz) / 2

        avg_delta = (avg_delta_x + avg_delta_y + avg_delta_z) / 3

        scale_x = avg_delta / avg_delta_x
        scale_y = avg_delta / avg_delta_y
        scale_z = avg_delta / avg_delta_z

        self._scale = (scale_x, scale_y, scale_z)

        return self._offset, self._scale

    def _read_int(self, buf=bytearray(2)):
	value = (buf[1] <<8) + buf[0]
	if (value >= 0x8000):
	    return -((65535 - value) + 1)
	else:
	    return value

    def _register_short(self, register, value=None, buf=bytearray(2)):
        if value is None:
            self.i2c.readfrom_mem_into(self.address, register, buf)
            return ustruct.unpack("<h", buf)[0]

        ustruct.pack_into("<h", buf, 0, value)
        return self.i2c.writeto_mem(self.address, register, buf)

    def _register_three_shorts(self, register, buf=bytearray(6)):
        self.i2c.readfrom_mem_into(self.address, register, buf)
        return ustruct.unpack("<hhh", buf)

    def _register_char(self, register, value=None, buf=bytearray(1)):
        if value is None:
            self.i2c.readfrom_mem_into(self.address, register, buf)
            return buf[0]

        ustruct.pack_into("<b", buf, 0, value)
        return self.i2c.writeto_mem(self.address, register, buf)

    def __enter__(self):
        return self

    def __exit__(self, exception_type, exception_value, traceback):
        pass

Getting better data using  i2c = I2C(scl=Pin(21), sda=Pin(22), freq=100000). There appears to be an issue with the default freq on an ESP32 communicating with the GY-91. Might be a timing issues?

@hortovanyi Did you get this to work?
I have been looking into using th GY-91 with a LoPy4.
The LoPy is running µ-Python and getting the IMU to work has but me in some deep waters i have to admit, so I was looking for something to a library or driver for the GY-91

seems like a common issue.. https://hackaday.io/project/28491-quick-draw-motion-tracking/log/71828-a-spark-a-doh-a-surprise-and-data-sheets

working fine for me using MicroPython v1.13-186-g5a7027915 on 2020-11-25; ESP module with ESP8266:

code changed as @hortovanyi posted here
freq=100000.

I'm getting similar results, but it doesn't seem right (only the temp seems to be correct)
acce  (-0.02154782, -0.02154782, 9.289502)
gyro  (-0.05249311, 0.01985146, -0.02264931)
magn  (-12.14364, -9.666079, 7.323508)
temp  27.6463

acce  (-0.01197101, -0.06943185, 9.315839)
gyro  (-0.05342574, 0.02251608, -0.01891884)
magn  (-11.57441, -9.490331, 6.794097)
temp  27.79007

acce  (-0.01197101, -0.05027823, 9.344569)
gyro  (-0.05449158, 0.01998469, -0.02118377)
magn  (-11.76415, -10.72056, 5.911747)
temp  27.6463

acce  (-0.01197101, -0.02394202, 9.358934)
gyro  (-0.05315928, 0.02451455, -0.02251608)
magn  (-11.19492, -9.490331, 6.794097)
temp  27.79007

Any luck with this?
I'm getting similar results, but it doesn't seem right (only the temp seems to be correct)
acce  (-0.02154782, -0.02154782, 9.289502)
gyro  (-0.05249311, 0.01985146, -0.02264931)
magn  (-12.14364, -9.666079, 7.323508)
temp  27.6463

acce  (-0.01197101, -0.06943185, 9.315839)
gyro  (-0.05342574, 0.02251608, -0.01891884)
magn  (-11.57441, -9.490331, 6.794097)
temp  27.79007

acce  (-0.01197101, -0.05027823, 9.344569)
gyro  (-0.05449158, 0.01998469, -0.02118377)
magn  (-11.76415, -10.72056, 5.911747)
temp  27.6463

acce  (-0.01197101, -0.02394202, 9.358934)
gyro  (-0.05315928, 0.02451455, -0.02251608)
magn  (-11.19492, -9.490331, 6.794097)
temp  27.79007

Cant seem to get this to work properly .. when I plot all the graphs for calibration the data doesnt look correct. Heres the MPU-9255 register map
Sample output and the code follows
6.53933,-1.681542,-6.165654
-42.7859,-6.165654,-12.14447
-42.41222,-12.14447,-12.14447
-6.165654,-12.14447,-12.14447
-6.165654,-24.84945,-12.14447
-3.176246,-36.05973,-12.14447
-1.681542,-39.04914,-12.14447
6.913006,-41.2912,-15.13388
-37.55444,-39.04914,-12.14447
-30.82827,-24.47578,-12.14447
-36.43341,-6.165654,-12.14447
-37.74128,-5.231464,-15.13388
-42.41222,-46.89634,-12.14447
-42.7859,-46.89634,-12.14447
-0.560514,-42.22539,-12.14447
0.0,-36.43341,-12.33131
-45.77531,-30.08092,-12.14447
3.923598,17.74961,-12.14447
-42.7859,-25.59681,-6.165654
-42.7859,-27.09151,-6.165654
-42.22539,-27.09151,-6.165654
-43.90693,-30.26776,-6.165654
-43.53325,-30.08092,-6.165654
-1.681542,-30.08092,-6.165654
-3.176246,-30.82827,-6.165654
-3.176246,-30.08092,-6.165654
-3.176246,-34.9387,-6.165654
-1.681542,-30.08092,-6.165654
-3.176246,-30.08092,-6.165654
-3.176246,-36.43341,-6.165654
-6.165654,-37.55444,-6.165654
-6.165654,-40.73068,-6.165654
-6.165654,-42.7859,-6.165654
-6.165654,-0.186838,-12.14447
-6.165654,-1.681542,-6.165654
-6.165654,-6.165654,-6.165654
-12.14447,-6.165654,-6.165654
-12.14447,-12.89182,-6.165654
-12.14447,-12.14447,-6.165654
-6.165654,-12.14447,-12.14447
-6.165654,-12.14447,-12.14447
-6.165654,-24.1021,-12.14447
-3.176246,-30.08092,-12.14447
-3.176246,-30.08092,-12.14447
-0.186838,-30.08092,-12.14447
-1.681542,-30.08092,-12.14447
-45.40164,-39.04914,-12.14447
-43.72009,-39.04914,-12.14447
-37.55444,-43.53325,-12.14447
-37.55444,-45.96215,-6.165654
-37.55444,-43.90693,-12.14447
-30.08092,-42.22539,-12.14447
-30.08092,-0.560514,-12.14447
-30.82827,-45.40164,-12.14447
-27.09151,-0.93419,-6.165654
-12.14447,-0.93419,-6.165654
-12.14447,-0.93419,-6.165654
-12.14447,-0.93419,-12.14447
-12.14447,-1.681542,-12.14447
-12.14447,-0.560514,-12.14447
-6.165654,-46.52266,-12.14447
-3.176246,-46.52266,-6.165654
-1.681542,-42.7859,-12.14447
50.81994,-39.79649,-12.14447
53.062,-39.04914,-12.14447
55.11721,-33.81768,-202.7192
-30.08092,-30.08092,-250.5497
-34.9387,-24.47578,-11.02344
62.96441,-12.14447,-6.165654
-30.82827,-12.14447,-6.165654
-30.08092,-12.14447,-12.14447
-30.08092,-25.59681,-6.165654
-39.42282,-36.05973,-6.165654
48.39104,-41.2912,-6.165654
-13.63917,-1.681542,-6.165654
-12.14447,-0.93419,-6.165654
15.32072,-47.08318,-12.14447
7.47352,-44.84112,-12.14447

# Copyright (c) 2018-2019 Mika Tuupola
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of  this software and associated documentation files (the "Software"), to
# deal in  the Software without restriction, including without limitation the
# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
# sell copied of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

# https://github.com/tuupola/micropython-mpu9250
# https://www.akm.com/akm/en/file/datasheet/AK8963C.pdf

"""
MicroPython I2C driver for AK8963 magnetometer
"""

__version__ = "0.2.1"

# pylint: disable=import-error
import ustruct
import utime
from machine import I2C, Pin
from micropython import const
# pylint: enable=import-error

_WIA = const(0x00)
_INFO = const(0x01)
_ST1 = const(0x02)
_HXL = const(0x03)
_HXH = const(0x04)
_HYL = const(0x05)
_HYH = const(0x06)
_HZL = const(0x07)
_HZH = const(0x08)
_ST2 = const(0x09)
_CNTL1 = const(0x0a)
_ASAX = const(0x10)
_ASAY = const(0x11)
_ASAZ = const(0x12)

_MODE_POWER_DOWN = 0b00000000
MODE_SINGLE_MEASURE = 0b00000001
MODE_CONTINOUS_MEASURE_1 = 0b00000010 # 8Hz
MODE_CONTINOUS_MEASURE_2 = 0b00000110 # 100Hz
MODE_EXTERNAL_TRIGGER_MEASURE = 0b00000100
_MODE_SELF_TEST = 0b00001000
_MODE_FUSE_ROM_ACCESS = 0b00001111

OUTPUT_14_BIT = 0b00000000
OUTPUT_16_BIT = 0b00010000

_SO_14BIT = 0.6 # μT per digit when 14bit mode
_SO_16BIT = 4912/32760 # μT per digit when 16bit mode

DRDY = 0b00000001
DOR = 0b00000010
HOFL = 0b00001000

class AK8963:
    """Class which provides interface to AK8963 magnetometer."""
    def __init__(
        self, i2c, address=0x0c,
        mode=MODE_CONTINOUS_MEASURE_2, output=OUTPUT_16_BIT,
        offset=(0, 0, 0), scale=(1, 1, 1)
    ):
        self.i2c = i2c
        self.address = address
        self._offset = offset
        self._scale = scale

	self._raw_buf_x = bytearray(2)
	self._raw_buf_y = bytearray(2)
	self._raw_buf_z = bytearray(2)
	self._raw_xyz = [0.0,0.0,0.0]

        #if 0x48 != self.whoami:
	whoami = self.whoami
	if whoami not in (0x48,0x5F,0x7F):
            raise RuntimeError("AK8963 not found in I2C bus - found 0x%02X." % whoami)

        # Sensitivity adjustement values
        self._register_char(_CNTL1, _MODE_FUSE_ROM_ACCESS)
        asax = self._register_char(_ASAX)
        asay = self._register_char(_ASAY)
        asaz = self._register_char(_ASAZ)
        self._register_char(_CNTL1, _MODE_POWER_DOWN)

        # Should wait atleast 100us before next mode
        self._adjustement = (
            (0.5 * (asax - 128)) / 128 + 1,
            (0.5 * (asay - 128)) / 128 + 1,
            (0.5 * (asaz - 128)) / 128 + 1
        )
        utime.sleep_us(100)

        # Power on
        self._register_char(_CNTL1, (mode | output))

        if output is OUTPUT_16_BIT:
            self._so = _SO_16BIT
        else:
            self._so = _SO_14BIT

    @property
    def magnetic(self):
        """
        X, Y, Z axis micro-Tesla (uT) as floats.
        """

	while not self.data_is_ready:
	    utime.sleep_ms(10)
	    pass

	self.i2c.readfrom_mem_into(self.address, _HXL, self._raw_buf_x)
	self.i2c.readfrom_mem_into(self.address, _HYL, self._raw_buf_y)
	self.i2c.readfrom_mem_into(self.address, _HZL, self._raw_buf_z)

	if not self.magnetic_sensor_overflow:
	    self._raw_xyz[0] = self._read_int(self._raw_buf_x)
	    self._raw_xyz[1] = self._read_int(self._raw_buf_y)
	    self._raw_xyz[2] = self._read_int(self._raw_buf_z)

	xyz=self._raw_xyz

        #self._register_char(_ST2) # Enable updating readings again

        # Apply factory axial sensitivy adjustements
        xyz[0] *= self._adjustement[0]
        xyz[1] *= self._adjustement[1]
        xyz[2] *= self._adjustement[2]

        # Apply output scale determined in constructor
        so = self._so
        xyz[0] *= so
        xyz[1] *= so
        xyz[2] *= so

        # Apply hard iron ie. offset bias from calibration
        xyz[0] -= self._offset[0]
        xyz[1] -= self._offset[1]
        xyz[2] -= self._offset[2]

        # Apply soft iron ie. scale bias from calibration
        xyz[0] *= self._scale[0]
        xyz[1] *= self._scale[1]
        xyz[2] *= self._scale[2]

        return tuple(xyz)

    @property
    def adjustement(self):
        return self._adjustement

    @property
    def whoami(self):
        """ Value of the whoami register. """
        return self._register_char(_WIA)

    @property
    def data_is_ready(self):
        return bool(self._register_char(_ST1) & DRDY)

    @property
    def data_overrun(self):
        return bool(self._register_char(_ST1) & DOR)

    @property
    def magnetic_sensor_overflow(self):
	return bool(self._register_char(_ST2) & HOFL)

    def calibrate(self, count=256, delay=200):
        self._offset = (0, 0, 0)
        self._scale = (1, 1, 1)

        reading = self.magnetic
        minx = maxx = reading[0]
        miny = maxy = reading[1]
        minz = maxz = reading[2]

        while count:
            utime.sleep_ms(delay)
            reading = self.magnetic
            minx = min(minx, reading[0])
            maxx = max(maxx, reading[0])
            miny = min(miny, reading[1])
            maxy = max(maxy, reading[1])
            minz = min(minz, reading[2])
            maxz = max(maxz, reading[2])
            count -= 1

        # Hard iron correction
        offset_x = (maxx + minx) / 2
        offset_y = (maxy + miny) / 2
        offset_z = (maxz + minz) / 2

        self._offset = (offset_x, offset_y, offset_z)

        # Soft iron correction
        avg_delta_x = (maxx - minx) / 2
        avg_delta_y = (maxy - miny) / 2
        avg_delta_z = (maxz - minz) / 2

        avg_delta = (avg_delta_x + avg_delta_y + avg_delta_z) / 3

        scale_x = avg_delta / avg_delta_x
        scale_y = avg_delta / avg_delta_y
        scale_z = avg_delta / avg_delta_z

        self._scale = (scale_x, scale_y, scale_z)

        return self._offset, self._scale

    def _read_int(self, buf=bytearray(2)):
	value = (buf[1] <<8) + buf[0]
	if (value >= 0x8000):
	    return -((65535 - value) + 1)
	else:
	    return value

    def _register_short(self, register, value=None, buf=bytearray(2)):
        if value is None:
            self.i2c.readfrom_mem_into(self.address, register, buf)
            return ustruct.unpack("<h", buf)[0]

        ustruct.pack_into("<h", buf, 0, value)
        return self.i2c.writeto_mem(self.address, register, buf)

    def _register_three_shorts(self, register, buf=bytearray(6)):
        self.i2c.readfrom_mem_into(self.address, register, buf)
        return ustruct.unpack("<hhh", buf)

    def _register_char(self, register, value=None, buf=bytearray(1)):
        if value is None:
            self.i2c.readfrom_mem_into(self.address, register, buf)
            return buf[0]

        ustruct.pack_into("<b", buf, 0, value)
        return self.i2c.writeto_mem(self.address, register, buf)

    def __enter__(self):
        return self

    def __exit__(self, exception_type, exception_value, traceback):
        pass

Where did you change?I can use this code!


When do I need apollo kernel?

I have a few scenarios which I'm not sure requires apollo kernel or not. Can you please comment on them?

Playing a bag file and running various modules on only my laptop?
Make a release docker image of my modified version of apollo and run it on the car. I understand that the car would need the apollo kernel but do I also have to have it on my laptop for it to finally work on the car?

From what I understand, if you want to play bag file and test some modules only on your laptop, it is not necessary to install Apollo kernel. However, if you want to run Apollo with an actual car, it is recommended to install kernel. Also I believe it would be better if you can test code with kernel before run it on the car.

Closing this issue as there has no communication in a while. Feel free to open it if you still have additional questions.


gulp server: ReferenceError: primordials is not defined error

I am on macOS. I have followed the installation guide but when I run  gulp server I get ReferenceError: primordials is not defined error. How can I fix this error?
$ git clone https://github.com/mozilla/pdf.js.git
$ cd pdf.js
$ npm install -g gulp-cli
$ npm install --legacy-peer-deps
$ gulp --version
[14:13:56] CLI version 3.9.1
[14:13:56] Local version 3.9.1
$ npm --version
8.6.0
$ node --version
v18.7.0
$ gulp server
fs.js:47
} = primordials;
    ^

ReferenceError: primordials is not defined
    at fs.js:47:5
    at req_ (/Users/alper/git/pdf.js/node_modules/natives/index.js:143:24)
    at Object.req [as require] (/Users/alper/git/pdf.js/node_modules/natives/index.js:55:10)
    at Object.<anonymous> (/Users/alper/git/pdf.js/node_modules/gulp/node_modules/graceful-fs/fs.js:1:37)
    at Module._compile (node:internal/modules/cjs/loader:1120:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1174:10)
    at Module.load (node:internal/modules/cjs/loader:998:32)
    at Module._load (node:internal/modules/cjs/loader:839:12)
    at Module.require (node:internal/modules/cjs/loader:1022:19)
    at require (node:internal/modules/cjs/helpers:102:18)

Node.js v18.7.0

$ npm install --legacy-peer-deps
$ gulp --version
[14:13:56] CLI version 3.9.1
[14:13:56] Local version 3.9.1

That looks strange, given the gulp version that we require; please note:
https://github.com/mozilla/pdf.js/blob/b8aa9c62217492c845ae3f27cc368aec7a58f38a/package.json#L33
Could you try running npm install instead (or if that doesn't work, npm install -f)?
If that still doesn't work, it's quite possible that it's actually a bug gulp or maybe it doesn't (yet) support the latest Node.js version. Does the current LTS release of Node.js work?

I had the same problem as @avatar-lavventura reported. But after I ran npm install<EMAIL_ADDRESS>--force , the problem was gone.

I can't solve this error in node version 22.0.

thanks.
I upgraded v20 and have solved this problem


Fix Document constructor Typescript bindings

Description of change
This replaces the constructor for WasmDocument to fix a type mismatch in the generated bindings. Several Document constructors are also replaced with a single new() function in both Rust and Wasm. This is for consistency and to clarify the canonical method of constructing a new Document.
The downside is that this removes the ability for a developer to pass in the desired fragment name for the authentication verification method on a newly generated Document in Wasm (e.g. instead of "#authentication" one could specify "#customkey123"). This was not possible in Rust, if that behaviour is still desired we should add it in this PR to both languages.
Edit: added the ability to set the default verification method fragment in both Rust and Wasm.
Breaking changes:

replace IotaDocument from_keypair(), from_keypair_with_network() with new(), new_with_options().
the default authentication verification method fragment for a new WasmDocument is changed from "#key" to "#authentication"

this discrepancy seemed unintentional and now matches the Rust behaviour

new Document() in the Wasm bindings now take a KeyPair instead of a KeyType and returns just a Document instead of a NewDocument { Document, KeyPair } struct.

BEFORE: pub fn new(type_: KeyType, network: Option<String>, tag: Option<String>) -> Result<NewDocument>
AFTER:  pub fn new(key: KeyPair, network: Option<String>, fragment: Option<String>) -> Result<Document>

Creating a new DID document in Wasm is also now a two-step process, which should be acceptable since this only affects the low-level API and matches the Rust experience.
BEFORE:
const { doc, key } = new Document(KeyType.Ed25519);

AFTER:
const key = new KeyPair(KeyType.Ed25519);
const doc = new Document(key);

NOTE: this will require updating the repl.it examples in the documentation.
Motivation
This is currently a mismatch in the expected type returned by the WasmDocument constructor compared to the generated Typescript definition:
  /// Creates a new DID Document from the given KeyPair.
  #[wasm_bindgen(constructor)]
  #[allow(clippy::new_ret_no_self)]
  pub fn new(type_: KeyType, network: Option<String>, tag: Option<String>) -> Result<NewDocument> {

where NewDocument is a struct holding the new WasmDocument and KeyPair:
#[wasm_bindgen(inspectable)]
pub struct NewDocument {
  key: KeyPair,
  doc: WasmDocument,
}

However, the generated Typescript constructor implicitly expects it to return WasmDocument, not NewDocument:
export class Document {
  free(): void;
/**
* Creates a new DID Document from the given KeyPair.
* @param {number} type_
* @param {string | undefined} network
* @param {string | undefined} tag
*/
  constructor(type_: number, network?: string, tag?: string);

Unfortunately there is no easy way to catch these types of bugs without creating dedicated Typescript tests or converting our examples to Typescript. I tried using tsc --allowJs --checkJs --noEmit on the examples but it doesn't link up to the Typescript definitions to catch the type-mismatch.
Minimal Reproducible Example
The following code snippet fails to compile when using Typescript:
example.ts:
import { Document, KeyType } from "@iota/identity-wasm/node";
const { doc, key } = new Document(KeyType.Ed25519);

Run tsc example.ts
Error:
example.ts:2:13 - error TS2339: Property 'doc' does not exist on type 'Document'.

7     const { doc, key } = new Document(KeyType.Ed25519);
              ~~~

example.ts:2:18 - error TS2339: Property 'key' does not exist on type 'Document'.

7     const { doc, key } = new Document(KeyType.Ed25519);
                   ~~~

Found 2 errors.

Note that this example runs fine in Javascript because it doesn't validate the expected type.
Type of change

[ ] Bug fix (a non-breaking change which fixes an issue)
[ ] Enhancement (a non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation Fix

How the change has been tested
Tests and examples were updated and pass locally. New constructor compiles to Typescript.
Change checklist
Add an x to the boxes that are relevant to your changes.

[x] I have followed the contribution guidelines for this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

One thought while reading over new_with_options was that we could ask for an Option<NetworkName> as the network name. But I suppose the reasoning is that this is the low-level API which shouldn't impose this restriction, and I would agree with that. We can think about changing IdentityCreate in the account though, which takes an arbitrary String as the network name.

The reason for using Opton<&str> was more to do with ergonomics. What I really wanted to use was Option<impl TryInto<NetworkName>> as the parameter. The problem I encountered was that using any generic as an optional parameter caused issues with None required a type annotation.
Example:
pub fn new_with_options(keypair: &KeyPair, network: Option<impl TryInto<NetworkName>>, fragment: Option<&str>)

Trying to use new_with_options with None as the network parameter causes annoying issues since it requires a concrete type for monomorphisation:
let document = IotaDocument::new_with_options(&keypair, None, Some("fragment"))?;

error[E0283]: type annotations needed
   --> identity-iota\src\did\doc\iota_document.rs:977:47
    |
119 |   pub fn new_with_options(keypair: &KeyPair, network: Option<impl TryInto<NetworkName>>, fragment: Option<&str>) -> Result<Self> {
    |                                                                   -------------------- required by this bound in `IotaDocument::new_with_options`
...
977 |     let result: Result<IotaDocument, Error> = IotaDocument::new_with_options(&keypair, None, Some("fragment"));
    |                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot infer type for type parameter `impl TryInto<NetworkName>` declared on the associated function `new_with_options`
    |
    = note: cannot satisfy `_: TryInto<network::NetworkName>`
help: consider specifying the type argument in the function call
    |
977 |     let result: Result<IotaDocument, Error> = IotaDocument::new_with_options::<impl TryInto<NetworkName>>(&keypair, None, Some(""));
    |                                                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Using just a NetworkName as the parameter is fine, and it is better than just a &str in that it communicates the restriction to the caller at compile time. It is, however, slightly more cumbersome:
E.g. with Option<NetworkName>:
pub fn new_with_options(keypair: &KeyPair, network: Option<NetworkName>, fragment: Option<&str>) -> Result<Self>;

// BEFORE:
// let document = IotaDocument::new_with_options(&keypair, Some("dev"), Some("auth-key"))?;

// AFTER:
let document = IotaDocument::new_with_options(&keypair, Some(Network::Devnet.name()), Some("auth-key"))?;
// OR
let network_name = NetworkName::try_from("dev");
let document = IotaDocument::new_with_options(&keypair, Some(network_name), Some("auth-key"))?;

One can also make the same argument to use NetworkName for IotaDID::new_with_network, which currently uses &str too:
pub fn new_with_network(public: &[u8], network: &str) -> Result<Self>

On a final note, we don't seem to expose NetworkName in the Wasm bindings yet, we probably should if only for developers to validate that arbitrary strings are valid IOTA DID network names using our API.

I mainly thought of it because you used
let document = IotaDocument::new_with_options(&keypair, Some(Network::Devnet.as_name_str()), Some("auth-key"))?;

before, which isn't less cumbersome than
let document = IotaDocument::new_with_options(&keypair, Some(Network::Devnet.name()), Some("auth-key"))?;

But the main question is if the low-level API should restrict that, or in other words, should the low-level API enforce the IOTA DID method spec? In this particular case, (after thinking more about this) I think it's actually okay to do so (contrary to what I said above).

True, then it really only affects private Tangle network names:
let document = IotaDocument::new_with_options(&keypair, Some("private"), Some("auth-key"))?;

// vs

let network_name = NetworkName::try_from("private")?;
let document = IotaDocument::new_with_options(&keypair, Some(network_name), Some("auth-key"))?;

@PhilippGackstatter I changed the parameters for IotaDocument::new_with_options(), IotaDID::new_with_network() and IdentityCreate::network() to use NetworkName instead of &str. However, I used impl TryInto<NetworkName> so developers can still pass (static/owned) strings if they so choose. During the refactor I started to feel like NetworkName should be an internal construct, since it's so similar to Network and is a bit confusing to deal with in multiple places. I'm not entirely happy with this last change since it's a bit less clean and doesn't translate well to Wasm, but the one benefit is that IdentityCreate::network is now fallible, so invalid NetworkNames are caught earlier.


CivitAI direct link method: slow download speed

taking forever to download from civitai

Naming is not the issue here . Download speed is slow from civitai.

Naming is not the issue here . Download speed is slow from civitai.

no no, you don't get it. hear me out
links with civitai.com/api/download/models doesn't have a filename in it.
request module is needed to get the filename correctly from content-disposition
downloading the entire file is a must to get that
it's a waste of time when downloading an entire file just to get the filename, then redownloading it again
so i just used the downloaded file from request

I made this branch to (hopefully) fix that problem
!rm -rf /content/stable-diffusion-webui/extensions/batchlinks-webui
!git clone -b civitupgrade https://github.com/etherealxx/batchlinks-webui /content/stable-diffusion-webui/extensions/batchlinks-webui

this should use the default downloader instead of requests for downloading civit stuff.
i need to make sure this works first before merging with main.

https://colab.research.google.com/drive/1wEa-tS10h4LlDykd87TF5zzpXIIQoCmq#scrollTo=8Uvqba0yKW7U
This colabs download mechanism got the code you could reuse

https://colab.research.google.com/drive/1wEa-tS10h4LlDykd87TF5zzpXIIQoCmq#scrollTo=8Uvqba0yKW7U This colabs download mechanism got the code you could reuse

mine is better coz it supports MEGA👀 (though for gdrive, i'll add it on the next update, it's complicated to implement that without bugs)
anyway, he's using direct download method, nothing special with it. It's literally the download method i use to download huggingface etc.
His code also can't differentiate .safetensors embedding/LoRA, his system is flawed

Download speed is slow from civitai.

Hello @faisalhr1997👋 The update is now live! It upgrades the method used to download from civitai. Now you can use aria2 and enjoy the maximum downlad speed! (it also now support gdrive links!✨)
Release note (every new feature): https://github.com/etherealxx/batchlinks-webui#release-v301


mogrify: no decode delegate for this image format `' @ error/constitute.c/ReadImage/746.

Thank you very much for your kind work.
I use “  bash scripts/local_colmap_and_resize.sh /multinerf-main/data/Lego”
but I encountered this problem “ mogrify: no decode delegate for this image format `' @ error/constitute.c/ReadImage/746.”
details as follow:
Bundle adjustment report
Residuals : 121368

Parameters : 40382
Iterations : 3
Time : 0.739306 [s]
Initial cost : 0.479008 [px]
Final cost : 0.467438 [px]
Termination : Convergence
=> Completed observations: 3
=> Merged observations: 0
=> Filtered observations: 27
=> Changed observations: 0.000494
=> Filtered images: 0
Elapsed time: 1.203 [minutes]
~/multinerf-main/data/Lego/images_2 ~/multinerf-main
mogrify: no decode delegate for this image format ' @ error/constitute.c/ReadImage/746. ~/multinerf-main ~/multinerf-main/data/Lego/images_4 ~/multinerf-main mogrify: no decode delegate for this image format ' @ error/constitute.c/ReadImage/746.
~/multinerf-main
~/multinerf-main/data/Lego/images_8 ~/multinerf-main
mogrify: no decode delegate for this image format `' @ error/constitute.c/ReadImage/746.
can you help me?

Did you get anywhere with this Issue @mints7 ? I am having the same problem

Same. Have u solve it？

Same problem. How to solve?


v3.alpha.2: @kurkle/color dependency

Feature Proposal

as far as I can see @kurkle/color dependency is bundled with chart.js in a build. Thus, there is no need to declare it as a dependency but just as a devDependency.
Feature Use Case

avoid installing not needed dependencies
Possible Implementation

It's imported by https://github.com/chartjs/Chart.js/blob/master/src/helpers/helpers.color.js, so I think dependency is right? I think devDependency is just for build-chain and tests

yes but rollup is configured to include it the chart.js bundle, thus you don't need the dependency during runtime making it a bundled or development dependency

Ah, so is the issue that by leaving it under dependencies it gets installed by npm which it doesn't need to because it's already bundled?


Use php 7.4.* as default version

php-dist v0.1.0 doesn't support 7.2.* anymore
Please confirm the following:

[X] I have viewed, signed, and submitted the Contributor License Agreement.
[ ] I have added an integration test, if necessary.

Seems to be done as part of https://github.com/paketo-buildpacks/php-web/pull/108


Tool tip Doesn't display on Hover

Hi,
Thanks for the great work on this tooltip. This is a life saver for many of us.
I have cloned this repo and tried to run it locally. On the rendered page, i do not get the tooltip on hover, It shows on click. Also the same behavior is seen when i include it in my project.
I have this as my options:
	'animation-duration': 300, 	'animation-duration-default': 300, 	'trigger': 'hover', 	'tooltip-class': '', 	'display': true,  	'display-mobile': true,
usage:
 <div *ngIf="alertAvailble  else noAlert" tooltip="Lorem ipsum dolor sit amet, his ut dicunt recteque ea."                    placement="top">some content here              </div>  <ng-template #noAlert>--</ng-template>
can you let me know what i am missing?

There seems to be an issue with isDisplayOnHover in directive, I changed it to this for making it work as expected:
get isDisplayOnHover(): boolean {
                  if (this.options['trigger'] === 'hover') {
                   return true;
                  } else if (this.isMobile || this.options['display'] === false) {
                    return false;
              }
              return true;
   }`

@returnlytom : sure will send it soon.


Delly Themes Centered Home Patch

Centered Home, a very popular theme on CSS Loader, requires a patch for transparent or blurred footers.

New to GitHub (not new to Git - thought I was creating a PR to my fork rather than your repo, planned to squash commits). My suggestion is to add a patch for Centered Home, to reposition the home screen, accommodating for transparent or blurred footers.


No pth checkpoint for HRSOD

Hello, thank you for your work, can not find pretrained weights for Sailent Object Detection, the gdrive contains only checkpoints for DIS and COD. Could you please provide well-trained weights on HRSOD too.

Sure. But as I mentioned in README, I've been recently re-training my model on only a single A100-40G card.
And training in HRSOD actually costs the most computation, since there are many settings and more images. It's still in training. But I just got the final results in the setting of using DUTS (D) as the only training set. Well-trained weights and predicted maps are being updated in google-drive, you can download them in 10 mins. And results are even better than the previous, which I will update in the next version of our paper:
+----------+-----------+----------+-------+--------+------+-------+--------+-----------+-------+-------+------+
& Dataset  &   Method  & Smeasure & maxFm & meanEm & MAE  & maxEm & meanFm & wFmeasure & adpEm & adpFm & HCE  &
+----------+-----------+----------+-------+--------+------+-------+--------+-----------+-------+-------+------+
& DAVIS-S & tm--ep110 &   .965   &  .964 &  .983  & .008 &  .992 &  .937  &    .937   &  .975 &  .905 & .000 &
& DUT-OMRON & tm--ep110 &   .870   &  .820 &  .884  & .043 &  .892 &  .806  &    .797   &  .884 &  .795 & .000 &
& TE-DUTS & tm--ep110 &   .938   &  .932 &  .956  & .020 &  .965 &  .912  &    .910   &  .937 &  .871 & .000 &
& TE-HRSOD & tm--ep110 &   .956   &  .956 &  .973  & .014 &  .982 &  .936  &    .929   &  .963 &  .911 & .000 &
& TE-UHRSD & tm--ep110 &   .932   &  .935 &  .946  & .028 &  .954 &  .921  &    .906   &  .925 &  .890 & .000 &
+----------+-----------+----------+-------+--------+------+-------+--------+-----------+-------+-------+------+


Update cartodb-ui to 4.6.20

Update cartodb-ui to 4.6.20 in order to grab cartodb.js@v4 updates, as
per the procedure described here:
https://github.com/CartoDB/cartodb.js/pull/1569#issuecomment-283940933
Deployment procedure for this issue:
https://github.com/CartoDB/tangram-carto/issues/28

@donflopez can you please review? I guess this normally goes straight into master but I prefered to do it via pull request to be on the safe this first time.

After second thought I think I'm being a bit overcautious with this :sweat_smile:
I'm merging and deploying now.


Make it clearer that this is the biocoin wallet source

Make it clearer that this is the biocoin wallet source, please.
For example by updating the README.md file and instead of or besides the port information state that this is the biocoin wallet.
Is that possible?  It would make it easier to find the wallet.

It needs to be built on Linux. all you need to build is here https://github.com/Blackithart/biocoin


Server hooks executing before client hooks

I have an after.update hook which modifies some associated collections. It seems these are running firstly on the server and then on the client. Before the client has a chance to run the appropriate code, it seems the doc has already synced the changes from the server, since the changes in the hook have already taken place. This caused my hook to fail an assertion and fail early. The end result is as expected though, but I still get the error logged unintentionally.
To avoid this, I've wrapped my hooks in if Meteor.isServer. This gets rid of the issue, but it means the client is now no longer executing the updates in the hook. Since this is only affecting other documents it's not import to run them on the client - they're synced anyway.
For some hooks, I think it's still important to have them run on both the client and server - in particular those which update the doc (e.g. before.insert). I'm not sure what would happen if the server doc after the insert was different to the client doc.
I just wanted to mention this in case others experience it.

I might be wrong but it sounds like a side-effect of allow/deny. Since the allow/deny logic has to run on the server, the client can't ever run an after.update as a direct result of the original call to update -- it has to be triggered by the sequence of events that allow/deny causes. Once allow/deny has done its work, the server will perform the update, which in turn will push changes via publish. The client would then feel the effects of the after.update.

Thanks, that makes sense. From what I can tell from this answer the allow rules are only run for the client when posting to the server? But the sequence of events would explain why I'm seeing the client hook executed second. I guess I'll leave all before hooks to run on both the client and server to ensure the models are consistent, and any after hooks should only run on the server and have their changes sync on the client.


Protractor fails on 5th test listed in scenarios.js

Just installed phonecat tutorial, node.js etc but when I run npm run protractor the 5th test seems to fail virtually every time - but not always. I'm running this on a fairly old laptop (using Lubuntu) so I wonder if that's the issue.
Here's the error in the terminal:
selenium standalone is up to date.
chromedriver is up to date.

><EMAIL_ADDRESS>protractor /home/david/angular-phonecat
> protractor test/protractor-conf.js

Using ChromeDriver directly...
....F..

Failures:

  1) PhoneCat App Phone detail view should display nexus-s page
   Message:
     Expected '' to be 'Nexus S'.
   Stacktrace:
     Error: Failed expectation
    at [object Object].<anonymous> (/home/david/angular-phonecat/test/e2e/scenarios.js:84:59)

Finished in 19.212 seconds
7 tests, 11 assertions, 1 failure

npm ERR<EMAIL_ADDRESS>protractor: `protractor test/protractor-conf.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>protractor script.
npm ERR! This is most likely a problem with the angular-phonecat package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     protractor test/protractor-conf.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls angular-phonecat
npm ERR! There is likely additional logging output above.
npm ERR! System Linux 3.13.0-39-generic
npm ERR! command "/usr/local/bin/node" "/usr/local/bin/npm" "run" "protractor"
npm ERR! cwd /home/david/angular-phonecat
npm ERR! node -v v0.10.33
npm ERR! npm -v 1.4.28
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /home/david/angular-phonecat/npm-debug.log
npm ERR! not ok code 0

If I comment out the 5th test within the scenarios.js file, shown below, then the rest of the tests succeed everytime:
it('should display nexus-s page', function() {
    expect(element(by.binding('phone.name')).getText()).toBe('Nexus S');
});

It seems strange that it passes sometimes.

I think this is because the tests are slowed down by the animations? i have fixed locally by disabling animation.


Missing await in sync_all_commands

Task exception was never retrieved
future: <Task finished name='Task-26' coro=<SlashCommand.sync_all_commands() done, defined at C:\Users\henry\AppData\Local\Programs\Python\Python39\lib\site-packages\discord_slash\client.py:294> exception=TypeError("object of type 'coroutine' has no len()")>
Traceback (most recent call last):
  File "C:\Users\henry\AppData\Local\Programs\Python\Python39\lib\site-packages\discord_slash\client.py", line 353, in sync_all_commands
    if len(existing) != 0:
TypeError: object of type 'coroutine' has no len()
C:\Users\henry\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py:1891: RuntimeWarning: coroutine 'HTTPClient.request' was never awaited
  handle = None  # Needed to break cycles when an exception occurs.
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

There is a missing await here, due to SlashCommandRequest.get_all_commands calling self.command_request, both of which are functions; command_request returns the result of calling self._discord.http.request, which is/returns a coroutine.
sync_all_commands needs an await, but get_all_commands and command_request should probably be asynchronous and themselves await that which they call.

From looking at the source, changing command_request would require a lot of functions to be changed to coroutines, but it would make the code much clearer; alternatively, type annotation could help clarify (ie. def f() -> typing.Coroutine: or collections.abc.Coroutine)


External storage: split checkBackend and configure (follow-up #545)

This fixes an issue with external storage in saas environment.
See https://github.com/gristlabs/grist-core/pull/546/files#r1259340397

Ran tests with an alternate backend and it looks good :+1:


Context for Gettable and Settable methods

Hi all,
I wonder what happens if I just return a NULL from gettable or settable method? Because I'm almost sure that no code in openssl uses these methods to check if it can get or set a value before calling get or set method. I have seen a value is requested in my provider's get methods while I have not mentioned it in my gettable list. I think there are no correct tests regarding gettable and settable methods too. As a result, you see very strange and misleading codes regarding these methods.
As an example, look at this wrapper over signature settable method:
https://github.com/openssl/openssl/blob/bbe9d2de6c643a2c6758fae4274c307943a59624/crypto/evp/signature.c#L381-L390
which is the only usage of signature settable which in turn is not used anywhere. As you see, the value of NULL is passed as first argument for calling context. Now see its implementation in RSA singature:
https://github.com/openssl/openssl/blob/67bfdfa17bc4ca8e2b819316299bb82748394c45/providers/implementations/signature/rsa_sig.c#L1403-L1411
or here in ECDSA signature:
https://github.com/openssl/openssl/blob/67bfdfa17bc4ca8e2b819316299bb82748394c45/providers/implementations/signature/ecdsa_sig.c#L554-L561
Even though these implementation don't create segmentation fault because of null check, from implementation it is obvious that they expect to get signature context to check flag_allow_md, but since value is NULL, it never happens.
From what I experienced in developing provider, I noticed that gettable and settable methods are almost useless because they don't affect how openssl works (at least in parts that I have checked till now).
I think it will be much better to remove the requirement of implementing them at all or at least, provide correct context to them.

You might be happier to know that this particular function is also called from EVP_PKEY_CTX_settable_params()
Here's the code that performs the call:
https://github.com/openssl/openssl/blob/bbe9d2de6c643a2c6758fae4274c307943a59624/crypto/evp/pmeth_lib.c#L813-L820

@levitte Sorry about my mistake. I somehow missed this call from ctx. Thanks for the info .


fix: updates MSI adapter

Reason for Change:

This PR upgrades the MSI adapter used in Arc extension.

Requirements

[x] squashed commits
[ ] included documentation
[ ] added unit tests and e2e tests (if applicable).

Issue Fixed:

Does this change contain code from or inspired by another project?

[ ] Yes
[x] No

If "Yes," did you notify that project's maintainers and provide attribution?
Special Notes for Reviewers:

/azp run pr-e2e-azure

It fetches the correct container image of msi-adapter
nilekh$ kubectl get pod -n kube-system akvsecretsprovider-arc-monitoring-86c8974765-4jw5t -o=jsonpath='{.spec.containers[*].image}' | tr -s '[[:space:]]' '\n'
upstreamarc.azurecr.io/prom-mdm-converter:v1.0.2
linuxgeneva-microsoft.azurecr.io/distroless/genevamdm:2.2023.1027.1417-08a588-20231027t1613
mcr.microsoft.com/azurearck8s/msi-adapter:1.0.6
mcr.microsoft.com/oss/mirror/docker.io/library/telegraf:1.21
linuxgeneva-microsoft.azurecr.io/amacoreagentaot:1.2.47
linuxgeneva-microsoft.azurecr.io/distroless/genevafluentd_td-agent:mariner_20230915.3


Git via agent fails behind a corporate proxy server

Agent version and platform
Version of your agent? 2.109.0

OS of the machine running the agent? Windows 7 Pro SP1.

VSTS type and version
VisualStudio.com

If VisualStudio.com, what is your account name? http://vsosse.visualstudio.com

What's not working?
The build agent is running on a corporate network and behind a proxy server. Using an Environment Variable VSTS_HTTP_PROXY set with the proxy server address, the agent registers fine with our visualstudio.com account using a PAT. However when we kick off a build it fails on Get sources trying to download from our Git repo. Error shown below in screenshot:

I've tried with adding the proxy creds as environment variables too and can see they get added to the http.proxy value in format username:password@proxyserver however to no avail.
Note* Git does work on the machine running the agent when using:
git config --global http.proxy http://mydomain\myusername:mypassword@myproxyserver:80
It asks for the PAT to authenticate with visualstudio.com. However the above setting allows it to communicate via the proxy server.

@yugmac https://www.visualstudio.com/en-us/docs/build/admin/agents/v2-windows#run-the-agent-behind-a-web-proxy
in addition to VSTS_HTTP_PROXY, also set VSTS_HTTP_PROXY_USERNAME and VSTS_HTTP_PROXY_PASSWORD follow the Doc

@TingluoHuang  Hello, I have followed the Doc for setting up the agent behind a proxy server. As I stated in the above issue I have tried to use the username and password variables too however to no avail.
Using VSTS_HTTP_PROXY_USERNAME and VSTS_HTTP_PROXY_PASSWORD appends the values onto the http.proxy url address but this still does not work.

I noticed that in the console the agent outputs text saying "Agent is running behind proxy server: 'http://myproxyserver:80'" and when it fails the error shows "Failed to connect to myproxyserver port 1080: Connection refused"
Does it matter that one states port 80 and the other port 1080?
Thanks.

Have you try put port info also in VSTS_HTTP_PROXY? like VSTS_HTTP_PROXY = http://localhost:8888

I have the Environment Variables as follows:
VSTS_HTTP_PROXY = http://myproxyserver:80
VSTS_HTTP_PROXY_USERNAME = domain\username
VSTS_HTTP_PROXY_PASSWORD = password
The agent then communicates with visualstudio.com through the proxy server using port 80 and it works. However Git still fails with the above error message stating port 1080.

have you restart the agent process after you set those vars? can you send me your build log and agent diag log<EMAIL_ADDRESS>

I have restarted, removed and reconfigured. Updated to the latest agent, tried on various machines but all to no avail. Also ran the agent in interactive mode too as well as a windows service.
I will email you the log info .
Thanks

also, what kind of Proxy are you using? base on the error, git.exe is trying to use port 1080, which is the default port for SOCKS proxy.

@TingluoHuang  I have sent an email with the logs attached. Also I have requested the information you have asked from our operational security department. I do not have access to the proxy server or any information regarding it.

@yugmac please try https://github.com/Microsoft/vsts-agent/releases/tag/v2.109.2

@yugmac does the new version agent unblock you?

@TingluoHuang sorry for not replying sooner, I was off work on annual leave. Anyway, the new version of the agent works and Git is able to communicate via the proxy server. Thanks!
However one more thing, is there another way to save proxy credentials rather than using environmental variables? These show the proxy authentication username and password in plain text!
Thanks

@yugmac cool, thanks for your validation. The agent will make sure your credential been masked out in any log file or console output. We haven't find a better way to store your proxy credential xplat, we can use dpapi in windows, about dotnet core haven't port that to linux/osx.
I will close this issue. Thanks for reporting.

@TingluoHuang  yeah you can close this issue! Thank you.
Well if you need to use Environment variables, what about not showing them on the VSTS agent dashboard under Capabilities.

Could this be done? I know you mask them in log files and console output however they are still shown in plain text as the screenshot above shows.

@yugmac i will fix it.

@TingluoHuang  Aww brilliant! Thanks! Would you like me to create a new issue for you to work against?

@yugmac sure

@TingluoHuang  Okay new issue #686 thank you! :+1:


Kubernetes   starrocks 2.5.4 be  errors

env：    Kubernetes  v1.26.1
operator:   starrocks/operator:latest
deployment:  starrocks/fe-ubuntu:2.5.4    starrocks/be-ubuntu:2.5.4
fe work well , be can't be started
[Mon Apr 17 08:00:50 UTC 2023] Empty $CONFIGMAP_MOUNT_PATH env var, skip it!
[Mon Apr 17 08:00:50 UTC 2023] Add myself (starrockscluster-be-0.starrockscluster-be-search.open.svc.k8s.com:9050) into FE ...
ERROR 1064 (HY000) at line 1: Unexpected exception: Same backend already exists[starrockscluster-be-0.starrockscluster-be-search.open.svc.k8s.com:9050]
[Mon Apr 17 08:00:50 UTC 2023] run start_be.sh
/opt/starrocks/be/bin/start_backend.sh: line 185:   807 Illegal instruction     (core dumped) ${START_BE_CMD} "$@" >> ${LOG_FILE} 2>&1 < /dev/null

please check your cpu info (/proc/cpuinfo)
by default, starrocks_be requires cpu to support avx2.

same problem.
[root@localhost starrocks]# kubectl logs starrockscluster-be-0
[Wed May 31 00:56:20 UTC 2023] Empty $CONFIGMAP_MOUNT_PATH env var, skip it!
[Wed May 31 00:56:20 UTC 2023] Add myself (starrockscluster-be-0.starrockscluster-be-search.starrocks.svc.cluster.local:9050) into FE ...
ERROR 1064 (HY000) at line 1: Unexpected exception: Same backend already exists[starrockscluster-be-0.starrockscluster-be-search.starrocks.svc.cluster.local:9050]
[Wed May 31 00:56:20 UTC 2023] run start_be.sh
/opt/starrocks/be/bin/start_backend.sh: line 185:   808 Illegal instruction     (core dumped) ${START_BE_CMD} "$@" >> ${LOG_FILE} 2>&1 < /dev/null

and k8s node already enable avx2.
[root@localhost starrocks]# kubectl get pod -o wide
NAME                                    READY   STATUS             RESTARTS   AGE   IP            NODE                NOMINATED NODE   READINESS GATES
starrocks-controller-79b5f48bf6-bkx48   1/1     Running            2          20h   <IP_ADDRESS>   rancher2-worker7    <none>           <none>
starrockscluster-be-0                   0/1     CrashLoopBackOff   16         61m   <IP_ADDRESS>   rancher2-worker15   <none>           <none>
starrockscluster-be-1                   0/1     CrashLoopBackOff   211        17h   <IP_ADDRESS>   rancher2-worker10   <none>           <none>
starrockscluster-be-2                   0/1     CrashLoopBackOff   210        17h   <IP_ADDRESS>   rancher2-worker11   <none>           <none>
starrockscluster-fe-0                   1/1     Running            0          17h   <IP_ADDRESS>   rancher2-worker12   <none>           <none>
starrockscluster-fe-1                   1/1     Running            0          17h   <IP_ADDRESS>   rancher2-worker14   <none>           <none>
starrockscluster-fe-2                   1/1     Running            0          17h   <IP_ADDRESS>   rancher2-worker13   <none>           <none>

on rancher2-worker15:
[root@rancher2-worker15 ~]# hostname
rancher2-worker15
[root@rancher2-worker15 ~]# lscpu | grep Flags|grep avx2
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni ssse3 cx16 sse4_1 sse4_2 x2apic avx hypervisor lahf_lm cpuid_fault pti avx2 bmi2

on rancher2-worker10:
[root@rancher2-worker10 ~]# hostname
rancher2-worker10
[root@rancher2-worker10 ~]# lscpu | grep Flags|grep avx2
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni ssse3 cx16 sse4_1 sse4_2 x2apic avx hypervisor lahf_lm cpuid_fault pti avx2 bmi2

on rancher2-worker11:
[root@rancher2-worker11 ~]# hostname
rancher2-worker11
[root@rancher2-worker11 ~]# lscpu | grep Flags|grep avx2
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni ssse3 cx16 sse4_1 sse4_2 x2apic avx hypervisor lahf_lm cpuid_fault pti avx2 bmi2

starrocks-fe-and-be.yaml:
apiVersion: starrocks.com/v1alpha1
kind: StarRocksCluster
metadata:
  name: starrockscluster
  namespace: starrocks
spec:
  starRocksFeSpec:
    image: starrocks/fe-ubuntu:2.5.4
    replicas: 3
    requests:
      cpu: 1
      memory: 3Gi
    nodeSelector:
      dedicated: starrocks
#    affinity:
#      nodeAffinity:
#        preferredDuringSchedulingIgnoredDuringExecution:
#        - weight: 1
#          preference:
#            matchExpressions:
#            - key: dedicated
#              operator: In
#              values:
#              - starrocks
    tolerations:
      - key: "dedicated"
        operator: "Equal"
        value: "starrocks"
        effect: "NoSchedule"
    storageVolumes:
      - name: fe-meta
        storageClassName: ceph-dba
        storageSize: 10Gi
        mountPath: /opt/starrocks/fe/meta # overwrite the default meta path
  starRocksBeSpec:
    image: starrocks/be-ubuntu:2.5.4
    replicas: 3
    requests:
      cpu: 1
      memory: 3Gi
    nodeSelector:
      dedicated: starrocks
    tolerations:
      - key: "dedicated"
        operator: "Equal"
        value: "starrocks"
        effect: "NoSchedule"
    storageVolumes:
      - name: be-data
        storageClassName: ceph-dba
        storageSize: 30Gi
        mountPath: /opt/starrocks/be/storage # overwrite the default data path

help

@welyss are you able to apply following pod spec and then manual start the BE process, to get the detailed error log
apiVersion: v1
kind: Pod
metadata:
  name: starrocks-be-single-pod
spec:
  containers:
  - args:
    - "86400"
    command:
    - sleep
    image: <IP_ADDRESS>:5000/starrocks/be-ubuntu:2.5.6
    imagePullPolicy: IfNotPresent
    name: be-single-pod
    resources:
      limits:
        cpu: "4"
        memory: 8Gi
      requests:
        cpu: "1"
        memory: 3Gi

After the pod is running, run following command to manual start the be process
kubectl exec -it starrocks-be-single-pod  -- /opt/starrocks/be/bin/start_be.sh

This command is expected to crash with the same error as you experienced.
And then please send us the following log files /opt/starrocks/be/log/be.out /opt/starrocks/be/log/be.INFO

@kevincai , thanks reply.
[root@localhost starrocks]# cat starrocks-be-single-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: starrocks-be-single-pod
  namespace: starrocks
spec:
  containers:
  - args:
    - "86400"
    command:
    - sleep
    image: starrocks/be-ubuntu:2.5.4
    imagePullPolicy: IfNotPresent
    name: be-single-pod
    resources:
      limits:
        cpu: "4"
        memory: 8Gi
      requests:
        cpu: "1"
        memory: 3Gi
  tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "starrocks"
    effect: "NoSchedule"
  nodeSelector:
    dedicated: starrocks

[root@localhost starrocks]# kubectl get pod starrocks-be-single-pod -o wide
NAME                      READY   STATUS    RESTARTS   AGE   IP            NODE                NOMINATED NODE   READINESS GATES
starrocks-be-single-pod   1/1     Running   0          29m   <IP_ADDRESS>   rancher2-worker15   <none>           <none>

then run the command:
[root@localhost starrocks]# kubectl exec -it starrocks-be-single-pod -- /opt/starrocks/be/bin/start_be.sh
/opt/starrocks/be/bin/start_backend.sh: line 185:   797 Illegal instruction     (core dumped) ${START_BE_CMD} "$@" >> ${LOG_FILE} 2>&1 < /dev/null
command terminated with exit code 132

here is log:
[root@localhost starrocks]# kubectl exec -it starrocks-be-single-pod -- /bin/bash
root@starrocks-be-single-pod:/opt/starrocks# cat /opt/starrocks/be/log/be.out
start time: Wed May 31 06:17:45 UTC 2023
root@starrocks-be-single-pod:/opt/starrocks# cat /opt/starrocks/be/log/be.INFO
cat: /opt/starrocks/be/log/be.INFO: No such file or directory

@welyss can you check if there is any related information from dmesg command output, after the Illegal instruction error?

@welyss can you check if there is any related information from dmesg command output, after the Illegal instruction error?

root@starrocks-be-single-pod:/opt/starrocks# /opt/starrocks/be/bin/start_be.sh
/opt/starrocks/be/bin/start_backend.sh: line 185:  1608 Illegal instruction     (core dumped) ${START_BE_CMD} "$@" >> ${LOG_FILE} 2>&1 < /dev/null
root@starrocks-be-single-pod:/opt/starrocks# dmesg

dmesg.txt

Thanks, will check the following error in dmesg
[81826.337385] traps: starrocks_be[652] trap invalid opcode ip:2c1101b sp:7ffdc6698860 error:0 in starrocks_be[27ec000+5884000]
[82142.091083] traps: starrocks_be[6919] trap invalid opcode ip:2c1101b sp:7ffe2e24ed50 error:0 in starrocks_be[27ec000+5884000]
[82461.279073] traps: starrocks_be[12117] trap invalid opcode ip:2c1101b sp:7ffdf09ce070 error:0 in starrocks_be[27ec000+5884000]
[82544.108128] traps: starrocks_be[14115] trap invalid opcode ip:2c1101b sp:7ffe2dd7f220 error:0 in starrocks_be[27ec000+5884000]

@welyss can you run uname -a in the pod, need to confirm the kernel version. thanks!

and also, if possible, please paste the full content of lscpu, help us to understand why it reports invalid opcode.

ok, this is on k8s worker node which starrocks-be scheduled.
[root@rancher2-worker15 ~]# hostname
rancher2-worker15
[root@rancher2-worker15 ~]# uname -a
Linux rancher2-worker15 4.20.7-1.el7.elrepo.x86_64 #1 SMP Wed Feb 6 13:17:46 EST 2019 x86_64 x86_64 x86_64 GNU/Linux
[root@rancher2-worker15 ~]# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    1
Core(s) per socket:    2
Socket(s):             2
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            15
Model:                 6
Model name:            Common KVM processor
Stepping:              1
CPU MHz:               2095.076
BogoMIPS:              4190.15
Hypervisor vendor:     KVM
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              4096K
L3 cache:              16384K
NUMA node0 CPU(s):     0-3
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni ssse3 cx16 sse4_1 sse4_2 x2apic avx hypervisor lahf_lm cpuid_fault pti avx2 bmi2

in the pod:
[root@localhost starrocks]# kubectl exec -it starrocks-be-single-pod -- /bin/bash
root@starrocks-be-single-pod:/opt/starrocks# uname -a
Linux starrocks-be-single-pod 4.20.7-1.el7.elrepo.x86_64 #1 SMP Wed Feb 6 13:17:46 EST 2019 x86_64 x86_64 x86_64 GNU/Linux
root@starrocks-be-single-pod:/opt/starrocks# lscpu
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         46 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  4
  On-line CPU(s) list:   0-3
Vendor ID:               GenuineIntel
  Model name:            Common KVM processor
    CPU family:          15
    Model:               6
    Thread(s) per core:  1
    Core(s) per socket:  2
    Socket(s):           2
    Stepping:            1
    BogoMIPS:            4190.15
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni ssse3 cx16 sse4_1 sse4_2 x2apic avx hypervisor lahf_lm cpuid_fault pti avx2
                          bmi2
Virtualization features:
  Hypervisor vendor:     KVM
  Virtualization type:   full
Caches (sum of all):
  L1d:                   128 KiB (4 instances)
  L1i:                   128 KiB (4 instances)
  L2:                    16 MiB (4 instances)
  L3:                    32 MiB (2 instances)
NUMA:
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-3
Vulnerabilities:
  L1tf:                  Mitigation; PTE Inversion
  Meltdown:              Mitigation; PTI
  Spec store bypass:     Vulnerable
  Spectre v1:            Mitigation; __user pointer sanitization
  Spectre v2:            Mitigation; Full generic retpoline, STIBP disabled, RSB filling

@welyss appreciated for the information.
Some update on this issue:
What we know

the binary crashes on a specific cpu instruction vpxor which is a AVX2 instruction

What we don't understand

your CPU supports avx2 extension, why it cores anyway.

In the meanwhile, can you try use this image lvlouisaslia/allin1-ubuntu:3.0.0-no-avx2-no-sse42 instead of starrocks/be-ubuntu:2.5.4 in the starrocks-be-single-pod.yaml, and try start the BE process by
[root@localhost starrocks]# kubectl exec -it starrocks-be-single-pod -- /data/deploy/starrocks/be/bin/start_be.sh

check if it still crashes or not.

@welyss appreciated for the information.
Some update on this issue:
What we know
* the binary crashes on a specific cpu instruction `vpxor` which is a AVX2 instruction

What we don't understand
* your CPU supports avx2 extension, why it cores anyway.

In the meanwhile, can you try use this image lvlouisaslia/allin1-ubuntu:3.0.0-no-avx2-no-sse42 instead of starrocks/be-ubuntu:2.5.4 in the starrocks-be-single-pod.yaml, and try start the BE process by
[root@localhost starrocks]# kubectl exec -it starrocks-be-single-pod -- /data/deploy/starrocks/be/bin/start_be.sh

check if it still crashes or not.

ok, thankyou for help, image lvlouisaslia/allin1-ubuntu:3.0.0-no-avx2-no-sse42 will hold on frontend, no crash, guess it works.
then we change cpu model of kvm from 'avx' to 'host', it works.  it looks like [lscpu] can't definitely tell the cpu instruction has been loaded in vm system, perhaps there is something wrong with our kvm conf in cpu model 'avx'.

ok, thankyou for help, image lvlouisaslia/allin1-ubuntu:3.0.0-no-avx2-no-sse42 will hold on frontend, no crash, guess it works.

Yes, it worked then.

then we change cpu model of kvm from 'avx' to 'host', it works. it looks like [lscpu] can't definitely tell the cpu instruction has been loaded in vm system, perhaps there is something wrong with our kvm conf in cpu model 'avx'.

That's good to know, glad that the issue is solved. I will take a look at KVM virtualization related info.
Close this issue for now.


bugfix/Reintroduce-remove-event-handling

Fix for unremoved event handling(#9) was fixed in #11 previously. Due to
an issue while removing string event listener I had to revert it. This
commit reintroduces the fixes done in that PR and fixes the bug in
removal case.

We should re-think/re-write this whole attributes.ts file maybe?

Couldn't agree more on that.


Save sample data to json after exit

Fixed in recent commit. Bug was in sample data, tag was in the wrong format.


patch: i think we got it!

:tada: This PR is included in version 9.2.10 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Unable to copy blockchain from laptop on win7 via SCP

Hi as described above, it took 9 days to copy the blockchain to my laptop (win7), now i am finding it near impossible to transfer the files via scp, I can ssh into my pi with out any issues. I have also installed winSCP and all i get is a permission error.
I'll post some screen shots  shortly to explain my issue more in depth. Thanks

I just tested again ... its a 2.3GB file that is a disk image:
https://github.com/rootzoll/raspiblitz#downloading-the-software

Ah right i was looking at the releases on the right hand of the front page, i'll download this and see how i get on, thanks.

Hi just flashed the new software to sd card, running into the same problem when trying to run -
scp -r ./chainstate ./blocks bitcoin@<IP_ADDRESS>:/mnt/hdd/bitcoin
scp is not recognised as a internal or external command, operable
I have putty, pscp.exe and winscp installed but not able to transfer the files.

I dont have a WIndows 7 to test it with ... please let me know if that woked with "pscp -scp -r -P 22 ./chainstate ./blocks bitcoin@<IP_ADDRESS>:/mnt/hdd/bitcoin" would be good to add that info to the script.

Yes that worked, its downloading right now. I ssh into pi on my win7 laptop (where i have installed bitcoin core), then opened a new terminal and wrote -
pscp -scp -r -P 22 ./chainstate ./blocks bitcoin@<IP_ADDRESS>:/mnt/hdd/bitcoin
I also installed putty, winscp and pscp.exe to get to the above solution (you may not need to install winscp)
This will take a few hours to complete.

@maaxxyy Good that is worked. Please clos this issue.


Upgrade Ruby to 2.7 and embedded chef-client to 16.13

Get on a currently supported release of chef-client within chef-server-ctl

This is now passing builds and ready for review @PrajaktaPurohit


New 3D printed mount for the camera

This is actually undergoing, I'm trying to modify this project https://www.thingiverse.com/thing:1579789

closed by https://github.com/mmajewsk/Tonic/pull/20


Add --status-line=<absolute|relative|none>

Users might want to see the relative path on the status instead of the
absolute path. This option allows them to show the relative path.
In addition, I added another option value "none" to hide the status-line
itself.
Honestly, this option is intended for testing use, but I hope this would
be convenient.

The issues by actionlint seem to be no longer problems. I just want to ignore them.
https://github.com/rhysd/actionlint/issues/22


Documentation on list comparison needs clarification

Docs here:
https://docs.rosetta-technology.io/rosetta/rosetta-dsl/rosetta-modelling-component/#list-comparison

contains description is incorrect (or unclear)
state that contains is equivalent to any = as this has caused confusion
the statement below also applies to disjoint

If the contains operator is passed an expression that has single cardinality, that expression is treated as a list containing the single element or an empty list if the element is null.

state that for all or any comparisons where one has a single cardinality, the single cardinality value must be on the right hand side

Note that due to rename from Rosetta DSL to Rune DSL, the url above has changed to https://docs.rosetta-technology.io/rosetta/rosetta-dsl/rune-modelling-component/#list-comparison


add synonyms to GO:1905368 peptidase complex

Please add synonyms:
protease complex [EXACT]
(cf GO:0008233 peptidase activity - protease activity [EXACT])
Tryptase complex [NARROW]

unless you think it should be a child term. But it doesn't differ in function from its parent. PMID:29661938
It would just be nice to have a search result for tryptases :)

Thanks,
Birgit

done.