Implementing a Velcro-sticking arrangement using the adhesion actuator.

Hello, I am trying to develop a simulation of 3 link shape changing robots using Mujoco. In the experiments the robots have Velcro patches on their arms which bind in a certain configuration only i.e. M-F and not vice versa. I have implemented this in my model using patches belonging to specific groups 1 and 2 as a proxy for the M and F parts of the Velcro and colored them B and G. However since these patches are children of the arms, when I try to declare the adhesion actuator later on in my code I am having to use the parent body which are the arms for the adhesion actuator. I try to manually enforce this constraint by implementing an adhesion function which  assigns an adhesion force to a pair of bodies coming in contact if they belong to groups 1 and 2 respectively, however since the whole robot arm is declared as the adhesion body it causes the arms to stick in unnatural ways. I am attaching my simulation file and the video I generated when  implementing this algorithm.
https://github.com/google-deepmind/mujoco/assets/48886152/6c532332-63a6-4dfb-bcf2-e47f3490fad9
Here is my code.. Couldn't attach the file hence pasting it out.
import mujoco as mj
from mujoco.glfw import glfw
from time import sleep
import numpy as np
import random
import matplotlib.pyplot as plt
Base XML configuration for a single robot with planes on the arms
modified_base_robot_xml = '''

     <!-- Arm 2 -->
    <body name="{robot_name}_arm_2" pos="0.07 0 0.0">
        <joint name="{robot_name}_hinge2" type="hinge" axis="0 -1 0" pos="-0.03 0 0.0" range="0 1.5708" damping="0.25" frictionloss="0.1" stiffness="0.25"/>
        <geom type="box" size="0.03 0.01 0.0025" mass="0.05" rgba="0 0 0 1" friction="1.0" margin="0.0001"/>
        <!-- Blue and Green Pads on one Face of Arm 2 -->
        <geom name="{robot_name}_pad1_arm2" type="box" size="0.02 0.002 0.00001" pos="0.0 0.004 0.0025" rgba="0.0 0.0 0.5 1.0" group="1" friction="1.0" margin="0.0001"/>
        <geom name="{robot_name}_pad2_arm2" type="box" size="0.02 0.002 0.00001" pos="0.0 -0.004 0.0025" rgba="0.2 0.8 0.2 1.0" group="2" friction="1.0" margin="0.0001"/>
        <!-- Blue and Green Pads on other Face of Arm 2 -->
        <geom name="{robot_name}_pad3_arm2" type="box" size="0.02 0.002 0.00001" pos="0.0 0.004 -0.0025" rgba="0.2 0.8 0.2 1.0" group="2" friction="1.0" margin="0.0001"/>
        <geom name="{robot_name}_pad4_arm2" type="box" size="0.02 0.002 0.00001" pos="0.0 -0.004 -0.0025" rgba="0.0 0.0 0.5 1.0" group="1" friction="1.0" margin="0.0001"/>
    </body>
</body>

'''
simend = 600  # Simulation time (15 seconds)
relaxation_time = 1  # Relaxation time (2 seconds)
print_camera_config = 0  # Set to 1 to print camera config
Initialize simulation parameters
num_robots = 10
x_range = (-0.15, 0.15)
y_range = (-0.15, 0.15)
robot_positions = []
gain = 8
ctrlrange = [0, 1]
For callback functions
button_left = False
button_middle = False
button_right = False
lastx = 0
lasty = 0
def init_controller(model, data):
# Initialize the controller here. This function is called once, at the beginning
pass
def apply_adhesion(model, data, gain, ctrlrange):
print(f"data.xfrc_applied.shape: {data.xfrc_applied.shape}")
# Iterate over all contacts
for contact in data.contact:
    geom1 = contact.geom1
    geom2 = contact.geom2

    # Check if both geometries are part of the adhesion groups
    if model.geom_group[geom1] in (1, 2) and model.geom_group[geom2] in (1, 2):
        distance = np.linalg.norm(data.geom_xpos[geom1] - data.geom_xpos[geom2])
        print(f"distance between the pads is {distance}")

        if distance < 0.001:  # Threshold distance for adhesion
            print(f"Applying adhesion force between geom {geom1} and geom {geom2}")

            # Calculate the adhesion force based on the distance and gain
            force_magnitude = gain * (ctrlrange[1] - ctrlrange[0])
            force_direction = (data.geom_xpos[geom1] - data.geom_xpos[geom2]) / distance
            force = force_magnitude * force_direction
            print(f"Force: {force}")

            if geom1 < data.xfrc_applied.shape[0] and geom2 < data.xfrc_applied.shape[0]:
                data.xfrc_applied[geom1, :3] += force[:3]
                data.xfrc_applied[geom1, 3:] += force[3:]
                data.xfrc_applied[geom2, :3] -= force[:3]
                data.xfrc_applied[geom2, 3:] -= force[3:]
            else:
                print(f"Index {geom1} or {geom2} out of bounds for data.xfrc_applied with shape {data.xfrc_applied.shape}")

def controller(model, data):
# Calculate the desired position for the arms based on the current time
# Allow some relaxation time before starting the controller
if data.time < relaxation_time:
    return

cycle_time = 1 # Total cycle time in seconds
t = data.time % cycle_time  # Current time within the cycle
angle = (1.0*np.pi / 2) * np.sin(2 * np.pi * t / cycle_time)

# Set the desired position for all robots
for i in range(len(data.ctrl)//2):
    #print("i is",i)
    #sleep(1)
    data.ctrl[2*i] = angle  # Hinge joint for Arm 1
    data.ctrl[2*i + 1] = angle  # Hinge joint for Arm 2

#apply_adhesion(model,data)

def keyboard(window, key, scancode, act, mods):
if act == glfw.PRESS and key == glfw.KEY_BACKSPACE:
mj.mj_resetData(model, data)
mj.mj_forward(model, data)
def keyboard(window, key, scancode, act, mods):
if act == glfw.PRESS and key == glfw.KEY_BACKSPACE:
mj.mj_resetData(model, data)
mj.mj_forward(model, data)
Function to generate a random quaternion with a fixed 90 degrees rotation about the x-axis
def euler_to_quaternion(roll, pitch, yaw):
"""
Convert Euler angles to quaternion.
Roll (x-axis rotation), Pitch (y-axis rotation), Yaw (z-axis rotation)
"""
qx = np.sin(roll/2) * np.cos(pitch/2) * np.cos(yaw/2) - np.cos(roll/2) * np.sin(pitch/2) * np.sin(yaw/2)
qy = np.cos(roll/2) * np.sin(pitch/2) * np.cos(yaw/2) + np.sin(roll/2) * np.cos(pitch/2) * np.sin(yaw/2)
qz = np.cos(roll/2) * np.cos(pitch/2) * np.sin(yaw/2) - np.sin(roll/2) * np.sin(pitch/2) * np.cos(yaw/2)
qw = np.cos(roll/2) * np.cos(pitch/2) * np.cos(yaw/2) + np.sin(roll/2) * np.sin(pitch/2) * np.sin(yaw/2)
return qw, qx, qy, qz
def random_quaternion():
"""
Generate a random quaternion with a fixed 90 degrees rotation about the x-axis
and random rotations about the y and z axes.
"""
roll = np.pi / 2  # 90 degrees about the x-axis
pitch = random.uniform(0, 2 * np.pi)  # Random angle about the y-axis
yaw = random.uniform(0, 2 * np.pi)  # Random angle about the z-axis
# Convert Euler angles to quaternion
return euler_to_quaternion(roll, pitch, yaw)

Function to initialize multiple robots in the simulation
def initialize_robots(num_robots, x_range, y_range):
robots = []
for i in range(num_robots):
x_pos = random.uniform(*x_range)
y_pos = random.uniform(*y_range)
#quat_w, quat_x, quat_y, quat_z = random_quaternion()
quat = random_quaternion()
robot_name = f"robot_{i}"
#robot_xml = modified_base_robot_xml.format(robot_name=robot_name, x_pos=x_pos, y_pos=y_pos,
#quat_w=quat_w, quat_x=quat_x, quat_y=quat_y, quat_z=quat_z)
robot_xml = modified_base_robot_xml.format(robot_name=robot_name, x_pos=x_pos, y_pos=y_pos,
quat_w=quat[0], quat_x=quat[1], quat_y=quat[2], quat_z=quat[3])
robots.append(robot_xml)
return ''.join(robots)
Function to save robot positions and orientations
def save_robot_positions(data):
global robot_positions
pos = data.qpos[:3]
quat = data.qpos[3:7]
robot_positions.append((pos.copy(), quat.copy()))
Function to plot robot positions and orientations
def plot_robot_positions():
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
for pos, quat in robot_positions:
ax.scatter(pos[0], pos[1], pos[2])
#ax.quiver(pos[0], pos[1], pos[2], quat[1], quat[2], quat[3], length=0.1)
plt.show()
def mouse_button(window, button, act, mods):
global button_left, button_middle, button_right
button_left = (glfw.get_mouse_button(window, glfw.MOUSE_BUTTON_LEFT) == glfw.PRESS)
button_middle = (glfw.get_mouse_button(window, glfw.MOUSE_BUTTON_MIDDLE) == glfw.PRESS)
button_right = (glfw.get_mouse_button(window, glfw.MOUSE_BUTTON_RIGHT) == glfw.PRESS)
glfw.get_cursor_pos(window)
def mouse_move(window, xpos, ypos):
global lastx, lasty, button_left, button_middle, button_right
dx = xpos - lastx
dy = ypos - lasty
lastx = xpos
lasty = ypos
if not button_left and not button_middle and not button_right:
    return

width, height = glfw.get_window_size(window)
mod_shift = (glfw.get_key(window, glfw.KEY_LEFT_SHIFT) == glfw.PRESS or
             glfw.get_key(window, glfw.KEY_RIGHT_SHIFT) == glfw.PRESS)

if button_right:
    action = mj.mjtMouse.mjMOUSE_MOVE_H if mod_shift else mj.mjtMouse.mjMOUSE_MOVE_V
elif button_left:
    action = mj.mjtMouse.mjMOUSE_ROTATE_H if mod_shift else mj.mjtMouse.mjMOUSE_ROTATE_V
else:
    action = mj.mjtMouse.mjMOUSE_ZOOM

mj.mjv_moveCamera(model, action, dx / height, dy / height, scene, cam)

def scroll(window, xoffset, yoffset):
mj.mjv_moveCamera(model, mj.mjtMouse.mjMOUSE_ZOOM, 0.0, -0.05 * yoffset, scene, cam)
Initialize MuJoCo model
robots_xml = initialize_robots(num_robots, x_range, y_range)
combined_xml = f'''

<default>
    <joint damping=".3" axis="0 1 0"/>
    <geom type="box" friction=".5"/>
    <default class="active_adhesion">
        <geom rgba="0.0 0.0 0.5 1" margin=".0001" gap=".0001"/>
    </default>
</default>

<worldbody>
    {robots_xml}
    <!-- Ground -->
    <geom name="floor" type="plane" size="1 1 0.1" rgba="0.9 0.9 0.9 1"/>
</worldbody>
<actuator>
    {''.join([f'<position joint="robot_{i}_hinge1" ctrlrange="0 1.5708" kp="5"/>' for i in range(num_robots)])}
    {''.join([f'<position joint="robot_{i}_hinge2" ctrlrange="0 1.5708" kp="5"/>' for i in range(num_robots)])}
    {''.join([f'<adhesion name="adhere_robot_{i}_arm1" body="robot_{i}_arm_1" ctrlrange="0 1" gain="8"/>' for i in range(num_robots)])}
    {''.join([f'<adhesion name="adhere_robot_{i}_arm2" body="robot_{i}_arm_2" ctrlrange="0 1" gain="8"/>' for i in range(num_robots)])}
</actuator>

'''
with open('multi_robot.xml', 'w') as file:
file.write(combined_xml)
Load the MuJoCo model from the XML string
model = mj.MjModel.from_xml_string(combined_xml)
data = mj.MjData(model)
cam = mj.MjvCamera()  # Abstract camera
opt = mj.MjvOption()  # Visualization options
Initialize visualization
glfw.init()
window = glfw.create_window(1200, 900, "MuJoCo Simulation", None, None)
glfw.make_context_current(window)
glfw.swap_interval(1)
Initialize visualization data structures
scene = mj.MjvScene(model, maxgeom=10000)
context = mj.MjrContext(model, mj.mjtFontScale.mjFONTSCALE_150)
Install GLFW mouse and keyboard callbacks
glfw.set_key_callback(window, keyboard)
glfw.set_cursor_pos_callback(window, mouse_move)
glfw.set_mouse_button_callback(window, mouse_button)
glfw.set_scroll_callback(window, scroll)
Initialize the controller
init_controller(model, data)
Set the controller
mj.set_mjcb_control(controller)
apply_adhesion(model, data, gain, ctrlrange)
Simulation loop
while not glfw.window_should_close(window):
time_prev = data.time
while data.time - time_prev < 0.00002:
    mj.mj_step(model, data)

if data.time >= simend:
    break

# Save robot positions at each step
save_robot_positions(data)

# Get framebuffer viewport
viewport_width, viewport_height = glfw.get_framebuffer_size(window)
viewport = mj.MjrRect(0, 0, viewport_width, viewport_height)

# Print camera configuration (help to initialize the view)
if print_camera_config == 1:
    print('cam.azimuth =', cam.azimuth, ';', 'cam.elevation =', cam.elevation, ';', 'cam.distance =', cam.distance)
    print('cam.lookat = np.array([', cam.lookat[0], ',', cam.lookat[1], ',', cam.lookat[2], '])')

# Update scene and render
mj.mjv_updateScene(model, data, opt, None, cam, mj.mjtCatBit.mjCAT_ALL, scene)
mj.mjr_render(viewport, scene, context)

# Swap OpenGL buffers (blocking call due to v-sync)
glfw.swap_buffers(window)

# Process pending GUI events, call GLFW callbacks
glfw.poll_events()

Plot the robot positions after the simulation ends
plot_robot_positions()
Terminate GLFW
glfw.terminate()

This didn't get pasted properly....
modified_base_robot_xml = '''

     <!-- Arm 2 -->
    <body name="{robot_name}_arm_2" pos="0.07 0 0.0">
        <joint name="{robot_name}_hinge2" type="hinge" axis="0 -1 0" pos="-0.03 0 0.0" range="0 1.5708" damping="0.25" frictionloss="0.1" stiffness="0.25"/>
        <geom type="box" size="0.03 0.01 0.0025" mass="0.05" rgba="0 0 0 1" friction="1.0" margin="0.0001"/>
        <!-- Blue and Green Pads on one Face of Arm 2 -->
        <geom name="{robot_name}_pad1_arm2" type="box" size="0.02 0.002 0.00001" pos="0.0 0.004 0.0025" rgba="0.0 0.0 0.5 1.0" group="1" friction="1.0" margin="0.0001"/>
        <geom name="{robot_name}_pad2_arm2" type="box" size="0.02 0.002 0.00001" pos="0.0 -0.004 0.0025" rgba="0.2 0.8 0.2 1.0" group="2" friction="1.0" margin="0.0001"/>
        <!-- Blue and Green Pads on other Face of Arm 2 -->
        <geom name="{robot_name}_pad3_arm2" type="box" size="0.02 0.002 0.00001" pos="0.0 0.004 -0.0025" rgba="0.2 0.8 0.2 1.0" group="2" friction="1.0" margin="0.0001"/>
        <geom name="{robot_name}_pad4_arm2" type="box" size="0.02 0.002 0.00001" pos="0.0 -0.004 -0.0025" rgba="0.0 0.0 0.5 1.0" group="1" friction="1.0" margin="0.0001"/>
    </body>
</body>

'''

I didn't quite follow the question, and the video was not helpful, but I think there is something you might be missing. Yes, adhesion applies to all geoms in a body, but you can wrap some subset with a static body (no joints).
So instead of
<body>
  <geom name="1"/>
  <geom name="2"/>
</body>

where I only want geom 2 to be adhesive but I get unwanted adhesion from geom 1, I can do
<body>
  <geom name="1"/>
  <body name="static_child">
    <geom name="2"/>
  </body>
</body>

and connect the adhesion actuator to the static_child body.

Hi, thank you for your reply. I realize my question could have been clearer and more precise. I want to implement adhesion between 2 specific groups of objects. These objects will be children to the robot arms (in my case) and I assigned them groups 1 and 2. I can follow your advice and make them static bodies and then connect the adhesion actuator to them in part where I generate the combined xml's for the robots. How would I be able to ensure that the adhesion between them is directional, which is to say adhesion happens only when a static body belonging to group 1 comes in contact with a static body belonging to group 2? Currently I have a function in my code which does this, where I check for the group identity of a pair of colliding bodies and assign them a force based on the specified gain and ctrl-range. I am not sure if this is the right way.

In the current design, if body 1 has an adhesion actuator then if body 2 can contact body 1, adhesion force can be generated between them. So given the current design you need a workaround similar to what you are doing.
Basically what you describe is a feature request. I doubt it will happen since this is a pretty rare use case.
However, it is very very easy to just modify the related code and rebuild from source, if you don't mind a bit of hacking. See this block here?
// irrelevant contact, continue
if (b1 != id && b2 != id) {
  continue;
}

just add similar thing just after like
// geom ids
int g1 = con->geom[0];
int g2 = con->geom[1];

// contact not between "velcro" geoms, continue
if ((m->geom_group[g1] != 1 && m->geom_group[g2] != 2) ||
     m->geom_group[g2] != 1 && m->geom_group[g1] != 2)) {
  continue;
}

(or whatever other condition you want)

Thanks for your help on this. I can try out the solution you suggested and post in the forum to let you guys know how it went. Btw, I work in the same lab as Tingnan Zhang who is a part of your team .


Fix failure when early calling to focus()

Fixes https://github.com/vaadin/vaadin-list-box/issues/13

This change is

bower.json, line 29 at r1 (raw file):

    "polymer": "^2.0.0",
    "vaadin-themable-mixin": "^1.1.3",
    "vaadin-context-menu": "^2.0.0"

Should be a dev dependency

Comments from Reviewable

Reviewed 4 of 4 files at r1.
Review status: all files reviewed at latest revision, 1 unresolved discussion.

Comments from Reviewable

vaadin-list-mixin.html, line 61 at r1 (raw file):

      this.addEventListener('click', e => this._onClick(e));

      Polymer.RenderStatus.afterNextRender(this, () => {

Simpler fix would be to remove this afterNextRender

Comments from Reviewable

bower.json, line 29 at r1 (raw file):
Previously, tomivirkki (Tomi Virkki) wrote…
Should be a dev dependency

Fixed

Comments from Reviewable

vaadin-list-mixin.html, line 61 at r1 (raw file):
Previously, tomivirkki (Tomi Virkki) wrote…
Simpler fix would be to remove this afterNextRender

Fixed

Comments from Reviewable

Reviewed 1 of 4 files at r1, 4 of 4 files at r2.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

Review status: all files reviewed at latest revision, all discussions resolved, some commit checks pending.

Comments from Reviewable


Prevent input tensors from being GC'ed before PInvoke occurs

We were seeing test failures because GC would occur after tensor handles were copied
to IntPtr array but before PInvoke finished.  GC would run the finalizer, which freed
the native tensors, while TF was still running.
Prevent the GC from finalizing these by telling it we're still using the managed object.
There may be more instances of this problem with other types.  The generic pattern to search for
is a manged object with a finalizer that exposes its native state (like TFDisposable), and that native
state is used after the last reference to the managed object in the method and no external
reference is guaranteed.
/cc @migueldeicaza

I started working on a fix using SafeHandles but lost interest after my first PR has been ignored for several months...

Thanks for these changes!


Fixed a WPF issue causing binding errors.

Fixes issue #97.

The code looks good to me, but it might be useful to add a comment referring to issue #97 so people know why its there.

Sorry. Where should I put the comment?

Above <Style TargetType="MenuItem"> in the code, a one line comment referring to the issue.

Added a comment as suggested.


[video_player] Publish fully federation version

Publishes the change from in-package Android and iOS implementations
to federated implementations.
Part of https://github.com/flutter/flutter/issues/68498
Pre-launch Checklist

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I read and followed the relevant style guides and ran the auto-formatter. (Unlike the flutter/flutter repo, the flutter/plugins repo does use dart format.)
[x] I signed the CLA.
[x] The title of the PR starts with the name of the plugin surrounded by square brackets, e.g. [shared_preferences]
[x] I listed at least one issue that this PR fixes in the description above.
[x] I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy, or this PR is exempt from version changes.
[x] I updated CHANGELOG.md to add a description of the change, following repository CHANGELOG style.
[x] I updated/added relevant documentation (doc comments with ///).
[x] I added new tests to check the change I am making, or this PR is test-exempt.
[x] All existing and new tests are passing.

I'll hold off on landing this until any trailing issues from https://github.com/flutter/plugins/pull/4712 are addressed.


[WIP] Remove FailedSync event from EventCatcher

Removing FailedSync as too verbose and not informative enough on large clusters.
cc @agrare @Fryguy

@simon3z is this ready?

@simon3z is this ready?

@agrare no finishing this up right now.

@miq-bot rm_label wip

@agrare now it's ready (unless the new line I removed is problematic).

Fine backport details:
$ git log -1
commit b7d89634a855fa5b133379a1833f38ab7d3714cc
Author: Adam Grare<EMAIL_ADDRESS>Date:   Tue Apr 11 11:00:28 2017 -0400

    Merge pull request #14633 from simon3z/failed-sync

    Remove FailedSync event from EventCatcher
    (cherry picked from commit e540ef425c3f40bf1df0cbe2fed3fba23979790a)

BZ https://bugzilla.redhat.com/show_bug.cgi?id=1441765

Euwe backport details:
$ git log -1
commit 2d70ea4d168f826eb60076872587f375a525438e
Author: Adam Grare<EMAIL_ADDRESS>Date:   Tue Apr 11 11:00:28 2017 -0400

    Merge pull request #14633 from simon3z/failed-sync

    Remove FailedSync event from EventCatcher
    (cherry picked from commit e540ef425c3f40bf1df0cbe2fed3fba23979790a)

    https://bugzilla.redhat.com/show_bug.cgi?id=1441855

Followup: Now policies with FailedSync event silently don't work.  Should remove the event , not sure if possible without migration.
https://bugzilla.redhat.com/show_bug.cgi?id=1448811


Updates @xmldom/xmldom package version from 0.8.3 to 0.8.4

This PR bumps the @xmldom/xmldom package that is used in Azure Data Studio to parse xml DOM's. This PR patches an issue around document object models that contain multiple root elements.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.005%) to 41.968%

Totals

Change from base Build<PHONE_NUMBER>:
0.005%

Covered Lines:
28919

Relevant Lines:
64014

💛  - Coveralls


Shellcheck scripts directory

Shellcheck is a linting tool for shell scripts:

So the proposed changes would be:
#!/bin/sh

set -u

TARFILE=$1

echo "Backing up database to $PWD/$TARFILE ..."

# don't let tar errors or warnings bubble up
exit 0

Shellcheck has also a CLI. I wonder whether you'll find it helpful. If so, I'll update the scripts.

Ah yes, that's great. I know about shellcheck but I always forget to actually run it 🤣
I'll add those changes or if you want to open up a PR that'll be great, too :+1:

Ah yes, that's great. I know about shellcheck but I always forget to actually run it 🤣
I'll add those changes or if you want to open up a PR that'll be great, too 👍

Yeah, I would like to open a PR if you don’t mind :)

Closed by #1135. thanks @elfenlaid!


Add encryptor type CUSTOM

I added encryption-ub module to our GitHub to use the CUSTOM encryptor type, https://github.com/unbound-tech/encryption-ub

Codecov Report

Merging #1094 into master will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##             master    #1094   +/-   ##
=========================================
  Coverage     99.39%   99.39%
  Complexity     2264     2264
=========================================
  Files           350      350
  Lines          6298     6299    +1
  Branches        360      360
=========================================
+ Hits           6260     6261    +1
  Misses            5        5
  Partials         33       33

Flag
Coverage Δ
Complexity Δ

#unittests
99.39% <100.00%> (+<0.01%)
2264.00 <0.00> (ø)

Impacted Files
Coverage Δ
Complexity Δ

.../java/com/quorum/tessera/config/EncryptorType.java
100.00% <100.00%> (ø)
1.00 <0.00> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8ebfce7...0911118. Read the comment docs.


Converting existing models

Amazing work and fantastic resource, thanks for sharing your work - this should jump start usage of llm on low resource devices.
Quick question - is there a guide to convert existing models to bitnet compliant format?!

i tried gemma-2-27b, gemma-2-9B, and many others all worked fine no errors encountered till now, although their 1 bit quants were a lot hallucinating

Unfortunately, no. If a model's weight parameters are not natively ternary, using the conversion function will result in the loss of weight values, leading to inaccurate results. We encourage more training of 1-bit models from scratch.

@Dead-Bytes  By 'tried Gemma-2-27B ' do you mean that you performed QAT from scratch? How did you quantize the Gemma-2 models?

No i did not performed QAT from Scratch, i used the present I_2S quants available for models, they show degrading performance this need to be researched to get a way out of it, however they are without ternary wieghts in 1 bit quants. They are working fine on my octacore cpu, human readable 7 tokens per second.

@dawnmsg Would training a 70B model from scratch with 1-bit precision require fewer resources than training with full precision? If similar resources are needed, would general developers still be able to perform QAT for a 70B model?

I think it should be @ qwen2.5 developers. I hope to get a 1-bit qwen2.5 model, and I estimate they also want it.
I don't know how to train from scratch, and I estimate I won't be able to afford such financial expenses, even if I use the cloud.
We need a foundation

Isn't this what HF1BitLLM/Llama3-8B-1.58-100B-tokens did though? They started with a base model and fine tuned it via this method: https://huggingface.co/blog/1_58_llm_extreme_quantization

This suggests that fine-tuning the model in low-bit mode on a specific dataset causes it to lose much of its general knowledge

While pre-training models in 1.58 bits is resource-intensive, we’ve demonstrated that, with some tricks, it’s possible to fine-tune existing models to this precision level, achieving efficient performance without sacrificing accuracy.

So it can be done, but it sacrifices original model data and you need to top it up with additional datasets, so possible but not perfect.
Training it from scratch would avoid this.

no help

----- 原始邮件 -----
@.***>
@.***>等3人
主题：Re: [microsoft/BitNet] Converting existing models (Issue #40)
日期：2024年11月08日 04:24:53
Isn't this what HF1BitLLM/Llama3-8B-1.58-100B-tokens did though? They started with a base model and fine tuned it via this method: https://huggingface.co/blog/1_58_llm_extreme_quantization
This suggests that fine-tuning the model in low-bit mode on a specific dataset causes it to lose much of its general knowledge
While pre-training models in 1.58 bits is resource-intensive, we’ve demonstrated that, with some tricks, it’s possible to fine-tune existing models to this precision level, achieving efficient performance without sacrificing accuracy.
So it can be done, but it sacrifices original model data and you need to top it up with additional datasets, so possible but not perfect.
Training it from scratch would avoid this.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>

I have just discovered bitnet. Before this, I was using models in gguf format. But they were below my expectations. Now I want to run the models I trained on bitnet. But I don't know how to do it. I currently have a model in gguf format like this: "falan42/llama_lora_8b_medical_parallax_2_gguf"


What about printing or reporting ?

Most,  if not all Enterprise Application will eventually require some type of report or pdf.
There are many "server side" packages out there to do that as well as lots and lots of commercial (paid) components for that reason.
Somehow that “server dependency"  reporting feels ancient and old-fashioned, as if does not belong to the new “front end power” of angular.
2 weeks ago I i was able to
-create a PDF from our data.
-nicely display the PDF on the UI, in our case is inside an Angular Material Tab.
-email it.
-print it.
All done on the front-end, works fine for web as well as on the mobile native apps.
using the following 3 libraries:
https://github.com/bpampuch/pdfmake
https://github.com/sayanee/angularjs-pdf
https://github.com/mozilla/pdf.js
@NathanWalker , Do you think that incorporating printing/reporting this way, will be of value to the advanced seed?

Definitely on a branch @OscarAgreda, very nice. If you have the project up on GitHub anywhere or can share with me, I can help extrapolate the details onto a branch here.

Sounds like a plan, I'll do that .

I'm going to close this @OscarAgreda but feel free to submit a PR to this branch: https://github.com/NathanWalker/angular2-seed-advanced/tree/pdf-reports if you'd like 👍


[Evaluator] Unable to run tool when using metadata file URL or upload as input

Bug description:
The FAIR evaluator tool is unable to calculate FAIR scores when using a metadata file as input.
To reproduce:

Go to the FAIRsoft evaluator app
In Step 1, select 'Metadata file' and 'Enter URL' as the metadata source
Enter the URL: 'https://raw.githubusercontent.com/inab/observatory-api/a825f37ecf6d557c9f7ad892da05ff32cebd35bc/app/helpers/metadata.jsonld'

The same issue occurs when uploading the metadata as a file instead of entering a URL

Unexpected output:
When the metadata source is entered as a URL, the tool hangs on step 3:

When the metadata source is entered as a file, the tool does not retrieve any metadata and is unable to calculate the FAIR scores.

Environment:
Device: Dell laptop (x64)
OS: Windows 11 Pro
Browser: Google Chrome

Apologies for the delay; I completely missed this! I'll review it and get back to you soon.

Thanks so much! To add some further context here -- I also tried downloading the metadata in JSON-LD format through the evaluator tool in the 'Results' step after using a GitHub URL as input (e.g. 'https://github.com/inab/oeb-visualizations'), and then using this file as input into the evaluator in Step 1. When I go to use the tool with this JSON-LD input, I can proceed to Step 3 ('Edit the metadata if needed') as expected but the tool hangs when attempting to evaluate FAIR indicators in Step 4 ('Results').

Hi Abner,
Thank you for your feedback. It appears that the file you were trying to upload is in an older format, which is no longer supported. As part of a major update this summer, we modified the tool to use the maSMP format. Unfortunately, this means that files in the previous format are now incompatible—apologies for any inconvenience this may have caused.
I also resolved a few additional issues, and the tool should now work with files in the maSMP format, whether uploaded directly from your local filesystem or through a URL. If you have any questions or run into further issues, please don’t hesitate to reach out 🙏.


Fsl roi replacement pr

replacing fsl_roi with pure python implementation.
merging into staging for running on AWS for robustness testing

99 commits - why is commit history so big when you're only making the one change?

closing for now until this is actually ready


After npm start > Login > Token issue

Issue
ClientAuthError: request_cannot_be_made: Token request cannot be made without authorization code or refresh token.
    at ClientAuthError.AuthError [as constructor] (/Users/michaelwalker/WebstormProjects/ms-identity-javascript-nodejs-tutorial/2-Authorization/1-call-graph/App/node_modules/@azure/msal-common/dist/index.cjs.js:477:24)

This issue is for the sample

    - [ ] 1-1) Sign-in with Azure AD
    - [ ] 1-2) Sign-in with Azure AD B2C
    - [x ] 2-1) Acquire a Token and call Microsoft Graph
    - [ ]   3) Deploy to Azure Storage and App Service
    - [ ] 4-1) Use App Roles for Role-based Access Control
    - [ ] 4-2) Use Security Groups for Role-based Access Control

This issue is for a

    - [ ] bug report -> please search issues before submitting
    - [ ] question
    - [ ] feature request
    - [ x] documentation issue or request

Minimal steps to reproduce

Input all necessary appSettiings.js things
Go to http://localhost:4000/home
Click Signin > Finish Oauth flow > Error Page

Any log messages given by the failure

See above

Expected/desired behavior

Match screen shot in readme

Library version

Most recent

Browser and version

Chrome, Edge, Firefox, Safari?

Chrome

Mention any other details that might be useful

Thanks! We'll be in touch soon.

Any updates?

Hi @MichaelWalker-git
Can you please share the logs, the request calls, and the appSettings object?


Excess property checks still appear in presence of index signature

interface A {
    a?: string
    b?: number
    [key: symbol]: string
}

const symbolKey = Symbol('key')

const a: A = {
    [symbolKey]: '123',
}

symbol still not working：ts playground
Originally posted by @liuweiGL in https://github.com/microsoft/TypeScript/issues/44512#issuecomment-869415311

A non-breaking fix for now would be to validate symbol-named properties against symbol index signatures, but also fall back to validating them against string index signatures if no symbol index signatures are present.


feat: disable trade preview while custom recipient is editing

Description

Pull Request Type

[ ] :bug: Bug fix (Non-breaking Change: Fixes an issue)
[ ] :hammer_and_wrench: Chore (Non-breaking Change: Doc updates, pkg upgrades, typos, etc..)
[ ] :nail_care: New Feature (Breaking/Non-breaking Change)

Issue (if applicable)

closes #
Risk

High Risk PRs Require 2 approvals

What protocols, transaction types or contract interactions might be affected by this PR?

Testing

Engineering

Operations

Screenshots (if applicable)

Current dependencies on/for this PR:

develop

PR #6132   👈

This stack of pull requests is managed by Graphite.


Django error

django.core.exceptions.ImproperlyConfigured: Cannot import 'jobs'. Check that 'regparser.web.jobs.apps.JobsConfig.name' is correct.
when using cmd version of the app

If you are using Django >= 3.2, there are changes that may cause this issue:
https://docs.djangoproject.com/en/3.2/releases/3.2/#what-s-new-in-django-3-2
We fixed this in our instance of the app by :
in base.py in regulations-parser AND  Eregulations:
DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'

And in regparser/web/jobs/apps.py in regulations-parser put full path to the app:
name = 'regparser.web.jobs'

Hope this helps.


Suggestions to improve SSH-attack recipe

In https://confluentinc.github.io/ksqldb-recipes/cybersecurity/SSH-attack/ I think that we could write a single query instead of two.

@mjsax thanks for the suggestion.  Can you please elaborate: which 2 queries, and how can they be combined?
Source: https://github.com/confluentinc/ksqldb-recipes/blob/master/docs/cybersecurity/SSH-attack/process.sql

There are only two queries:
CREATE STREAM invalid_users AS
SELECT
  [...]
FROM syslog
WHERE message LIKE 'Invalid user%'
EMIT CHANGES;

and
CREATE STREAM ssh_attacks AS
SELECT
  syslog_timestamp,
  host,
  facility_description,
  SPLIT(REPLACE(message, 'Invalid user ', ''), ' from ')[1] AS attack_user,
  remote_address AS attack_ip
FROM invalid_users
EMIT CHANGES;

The second one is just doing an additional projecting that can be done inside the first query directly:
CREATE STREAM ssh_attacks AS
SELECT
  FORMAT_TIMESTAMP(ts, 'yyyy-MM-dd HH:mm:ss') AS syslog_timestamp,
  host,
  facility,
  CASE WHEN facility = 0 THEN 'kernel messages'
       WHEN facility = 1 THEN 'user-level messages'
       WHEN facility = 2 THEN 'mail system'
       WHEN facility = 3 THEN 'system daemons'
       WHEN facility = 4 THEN 'security/authorization messages'
       WHEN facility = 5 THEN 'messages generated internally by syslogd'
       WHEN facility = 6 THEN 'line printer subsystem'
       ELSE '<unknown>'
     END AS facility_description
  SPLIT(REPLACE(message, 'Invalid user ', ''), ' from ')[1] AS attack_user, -- this line
  remote_address  AS attack_ip, -- this line
FROM syslog
WHERE message LIKE 'Invalid user%'
EMIT CHANGES;

Thanks @mjsax .  PR: https://github.com/confluentinc/ksqldb-recipes/pull/96


Not working for Garena server.

It's not working for SEA server. Any suggestions on making it to work?

I'm playing on Garena Server patch 10.24
Still work fine for me


Support arrays/lists

Thinking initially lists of base types, for cases like this:
    "menu_items": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },

Other cases might be: items cooked during a cooking session, items stored in the fridge at 5 degrees
The user might want an option to just render it as a list of strings, a numbered list, or a bulleted list.
So a list of base types seems manageable by my imagination.  But what if it was a list of object, like bicycles, which have make, model, colour etc.  Maybe wait until we have a use case?

done


⚠ Bofrost.de has degraded performance

In 790fee6, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1527 ms

Resolved: Bofrost.de performance has improved in b8f45cd after 6 minutes.


Flushing out TE test

@jasonsandlin,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


[ConstraintSystem] new argument matching algorithm

I created new algorithm of argument matching that produces expected result in all difficult situation we discussed at #32037 #30444.
I am going to write full explanation soon.

@swift-ci please smoke test

Unlabeled matching in stage 2 is similar to what I wrote at #32037.
This new way is processed inside separated group by bindings from labeled matching in stage 1.
So I don't think that cause bad effect.

@swift-ci please smoke test

I explain following case:
func r27212391(x: Int, _ y: Int) { }

r27212391(3, y: 5)
/*
  current:
    3 => x:
    y: => _ y:

  patch:
    3 => none (extra argument)
    y: => x:
    none => _ y:
*/

First, current typo correction considers y is typo of x.
So arguments and parameters are aligned with anchoring y: and x: like below.
func r27212391(x: Int, _ y: Int) { }
//             ^         ^missing
  r27212391(3, y: 5)
//          ^extra

I think typo correction in one character case is too strong.
I will change this but I didn't here.
And I also explain following case:
func f(aa: Int, _ bb: Int) { }

f(0, xx: 1)
/*
  current:

  patch:
    none => aa: (missing argument)
    0 => _ bb:
    xx: => none (extra argument)
*/

In this case, xx: isn't matched aa:.
Only one unlabeled argument and parameter are matched.
Arguments and parameters are aligned with anchoring with them like below.
func f(aa: Int, _ bb: Int) { }
//      ^missing   ^
                 f(0, xx: 1)
//                     ^extra

They are much different from current.
But I think it is consistent rule that keeps left to right ordering aligned with anchor which is from more strong binding.

I added result section in document.

Please update the base branch to main by Oct 5th otherwise the pull request will be closed automatically.

How to change the base branch: (Link)
More detail about the branch update: (Link)


Add resource monitor in CPU tray

Description of the new feature / enhancement
Add resource indiators in system tray similar to indicator-multiload app in ubuntu.

CPU
Memory
Temperature
Disk IO
Network usage

Scenario when this would be used?
Resource indicators can be used to detect/monitor any usually activity.
Supporting information
No response

Duplicate of #8756

Thanks for your suggestion! This issue is a duplicate.
I'm linking this issue to #8756 to centralize the discussion thread, please go to that thread for discussion.
Thanks for your help to make PowerToys a better piece of software!


More substantial documentation

It seems that the help for this tool is limited to --help and man, which is decent for most purposes, but I feel like there could be more substantial documentation out there.
That being said, I'm not particularly one to say "You need X" unless I'd be willing to do it myself. If there's interest in getting the wiki more fleshed out to detail functionality in the man page I can get something done on that in early November.
Is that of interest?

I noticed this issue while crawling the repo for some documentation myself and it certainly feels like either the wiki and/or the README could do with some expansion.
Likewise, I have no problem with adding some bits here myself but some general guidance at the project level of where to put stuff might be a nice starting point.

+1


WABI Missing

The icons look great!

Added - https://github.com/cjdowner/cryptocurrency-icons/commit/1f918b8910627f7d13ac122f641808750b122c30

Thank you for your support. If you’d like to tip me for my time in maintaining and updating this project, it would be very much appreciated.


Added a score tracking feature to the app.

Score is based on the number of letters in a correctly answered rebus multiplied by 10 because.... well why not?
Summary of Changes

Modified Rebus.js component to calculate the number of characters per each correctly answered rebus and multiply it by 10.
Modified components.spec.js.snap test case to include the new snapshot of the Rebus component.

Appreciate any feed back as this was my very first contribution to an open source project.  The most time was spent on dealing with line feeds, code prettification and the testing framework as I'd never used it before.
Thank you to everyone responsible for this project.

Looks like I forgot to push my main.css file
I have done that now.  Do I need to create another pull request?  I'm in uncharted territory :)

Pull Request Test Coverage Report for Build 912

5 of 5 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.2%) to 87.368%

Totals

Change from base Build 514:
0.2%

Covered Lines:
163

Relevant Lines:
185

💛  - Coveralls


Usability: PathString.StartsWithSegments() with a trailing slash is not intuitive behavior

I was trying to do effectively this yesterday:
new PathString("/profiler/test").StartsWithSegments(new Path("/profiler/");

This was, to me, unexpectedly false.  Once I was in source, I finally realized that a starts-with check of "/profiler" works, while including a trailing slash of "/profiler/" does not. Is this intended behavior? The PathString tests don't actually include a case testing it either way.
Current behavior examples:
	new PathString("/a/b").StartsWithSegments(new PathString("/a/"));  // false
	new PathString("/a/b").StartsWithSegments(new PathString("/a"));   // true
	new PathString("/a//b").StartsWithSegments(new PathString("/a/")); // true

Currently the check of value1[value2.Length] == '/' means whatever's checked against must match or have a trailing slash after it, unless it matches exactly. It seems to be that a /path/ should work here. Is there any objection to this working?
The check could change from:
value1.Length == value2.Length || value1[value2.Length] == '/'

to:
value1.Length == value2.Length || value1[value2.Length] == '/' || value2[value2.Length-1] == '/'

and in the remainder methods, it'd be slightly different with an additional if instead:
if (value2[value2.Length - 1] == '/')
{
    remaining = new PathString(value1.Substring(value2.Length - 1));
    return true;
}

Thoughts? Is there a reason it doesn't behave like this already, or just an ambiguous case? I'd be happy to submit a PR if it's a welcomed change.

Fuzzy matching like this creates ambiguity that leads to subtle bugs. We may be able to do this, but I'd want a lot of test cases. Give it a try.


NTR 'intermembrane lipopolysaccharide transporter activity'

Required to annotate proteins form the bacterial LptBCAD system, see
https://github.com/geneontology/go-annotation/issues/2338
Reference: PMID:24639492

Changed label 'intermembrane lipopolysaccharide transporter activity': https://github.com/geneontology/go-ontology/pull/17322


Fix circularize medaka missing contigs file

Potentially fixes issue https://github.com/rotary-genomics/rotary/issues/193

@jmtsuji When is medaka ran during the circularization process? Is it only ran on the circular contigs? Since the circular contig list only contains the circular contigs names it is identical to the contig manifest that medaka needs. So I take a copy and place that where the snakemake code is looking.


feat: force new release as previous failed to create docker image

Description
GH Release Action workflows have been failing when trying to release the Docker image.
Latest released Docker image is 0.3.4. See https://hub.docker.com/r/asyncapi/server-api/tags
Latest released GH tag is 0.13.10.
After latest merged changes, https://github.com/asyncapi/server-api/pull/352, the release should work as expected.
This PR forces a new release to happen so Docker image is published properly.
cc @BOLT04 @magicmatatjahu

/rtm


jmeter performance test slower

I tested according to the examples in the seata-sample and got such results

The throughput without globaltransaction annotation is 12 times higher than with annotations.There are also some errors when adding annotation.

I used the following code for the test.
https://github.com/seata/seata-samples/tree/master/seata-spring-boot-starter-samples
Ⅵ. Environment:

JDK 1.8:
Seata 1.4.2:
The same goes for seata version 2.0.0

如果你了解两阶段都增加了多少次网络和磁盘io开销,那么你对这个结果就不会太过意外了?其中包括了seata-server与数据库,client与seata-server,一阶段和二阶段决议.我建议你使用store.mode为file模式进行测试,这样的话可以排除掉seata-server数据库的网络io和磁盘io的开销. 这段话翻译成英文
If you understand how much additional network and disk IO overhead was added in both phases, you won't be too surprised by this result. This includes interactions between the Seata server and the database, the client and the Seata server, as well as the first and second phase resolution. I suggest conducting tests using store.mode set to file mode; this can eliminate the network and disk IO overhead of the Seata server database.


Enable viewing methods full documentation in the online documentation of CUQIpy

Methods in the documentation is not clickable and their full documentation cannot be viewed. e.g. plot_ci in Samples documentation. The docstring for plot_ci has description of the parameters that does not show in the documentation.

🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉

😃👍


Extension “Volumes Backup & Share” fails with “Internal Server Error HTTP status code: 500”

Issue type: Unable to backup volume using extension “Volumes Backup & Share”
OS Version / Build: Windows 11 Version 22H2 (Build: 22621.674)
App version: v.4.13.1
Steps to reproduce:
“Volumes Backup & Share” > “Export volume” > “To: Local File” > “.tar.zst” > “Directory” > “Select directory” > “” returns
Failed to backup volume to C:\Users\Desktop: {“message”:“Internal Server Error”}. HTTP status code: 500
Diagnostic ID:
6FD65E53-DD8F-44DB-A774-357A2EE71651/20221102203305
Hello,
Trying to export a volume using the Docker extension “Volumes Backup & Share” but it fails regardless of destination:
C:
C:\Users\Desktop
D:
^ External USB
with the error:
Failed to backup volume to C:\Users\Desktop: {“message”:“Internal Server Error”}. HTTP status code: 500
The container the volume belongs to isn’t started.
Would appreciate any and all help!
Best Regards - TheSwede86

Hi @TheSwede86, thank you for reporting this issue.
Could you attach a text file with the logs of the extension container? (i.e. docker logs docker_volumes-backup-extension-desktop-extension-service)
If you don't see that container, enable the Extension systems containers option in Preferences > Extensions.

After some investigation, I think this may be the error you're seeing:
errdefs.errInvalidParameter Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/run/desktop/mnt/host/c/openstreetmap-tile-server/sweden-latest/sweden-latest.osm.pbf" to rootfs at "/data/region.osm.pbf": mount /run/desktop/mnt/host/c/openstreetmap-tile-server/sweden-latest/sweden-latest.osm.pbf:/data/region.osm.pbf (via /proc/self/fd/14), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Could you confirm it?

Thanks for your reply and advice!
I changed the path from:
C:\openstreetmap-tile-server\sweden-latest\sweden-latest.osm.pbf
to
C:\openstreetmap-tile-server\sweden-latest.osm.pbf
but before that I deleted the volume "osm-data", the container and the image and re-ran everything and it worked now!
Thanks again, I'll close


Add error context to coverage to support debugging

Use the Context trait to add e.g. file names to fallible IO operations in the coverage module, particularly around module and symbol search operations.
AB#39994528

This code was rewritten, no longer relevant.


Generate forecasts for the future time stamps

I am trying to use the model on a pandas dataframe to generate forecast for future time stamps that the true values for are unknown. The provided examples focus on splitting a given dataframe into train/test and generate forecast for the test set. Can you please provide more examples on how to use it to generate forecast for new time stamps that are not present in the pandas dataframe?
Thanks for the great work!

Hi @Mo-Abdelhameed thanks for using our model, and sorry for the delayed response. Here's an example on generating forecasts for a new time stamp:
import torch
import pandas as pd
from gluonts.dataset.pandas import PandasDataset
from huggingface_hub import hf_hub_download

from uni2ts.model.moirai import MoiraiForecast

SIZE = "small"  # model size: choose from {'small', 'base', 'large'}
PDT = 20  # prediction length: any positive integer
CTX = 200  # context length: any positive integer
PSZ = "auto"  # patch size: choose from {"auto", 8, 16, 32, 64, 128}
BSZ = 32  # batch size: any positive integer
TEST = 100  # test set length: any positive integer

# Read data into pandas DataFrame
url = (
    "https://gist.githubusercontent.com/rsnirwan/c8c8654a98350fadd229b00167174ec4"
    "/raw/a42101c7786d4bc7695228a0f2c8cea41340e18f/ts_wide.csv"
)
df = pd.read_csv(url, index_col=0, parse_dates=True)

# Convert into GluonTS dataset
ds = PandasDataset(dict(df))

# Prepare pre-trained model by downloading model weights from huggingface hub
model = MoiraiForecast.load_from_checkpoint(
    checkpoint_path=hf_hub_download(
        repo_id=f"Salesforce/moirai-1.0-R-{SIZE}", filename="model.ckpt"
    ),
    prediction_length=PDT,
    context_length=CTX,
    patch_size=PSZ,
    num_samples=100,
    target_dim=1,
    feat_dynamic_real_dim=ds.num_feat_dynamic_real,
    past_feat_dynamic_real_dim=ds.num_past_feat_dynamic_real,
    map_location="cuda:0" if torch.cuda.is_available() else "cpu",
)

predictor = model.create_predictor(batch_size=BSZ)
forecasts = predictor.predict(ds)
forecast_it = iter(forecasts)
forecast = next(forecast_it)
forecast.plot()

Thanks for the reply!
When I run the code as it is, I get the following error:
Exception ignored in: <generator object PyTorchPredictor.predict at 0x16f886ac0>
Traceback (most recent call last):
File "/Users/moo/Desktop/ReadyTensor/Models/Forecasting/moirai/.venv/lib/python3.11/site-packages/gluonts/torch/model/predictor.py", line 89, in predict
File "/Users/moo/Desktop/ReadyTensor/Models/Forecasting/moirai/.venv/lib/python3.11/site-packages/torch/autograd/grad_mode.py", line 84, in exit
TypeError: 'NoneType' object is not callable
do you have an idea on how to solve it?

@Mo-Abdelhameed @jboornlp are you using windows?

@gorold No, i'm on mac

I can't reproduce this error.

Were you able to run the notebook examples previously?

I'm on a Linux AMI - this is the first thing I'm trying to run - just a copy of the "getting started" code.  The environment seemed to install fine.

Actually when I run the script in a notebook it runs fine

I copied the whole script in one cell, but I still get the same error when running the same code in .py file

Closing this issue since the original problem has been solved. See #3 for further discussion on the error.

@gorold  Thanks!


Improve commandline argument processing in service_config.py

Command-line argument processing in service_config.py could be improved using something like python's argparse module. Documentation and user experience of the chroma-config tool is sub-optimal could be improved with little effort.

This change is

I think docopt would be a better fit here. That said this is a really low priority task that is subject to removal / large changes.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?


Possibility to reduce memory consumption

Hello. I'm from the Kopia project.  Kopia has a benchmark command that given an input file, we run all the compressors on it and report metrics, such as compression ratio, throughput and memory consumption. pgzip seems to have huge stats comparing to, say s2.
The memory consumption stats is calculated by calling runtime.ReadMemStats() before and after the compression loop, then compare the delta. Note that this is not about memory leak, just allocation.

Baseline: compressing a 400MB highly compressible file just once. All compressors behave similarly
Repeating 1 times per compression method (total 466.7 MiB).

     Compression                Compressed   Throughput   Memory Usage
------------------------------------------------------------------------------------------------
  0. s2-default                 127.1 MiB    4 GiB/s      3126   375.4 MiB
  1. s2-better                  120.1 MiB    3.4 GiB/s    2999   351.7 MiB
  2. s2-parallel-8              127.1 MiB    2.8 GiB/s    2981   362.2 MiB
  3. s2-parallel-4              127.1 MiB    2.3 GiB/s    2951   344.1 MiB
  4. pgzip-best-speed           96.7 MiB     2.1 GiB/s    4127   324.1 MiB
  5. pgzip                      86.3 MiB     1.2 GiB/s    4132   298.7 MiB
  6. lz4                        131.8 MiB    458.9 MiB/s  17     321.7 MiB
  7. zstd-fastest               79.8 MiB     356.2 MiB/s  22503  246 MiB
  8. zstd                       76.8 MiB     323.7 MiB/s  22605  237.8 MiB
  9. deflate-best-speed         96.7 MiB     220.8 MiB/s  45     310.8 MiB
 10. gzip-best-speed            94.9 MiB     165 MiB/s    40     305.2 MiB
 11. deflate-default            86.3 MiB     143.1 MiB/s  34     311 MiB
 12. zstd-better-compression    74.2 MiB     104 MiB/s    22496  251.4 MiB
 13. pgzip-best-compression     83 MiB       55.9 MiB/s   4359   299.1 MiB
 14. gzip                       83.6 MiB     40.5 MiB/s   69     304.8 MiB
 15. zstd-best-compression      68.9 MiB     19.2 MiB/s   22669  303.4 MiB
 16. deflate-best-compression   83 MiB       5.6 MiB/s    134    311 MiB
 17. gzip-best-compression      83 MiB       5.1 MiB/s    137    304.8 MiB

Compressing the first 128KB of the same file but repeat 10 times, you can see the higher memory consumption of pgzip among compressors
Repeating 10 times per compression method (total 1.2 MiB).

     Compression                Compressed   Throughput   Memory Usage
------------------------------------------------------------------------------------------------
  0. s2-default                 43.6 KiB     625.3 MiB/s  71     2.1 MiB
  1. s2-parallel-4              43.6 KiB     625.3 MiB/s  67     2.1 MiB
  2. s2-parallel-8              43.6 KiB     624.5 MiB/s  67     2.1 MiB
  3. s2-better                  41.3 KiB     416.8 MiB/s  72     2.1 MiB
  4. deflate-best-speed         34.3 KiB     208.3 MiB/s  22     874.6 KiB
  5. zstd-fastest               28.6 KiB     178.6 MiB/s  160    9.4 MiB
  6. lz4                        44.7 KiB     178.5 MiB/s  38     88.6 MiB
  7. gzip-best-speed            33.7 KiB     138.9 MiB/s  28     1.2 MiB
  8. deflate-default            31.2 KiB     125 MiB/s    22     1.1 MiB
  9. zstd                       26.8 KiB     113.6 MiB/s  174    18.4 MiB
 10. pgzip-best-speed           34.3 KiB     113.6 MiB/s  252    27.3 MiB
 11. zstd-better-compression    26.3 KiB     96.2 MiB/s   156    37.2 MiB
 12. pgzip                      31.2 KiB     74.5 MiB/s   342    31.7 MiB
 13. gzip                       30.4 KiB     39.1 MiB/s   26     874.7 KiB
 14. deflate-best-compression   30.4 KiB     25.5 MiB/s   21     1 MiB
 15. gzip-best-compression      30.4 KiB     24 MiB/s     26     874.7 KiB
 16. pgzip-best-compression     30.4 KiB     23.2 MiB/s   285    30.2 MiB
 17. zstd-best-compression      25.1 KiB     16.9 MiB/s   155    99.2 MiB

Repeating 100 times. s2 has exactly same stats, while pgzip grows accordingly
Repeating 100 times per compression method (total 12.5 MiB).

     Compression                Compressed   Throughput   Memory Usage
------------------------------------------------------------------------------------------------
  0. s2-parallel-4              43.6 KiB     833.4 MiB/s  533    2.1 MiB
  1. s2-parallel-8              43.6 KiB     833.3 MiB/s  555    2.1 MiB
  2. s2-default                 43.6 KiB     833.3 MiB/s  579    2.1 MiB
  3. s2-better                  41.3 KiB     500 MiB/s    610    2.1 MiB
  4. zstd-fastest               28.6 KiB     240.4 MiB/s  925    9.5 MiB
  5. deflate-best-speed         34.3 KiB     198.4 MiB/s  22     874.6 KiB
  6. zstd                       26.8 KiB     165.4 MiB/s  907    18.5 MiB
  7. zstd-better-compression    26.3 KiB     162.3 MiB/s  881    37.3 MiB
  8. gzip-best-speed            33.7 KiB     150.6 MiB/s  28     1.2 MiB
  9. pgzip-best-speed           34.3 KiB     143.7 MiB/s  1649   220.2 MiB
 10. deflate-default            31.2 KiB     126.3 MiB/s  22     1.1 MiB
 11. lz4                        44.7 KiB     112.6 MiB/s  435    816.7 MiB
 12. pgzip                      31.2 KiB     94.6 MiB/s   2634   277.5 MiB
 13. gzip                       30.4 KiB     39.5 MiB/s   26     874.7 KiB
 14. deflate-best-compression   30.4 KiB     25.4 MiB/s   21     1 MiB
 15. gzip-best-compression      30.4 KiB     24.5 MiB/s   27     874.9 KiB
 16. pgzip-best-compression     30.4 KiB     23.1 MiB/s   2646   281.8 MiB
 17. zstd-best-compression      25.1 KiB     19.3 MiB/s   882    99.3 MiB

I did some experiments around SetConcurrency() and found that:

The consumption grows slowly as blocks increases, and exponentially as blockSize increases, possibly due to z.dstPool.New = func() interface{} { return make([]byte, 0, blockSize+(blockSize)>>4) } line.
Even by just creating a new writer and immediately close it, the allocation still happens, possibly due to the internal compressCurrent().

Is there a bug here? Why allocate memory when no data is compressed? And can Reset() reuse previously allocated memory instead of creating new (like s2)?

@CrendKing pgzip is for compressing large streams, for small payloads it doesn't make sense to use.
You are welcome to send in a PR that addresses this, but I can't spend time on it for the foreseeable future.

I see. I reran the benchmark with the full 400MB file while repeating 10 times. I was expecting an explosive number on the pgzip line. Instead, I find pgzip is in line with other compressors (see output below). So it seems pgzip basically has "plateau" pattern (like "allocate all memory needed early and keep recycling them"), while s2 has "linear" pattern ("allocate as more required").
So, as you mentioned, pgzip is optimized for large streams. If our users mostly compress small data blocks, they should consider other compressors. Thanks!

Compressing 400MB highly compressible file, repeat 10 times
Repeating 10 times per compression method (total 4.6 GiB).
...
     Compression                Compressed   Throughput   Memory Usage
------------------------------------------------------------------------------------------------
  0. s2-default                 127.1 MiB    6 GiB/s      28390  376.3 MiB
  1. s2-parallel-8              127.1 MiB    4 GiB/s      28475  365.3 MiB
  2. s2-better                  120.1 MiB    3.5 GiB/s    28431  353.7 MiB
  3. s2-parallel-4              127.1 MiB    3 GiB/s      28198  348.1 MiB
  4. pgzip-best-speed           96.7 MiB     2.1 GiB/s    38278  345.9 MiB
  5. pgzip                      86.3 MiB     1 GiB/s      37863  319.5 MiB
  6. gzip-best-speed            94.9 MiB     167.2 MiB/s  62     305.2 MiB
  7. pgzip-best-compression     83 MiB       48.7 MiB/s   38194  317 MiB
  8. gzip                       83.6 MiB     40.9 MiB/s   159    304.8 MiB
  9. gzip-best-compression      83 MiB       5.2 MiB/s    1084   304.9 MiB


Tool tip for Elevation Layer Tool says "ElevationLayer.tool"

If you mouse over the icon for the Elevation Layer tool, the tool tip says "ElevationLayer.tool" in 0.0.2

Will be fixed in 0.0.3. Thanks!


assertThatExceptionOfType does not show stack trace when unexpected exception thrown

Summary
The assertThatExceptionOfType does not show the stack trace of an unexpected exception.
In the example output below it is difficult to see what threw the UnsupportedOperationException and diagnose the problem.
Example
java.lang.AssertionError:
Expecting:
 <java.lang.UnsupportedOperationException>
to be an instance of:
 <java.lang.IllegalArgumentException>
but was instance of:
 <java.lang.UnsupportedOperationException>

Java 8 specific ?

YES : create Pull Request from the master branch
NO  : create Pull Request from the 2.x branch

Can you give an example of a more helpful error message ?

I was thinking about two possible approaches.
Having the stack trace as a separate message at the end:
java.lang.AssertionError:
Expecting:
 <java.lang.UnsupportedOperationException>
to be an instance of:
 <java.lang.IllegalArgumentException>
but was instance of:
 <java.lang.UnsupportedOperationException>
full stack trace:
 <java.lang.UnsupportedOperationException
	at mu.app.MyClass.myMethod(MyClass.java:52)
	at mu.app.MyClass.callingMethod(MyClass.java:143)
  ...>

Or, replacing but was instance of: with but was: and printing the full stack trace.
java.lang.AssertionError:
Expecting:
 <java.lang.UnsupportedOperationException>
to be an instance of:
 <java.lang.IllegalArgumentException>
but was:
 <java.lang.UnsupportedOperationException
	at mu.app.MyClass.myMethod(MyClass.java:52)
	at mu.app.MyClass.callingMethod(MyClass.java:143)
  ...>

Let me know your thoughts.

I'll go for the second option, the exception type is not repeated

Thanks for the quick response! 👍


Xiaomi Aqara buttons/sensors respond very slow since 0.83 update

Since upgrading to 0.83.x my xiaomi buttons and sensors respond very slow.
After pressing a button or passing a motion sensor it takes up to 5 seconds to have the action fired.
Triggering the automation manually fires it normally.
Getting quite annoying to pee in the dark.
Could it have anythin to do with
https://github.com/home-assistant/home-assistant/pull/18294

What kind of xiaomi buttons or sensors? What xiaomi component are you using, as there are multiple?

I see the issue with
both types motion sensors
xiaomi round button
single (square) aqara button
double (square) aqara button

Yes, I have the same issue.
I have the aquara square base/round push button and it does no longer work to turn on/off my lights with any longer at all, still works fine in mi home app but not in HA since the upgrade to 0.83.1

Different issue, read the breaking changes. the event is no longer called click, but xiaomi_aqara.click

The issue seems to have solved itself in 0.83.3.
Apologies for not sticking to the template

Yes, my bad for not seeing the comment.

Sorry if I am hijacking the thread.
I am experiencing something different since updating to 0.83.3.
After updating from 0.82.1 to 0.83.3 of Hass.io, the state of Xiaomi wireless button no longer changes on the dev-state page for single or double click on the button. The state and last_action fields on the dev-state page only change when I click and hold the xiaomi button.
There is a community thread that reports this.
https://community.home-assistant.io/t/xiaomi-button-no-longer-registers-single-or-double-press/83129
Please let me know if I should open a new issue instead.
Thanks.

Can this one be closed? All issues are solved here, right?

Yes, no issues for me


NSU for released December 2023 DLCs missing titleid

For example in US.en.json:
70050000038769 (id should be 010046B00DE63005 but it's null)
70050000042314 (id should be<PHONE_NUMBER>D3F003 but it's null)

DLC are problematic/difficult to get updates to without doing a full scan. Generally the full scan to pick up missing DLC’s is done once a month or so.
this situation usually comes up when a title launches with a set number of dlc, and then new dlc are added later which aren’t picked up as quickly as most would like.

Got it, now that I understand they will eventually update then it's not an issue.
Thank you for your work! 🙏


[package_info_plus] migrate to null-safety (#50)

Geez, I'll push some of the preparing changes separately and get back to this later.

The integration test cannot pass because of https://github.com/flutter/flutter/issues/76446.


Project: OnboardEase - A Chatbot to make onboarding experience seamless..

Project Name
OnboardEase
Description
OnboardEase is a chatbot that retrieves company-specific information such as policies, procedures, team structures, and project details to answer questions new employees might have. It can also offer personalized guidance based on their role or department.
Employees can benefit from the App making their onboarding experience as seamless as possible.
Data Source Used:
A list of Frequently Asked Questions was collected and recorded in a PDF file.
Technology & Languages

[ ] JavaScript
[ ] Java
[ ] .NET
[X] Python
[X] AI Studio
[X] AI Search
[ ] PostgreSQL
[ ] Cosmos DB
[ ] Azure SQL

Project Repository URL
https://github.com/Shunlexxi/OnboardEase
Deployed Endpoint URL
https://app-backend-ybmccerwckeci.azurewebsites.net/
Project Video
https://youtu.be/UC2VMQjnjtw
Team Members
Shunlexxi

Project Name
OnboardEase
Description
OnboardEase is a chatbot that retrieves company-specific information such as policies, procedures, team structures, and project details to answer questions new employees might have. It can also offer personalized guidance based on their role or department.
Employees can benefit from the App making their onboarding experience as seamless as possible.
Data Source Used: A list of Frequently Asked Questions was collected and recorded in a PDF file.
Technology & Languages

[ ]  JavaScript
[ ]  Java
[ ]  .NET
[x]  Python
[x]  AI Studio
[x]  AI Search
[ ]  PostgreSQL
[ ]  Cosmos DB
[ ]  Azure SQL

Project Repository URL
https://github.com/Shunlexxi/OnboardEase
Deployed Endpoint URL
https://app-backend-ybmccerwckeci.azurewebsites.net/
Project Video
https://youtu.be/6W2TxLA9Ch8
Team Members
Shunlexxi

Updated Project Video

Hello @Shunlexxi, thank you for participating in RAG Hack!
The team is working hard to distribute badges. Please have each team member fill out this form:
aka.ms/raghack/badge-dist
Thank you!


Contraceptive Use Barriers - Subnational

Create a file with the distribution of reasons for non-use, stratified by region.
Format: Region, Barrier, Percentage

Reasons for nonuse calculated from Ethiopia PMA 2019. DHS data data collection for this variable stops in 2005 for Ethiopia.
National level results:

barrier
perc

Access
2.73

Health
20.9

Knowledge
1.28

No need
58.5

Opposition
16.6

Subnational results

barrier
region
perc

Access
Addis Ababa
0.170648

Access
Afar
0.170648

Access
Amhara
0.767918

Access
Benishangul-Gumuz
0.085324

Access
Dire Dawa
0

Access
Gambela
0.170648

Access
Harari
0

Access
Oromia
0.511945

Access
SNNPR
0.767918

Access
Somali
0

Access
Tigray
0.085324

Health
Addis Ababa
1.194539

Health
Afar
0.170648

Health
Amhara
3.83959

Health
Benishangul-Gumuz
0.085324

Health
Dire Dawa
0.682594

Health
Gambela
0.255973

Health
Harari
0.511945

Health
Oromia
5.546075

Health
SNNPR
6.143345

Health
Somali
0

Health
Tigray
2.474403

Knowledge
Addis Ababa
0.085324

Knowledge
Afar
0

Knowledge
Amhara
0.255973

Knowledge
Benishangul-Gumuz
0

Knowledge
Dire Dawa
0

Knowledge
Gambela
0.170648

Knowledge
Harari
0

Knowledge
Oromia
0.426621

Knowledge
SNNPR
0.170648

Knowledge
Somali
0.170648

Knowledge
Tigray
0

No need
Addis Ababa
5.460751

No need
Afar
1.109215

No need
Amhara
11.51877

No need
Benishangul-Gumuz
2.645051

No need
Dire Dawa
4.010239

No need
Gambela
2.21843

No need
Harari
2.901024

No need
Oromia
8.617747

No need
SNNPR
11.34812

No need
Somali
0.511945

No need
Tigray
8.191126

Opposition
Addis Ababa
0.59727

Opposition
Afar
0.59727

Opposition
Amhara
2.901024

Opposition
Benishangul-Gumuz
0.426621

Opposition
Dire Dawa
1.962457

Opposition
Gambela
0.170648

Opposition
Harari
1.279863

Opposition
Oromia
3.583618

Opposition
SNNPR
3.924915

Opposition
Somali
0.255973

Opposition
Tigray
0.853242

Issue Closed: PR https://github.com/fpsim/fpsim/pull/241


Service Description isn't shown anywhere

v0.22.0-rc2
Same as https://github.com/rancherio/rancher/issues/963
Create a description to my service.
No place shows this description.

Never mind. I found the service description in the detailed view of the service.


CVE-2024-44792 (Reserved): Cross Site Scripting (XSS) in PHPFlasher v2.0.1

CVE-2024-44792 (Reserved): This CVE ID has been assigned but is not yet public. PHPFlasher v2.0.1 is vulnerable to Cross Site Scripting (XSS) via the $flasher, flash() helper.
Description
PHPFlasher v2.0.1 is vulnerable to Cross Site Scripting (XSS) via the $flasher or flash() helper. This issue allows for the injection of malicious JavaScript through flash messages.
Vulnerability Type
Cross Site Scripting (XSS)
Affected Component

$flasher or flash() helper/service in PHPFlasher v2.0.1

Attack Type
Local
Impact

Code execution is possible due to XSS.

Attack Vectors
To exploit this stored XSS vulnerability, you can use string interpolation within the flash() helper or service to set a success message. Example payloads:
flash()->success("User '{$user->name}' updated successfully.");
flash("User '{$user->name}' updated successfully.");
$flasher->success("User '{$user->name}' updated successfully.");

Proof of Concept
Here is a video demonstrating the XSS vulnerability:
Proof of Concept Video
Password: 12345jun
Discoverer
Arjun P. Rosell

Hi Arjun, Thank you for raising this issue.
To clarify, the ability to inject HTML or JavaScript through flash messages is an intentional feature of PHPFlasher, designed to give developers flexibility in how they style and structure their flash messages. Similar to the raw filter in Twig, this feature allows developers to use HTML elements like  or  to customize their messages.
However, I completely agree that it's important to ensure sanitization when dealing with dynamic data, especially when user-generated content is involved.
Here are some examples to illustrate this:
Example of Safe Usage:
When rendering static content or sanitized inputs, it's perfectly safe to use HTML in flash messages:
// Safe static usage with HTML elements
flash()->success('User profile updated successfully!');

Example of Risky Usage:
In cases where user input is directly passed into the flash message without sanitization, it becomes risky:
// Risky usage with unsanitized user input
flash()->success("User '{$user->name}' updated successfully.");

If $user->name contains malicious code, it could lead to XSS attacks.
Example of Safe Usage with Sanitization:
To prevent XSS, the developer should sanitize or escape the input before passing it to the flash message:
// Safe usage with sanitized input
$sanitizedUserName = htmlspecialchars($user->name, ENT_QUOTES, 'UTF-8');
flash()->success("User '{$sanitizedUserName}' updated successfully.");

In this example, htmlspecialchars() ensures that any potentially dangerous characters are safely escaped before being rendered in the flash message.

Next Steps
I’d love to hear your thoughts on how we could improve this further. Since PHPFlasher is designed to allow flexibility, maybe we could explore adding a note to the documentation to remind developers to sanitize inputs when passing dynamic content into flash messages.
Would you be open to contributing by:

Creating a PR to add this note to the documentation?
Suggesting other potential improvements to help make this behavior clearer to developers?

Let me know what you think, and thanks again for your contribution!
Younes

Hi Younes,
Thank you for your detailed response and for acknowledging the importance of sanitization.
I understand that the ability to inject HTML or JavaScript through flash messages is an intentional feature designed for flexibility. However, I believe that security should be a top priority, especially when dealing with user-generated content.
While experienced developers may be aware of the need to sanitize inputs, there is a significant risk that less experienced developers might overlook this, leading to potential XSS vulnerabilities.
Additionally, I already have a method to sanitize inputs using Laravel's e() helper function, which can be used as follows:
// Safe usage with sanitized input
flash()->success("User '" . e($user->name) . "' your success message here.");

I would be happy to contribute by creating a PR to add these notes to the documentation and suggesting other potential improvements but I am currently busy with my capstone project, so it might take some time.
Thank you again for your attention to this matter.
Best regards,
Arjun P. Rosell

Hi @arjunrosell, any updates on the issue regarding CVE-2024-44792 and the potential improvements we discussed?

Hi Arjun,
I wanted to let you know that i've released a new version of PHPFlasher that includes an escapeHtml option. This feature allows developers to control whether HTML content in flash messages should be escaped or rendered as-is. By setting escapeHtml to true, you can ensure that any HTML tags are properly escaped, which effectively sanitizes the title and message content to prevent potential XSS vulnerabilities.
Given this update, I will be closing the issue for now. If you have any further questions or if the issue persists, please feel free to reopen it or create a new issue.


Maintained?

There haven't been any commits in literal years to this project. @maslianok only has one commit for this month (roughly half-way through the month as of writing this), and issues are no longer being responded to.
Issue such as #113 are causing major issues for me, and the solution provided does not work with TypeScript.

does any alternative


worst API ever

It doesn't explain anything. nor does it make sense. I've been programming in javascript for 10 years now so i'm no stranger to it and to APIs, but this one is a mystery.
First of all, nowhere is explained what are the parameters you might pass, for example, to a line chart. I see you can pass data, but what else? where is the full list?
In the API page, the first thing you read is Module Chartist.AutoScaleAxis, but what does this name means?? how should even use object variable shown in the API? for example, I want to disable the vertical axis for a line chart. what is the parameter for that? searching the word vertical show no results.
There are all these functions and code..but how should one use them? use-case examples?
This whole API is one big WTF
I'm not trying to be annoying or anything, I myself have been open-sourcing a lot and if I think so, I guess countless other devs are also frustrated by the lack of proper documentation

If you're interested in the usage of line chart API you should look for http://gionkunz.github.io/chartist-js/api-documentation.html#chartistline-function-line in order to see the parameters for line charts. You can see the parameters for the Line constructor listed. In order to check the available options, just expand the default options. There are also simple usage examples directly there. This is an automatically generated documentation based on JSdoc. There are maybe a few drawbacks with automatically generated docs, but in therms or maintenance this is definitely the way to go.
If you have specific ideas please go a head and give constructive feedback.

it most certainly does not say what are all the available options to pass to the line function. I quote:

The options object with options that override the default options. Check the examples for a detailed list.

There is nothing useful in the examples below. This API is really shit, you should re-write everything, so people could actually use this library instead of wasting countless hours understanding how to do a simple thing.
An API must state all the available parameters a function can take, and this API doesn't even to that.
I don't know what JSdoc is, but I guess it generates some low-quality API from real code. Since obviously a developer can't understand anything from the API, then what is the point? The library took you countless hours to create but you won't take the time to properly explain how to use it?

.. and yet many other developers have managed to use the API for this rather nifty royalty-free graphing library to get themselves set-up.
I'm quite a novice developer yet I managed to quickly follow the documentation and figure out how to setup everything to get some great-looking charts working with my application.
.. and the developer (Gion) is very active in answering questions and providing free, incisive support for his library - much more than most other libraries I've seen ..
If you're such an experienced developer yourself, I'm surprised you're having such trouble following the documentation - or knowing that you don't get much help online by acting like an asshole.

@yairEO I'm not sure if the tone you're using in your comments helps to have a healthy discussion. I suggest we keep discussions here to the point and refrain from using all kinds of wtf's.
In addition, I find the docs clear enough for most cases and if I really must dive in deep I have a look in the source code. Any experienced developer should be able to do that.
Also, I'd like to remind you that this library has not reached a stable version yet and therefore I understand completely that @gionkunz hasn't taken the effort to make the docs 100% bulletproof. And because this is an open source project, you can help to make the docs better! That's one of the good things any developer can do, experienced or not.

Your comment doesn't address the fact that the job of an API is to explain what a function does and what can be passed to it, and yet, it does not say what are the things which can be passed.
Your specific needs might not be as the next person, and you cannot possibly know how many devs switched to another (one of the countless other free) library because they could not find what are the options to be passed, and how to fully customize a chart.
I am surprised you are surprised. I just said couple of times a full list of settings that can be passed is not in the API, so how would I know what are they? regardless of my experience as a developer, I cannot guess parameters names, it's like guessing lottery numbers.

@yairEO when you phrase feedback like that, it leads authors to believe the community is unappreciative of their work and demotivates them. I know because I've been there. It's an unfortunate reality that when you open up feedback to the whole internet, you get some knuckleheads once in a while.
Surely you understand that the author has absolutely no obligation to appease you - this is an open source project. If it's not good enough for you, fix it! Submit a pull request. Alternatively, there are plenty of other charting libraries out there. But for the sake of all of us who benefit from the author's continued work on this project, leave out the negativity, the profanity, and the rudeness from your comments.

@timwis

Not everyone are from united states and share the same culture of speech. you are a part of a global planter, and in my culture it is normal and encouraged to be as open as possible about your real views and feedback. Stop mixing feelings with computer programming, it will not help in anything. we are programmers and shouldn't feel "bad" because someone on the internet said API sucked. that's childish. instead it would be wiser to look inwards why would a person say such a thing. doesn't matter in what manner was it spoken, since you cannot imagine how my culture is, just accept the content of the feedback.

I cannot submit pull-request since I don't understand the API. It's absurd to tell me to write something which the whole point in this discussion is saying that I don't understand that very thing. The logic contradicts itself here.

I myself have been doing open-source for years and have many active projects. I'm not just a random douche from the internet who only takes and never contributes back. I am doing my part daily.

The whole point of having your code open-sourced is so others could use it. the creator probably spent months creating this. so saying he has no obligation to "appease" me is totally counter-productive to the essence of open-source, and soul of it, which is to make your project available to the public so as many people would be able to enjoy it as possible. why would you make a project and then make the community's life extremely difficult by not taking the take to write a proper API seems illogical, wouldn't you agree? unless the creator is under the illusion that his creation is as obvious to others as to him, which a creator should never assume. That would not make his open-source project truly "open".

I am deeply deeply sorry to have offended you guys, I love your project it is awesome and I have high respects for anyone taking the time writing open-source, regardless if I find it good or not. Helping the global community is the most important thing a programmer could do.

Sorry if my English isn't perfect.

Alright, since the back and forth compliments aren't leading this issue anywhere useful, let's try to pinpoint what we could do to contribute an actual improvement. So my question is: what would be a good first step to improve the documentation? What's the biggest thing missing?

well, the "API documentation" page lack texts which clearly states what are all the available options you can pass when creating a Pie, Line, or Bar chart. it just says "Check the examples for a detailed list." but there is no such "detailed list" in the examples shown. just say right there what are the options, better than sending a poor person into a research across the page which will be futile.
another thing is the clean up methods and functions which are totally irrelevant to the API, such as "extend" which for some reason appears twice and is obviously some internal method which combines object and just creates a confusion as to why would this be in a graph's API.
The main question is why would a person go this the API page and what answers would he find there? common sense suggests a user would like to configure his graph to be just the way he needs it to be, therefor, if a user wants a line chart without the "Y" axis or whatever other settings, he will look for all the available options under the CHARTIST.LINE section. that is the most logical place to look for answers. All those tons of other irreverent functions and methods only cluttering the API. For example the section: SVG-PATH.JS says "Chartist SVG path module for SVG path description creation and modification." okkk what did I just read? there are no examples why would I ever need this method nor the countless which follows it.. would be of great value to understand the usecase and the thought which was put behind those functions and in what scenario would they be of help.
I could go on

@yairEO Are the default options with comments not enough to find all available options?

Actually I'd not like to remove the internal functions from the documentation but maybe we could flag them as internal or separate them into a different section?
Certainly some more inline jsdoc in the headers of the functions / modules would help too.

@gionkunz  - well, lets see. A regular dev comes to the API page, wanting to configure their line chart. he clicks the defaultOptions link under the CHARTIST.LINE title on the side menu. He sees no options, and the title of the section it leads isn't what he/she just clicked on. Where are all the option they wonder? maybe scroll down? scrolling shows this:
var options = {
  width: '300px',
  height: '200px'
};

obviously these aren't all the options. where are ALL the options? a dev needs a list and he needs it now, not by gathering bits of options from scrapes of examples scattered around. Even if they did scroll down, they won't ever see a coherent list of all the available options and what each does. For example, where does it say that there a fullWidth option? nowhere. I had to dig it up, charging my client time for this, which in result he was wondering why it takes me so long to create graphs the way he wanted. (in some specific way) well, if the API was good it wouldn't have taken me so long to find how to set things up, saving time and money for my client. now double that with couple hundred devs which are facing the same problem. thousands of human hours wasted world-wide...for what? so difficult to have one single list of all available options and what do they do? maybe even a section of commonly used setups which one could simply copy paste? Maybe open a thread in the issues where people can post their setups in JSBIN or whatever and you can use those links in your site for examples. just ideas from the top of my head.. thanks for reading!

@yairEO You have to click on "SHOW CODE" next to "declaration defaultOptions". This will open a list of all default options.

that is the most logical place to look for answers.

actually, that would be the source code.
if you find the documentation to be lacking, perhaps write some yourself.

@yairEO

Ha thanks, now I see. never thought that a list of settings will be refereed to as "code". Never in my life I would have thought clicking that button. Instead of getting angry maybe you should re-think the UX. sorry for not mind-reading how the API works. I get FAR dumber question on my projects from users and I answer them all and question my own README and tutorials so those questions won't be raised again by another.

This is the most un-constructive issue that I've ever seen. For someone that present himself as a 10 years dev, never heard about JSdoc (the name is pretty closed to tons of other solutions to create documentation) and believe that it creates an "API" ...

@yairEO You do understand the sentence Expand the code view to see a detailed list of options with comments., though? No need for mind-reading.

@yairEO all the energy you've put in pointing how the docs failed you would have been better spent improving them. Phrasing inspired from http://www.catb.org/esr/faqs/hacker-howto.html

Nope, never heard about JSdoc, shoot me. I can give you examples of things I use which you also have never heard about.. that's not the point. Sorry, didn't see the "show code" button. I thought the test was referring to some other code later on and thought it was really weird.
guess i'm just used to APIs which looks like this - http://jqapi.com/
Sorry to have made ya'll so mad, just didn't see that button is all

I'm just going to leave this here..

I've been programming in javascript for 10 years now so i'm no stranger to it and to APIs

..

Nope, never heard about JSdoc

I'm just going to leave this here..

I've been programming in javascript for 10 years now so i'm no stranger to it and to APIs

..

Nope, never heard about JSdoc

Sorry to have made ya'll so mad, just didn't see that button is all

Behave like a big boy next time

You're on Y Combinator!

@JesseTG - where? I haven't been on that website for like 6 years maybe

i just grabbed some popcorn and watched the whole discussion. But i think its time to end the bashing? I know @yairEO could have asked a little bit nicer. But were all here for the same reason.  never thought about clicking that yellow button lol. soooo much time wasted on hacking and digging pieces of info from stackoverflow...ho well.

It's still no excuse to address the maintainers with such a demeanor. Being an open source contributor yourself, you should understand by now that this community is based on sharing constructive ideas, contributions, and collaborations. Not shooting someone down for their implementation or your lack of understanding in how to use it. Let's maintain a positive and progressive attitude and try to suppress our emotions when filing issues like this. Just uncalled for, man.
You should just apologize for the attack and move on.

I am deeply sorry for being a jerk..

@yairEO made my day :) it's not often you see a troll apologising.
I think the maintainer should feel proud that he has built such an awesome library that people would rather rant about lack of documentation instead of silently moving on to another charting library.
+1 to @gionkunz

This is a perfect example of UX failure. I am with @yairEO on this one. I feel his pain because I too have had this happen to me. Why would anyone hide the most important part of an API doc, and then force the user to click a button to display the answer they've been so desperate to find? Not even Ctrl+,f search on the web page will display what we're looking for, not until the button is pressed. Pure UX hell.

:+1: Yeah this is why I was so frustrated, ctrl+F didn't find anything on the page :/
@Wingie  - The only reason I use it is because it's not Canvas and it's responsive.  For example, client wanted a filled area graph which has lines or divided into segments, where the user can highlight with mouseover each one see a tooltip. Good luck doing that. Or having a chart fill 100% of the height without AXIS. good luck with that.

I'm completely agree with some of the ideas here but you always need to consider priorities of developers working on a library. I have tons of things I'd want to enhance on the documentation site as you can also see from the open issues. There are lot's of things that can be improved in usability of the documentation but I just didn't had the time to work on this given other priorities like fixing bugs and implementing most wanted features. This is an open source project and everyone who has spare time and the necessary motivation can help to increase the quality of Chartist. Also I'm selecting most of my priorities based on open github issues, and this issue is the first one which criticizes the usability of the documentation site (expect a few small enhancements). This does not mean that the UX on the docs is great or even sufficient, but that there are just other things that are more important at the given moment.
If you have spare time, you're more than welcome to work on the documentation system. I spent some time to automate it nicely and building more UX on the documentation site should be quite painless.
All, I appreciate your comments on this and there's clearly a clash of different opinions. Not about the user experience of the API and Documentation (which somehow got mixed up here), but more about how to communicate feedback which starts with observing why things are done like they are. In the end, everyone involved here has spend time and energy in reading and commenting that could have been put into a constrictive collaborative work to increase the documentation (or just enjoying yourself and watching a funny cat videos on youtube).
All that said I think it's really time to close this issue and thanks again to everyone for their participation!
 If you took more than ten seconds to actually read what is on the documentation page rather than coming in here and blasting the developers of this fantastic library with your immature and unhelpful comments, you might find that in reality it's just your reading comprehension skills that are lacking, not the documentation.

@thesnackalicious if you had read the thread you'd had seen that @yairEO
already apoligized ... :)

@jpic I did read through the thread before I replied and saw that he already apologised... doesn't really excuse the fact that he decided to blast everyone. Maybe I was a bit harsh though.

@jpic & @thesnackalicious ...why is it so important for @yairEO to calm your egos by apologising for expressing his frustrations about a poorly implemented UX feature that completely hides the most critical part of the documentation from the user's eyes, AND the browsers find box?  It makes me upset seeing so many people get offended by a user who is frustrated, demanding an apology, and STILL doesn't try to find a solution afterwards.

Your bad manners are only exceeded by your bad manners.
Dictated but not read.

@markentingh @jpic Immature of me to comment like that. I apologies to @yairEO and everyone else for  any offence.

None taken, don't worry about it bro B)

I know this is a long thread, but I must thank this issue to have explained me that in order to read the default options I need to click the yellow icon "Show Code".

test

It doesn't explain anything. nor does it make sense. I've been programming in javascript for 10 years now so i'm no stranger to it and to APIs, but this one is a mystery.
First of all, nowhere is explained what are the parameters you might pass, for example, to a line chart. I see you can pass data, but what else? where is the full list?
In the API page, the first thing you read is Module Chartist.AutoScaleAxis, but what does this name means?? how should even use object variable shown in the API? for example, I want to disable the vertical axis for a line chart. what is the parameter for that? searching the word vertical show no results.
There are all these functions and code..but how should one use them? use-case examples?
This whole API is one big WTF
I'm not trying to be annoying or anything, I myself have been open-sourcing a lot and if I think so, I guess countless other devs are also frustrated by the lack of proper documentation

Yeah I feel too that the documentation is completely lacking. For example, for a Pie chart, I scroll down to their documentation http://gionkunz.github.io/chartist-js/api-documentation.html#chartistpie-declaration-defaultoptions and there is no definite, and explanatory list of what all options can be passed and what does each option do.
yairEO is right IMO people just couldn't stand the tone haha

And the "Show Code" button - the so called "proposed" solution - does not give what is required of an API documentation - it just shows a sample code - as a developer I want to know all possible options, and what each option means. That's not there in "Show Code".

Developers need to work on the documentation. I need to develop, not waste time digging through the source code.

the guy did act a bit arrogant, but honestly he was right...this api documentation is awful. Please improve it

the guy did act a bit arrogant, but honestly he was right...this api documentation is awful. Please improve it

Arrogant: having or revealing an exaggerated sense of one's own importance or abilities.
I think you need to learn English, @Instrumedley. @yairEO was genuinely confused and frustrated.

the guy did act a bit arrogant, but honestly he was right...this api documentation is awful. Please improve it

Arrogant: having or revealing an exaggerated sense of one's own importance or abilities.
I think you need to learn English, @Instrumedley. @yairEO was genuinely confused and frustrated.

just stfu. U got enough thumbs down already

It is the year 2021, Chart.js is in version 3.2 and still the documentation is just plain awful...

I came here after typing "why chart.js sucks" in the google search bar. And I felt relieved when I saw a person who was genuinely frustrated with the docs. But after hearing what gion said, i felt bad for feeling like that. Because I am a rookie who has never  really contributed to an open source. Anyway the documentation is still hard for me to understand rn. In the future, When I gain some experience, I will be sure to help make things better. Cheers.


Opening a tiff file spins forever

Testing #81674
Regardless of remote or local, opening a .tif file spins forever

Hmm, looks like chrome doesn't tiff and neither does VS Code 1.38. Will update extension to note this is not supported

We should also handle loading errors like this better


chore(bot): configure heroku procfile

Uses https://elements.heroku.com/buildpacks/heroku/heroku-buildpack-multi-procfile to allow multiple Heroku Procfiles.
Serving the app at https://damp-temple-23212.herokuapp.com/. Currently works off of my test app, but the Heroku Config Vars can be changed to use the actual Electron one.

Will do in a future PR


[3.2.0] Signup UI input field UI validation is not available.

Describe your problem(s)
Need to validate the input fields of the Self SignuP UI.
password
email

I hope you are referring to this self signup page for devportal

when I tried to register with an invalid email and  a password which not complies with the password policy I got error messages as above and below

Can you please confirm the behavior ?

Stale issue. hence closing


Allow tune of usage reverse/forward index

Allow tune of usage reverse/forward index.
For example, reverse query of a.b.c.*.e might be slow in some cases. With this we can force use forward index for this queries.

This is a really cool feature that will help in one of our edge cases!

Also I have similar idea for switch some queries from reverse table. As a finder queries, some render data fetch queries with bad index distribution (at the end) have poor performance against reverse table and have better performance for non-reverse.

Also may be better use one regex for check instead of prefix/suffix. But it has a some performance disadvantages, but more flexible.

Also may be better use one regex for check instead of prefix/suffix. But it has a some performance disadvantages, but more flexible.

But not a too bad.
Benchmark_useReverseDepth-6               	17004717	        68.95 ns/op
Benchmark_useReverseDepthPrefixSuffix-6   	16678562	        72.08 ns/op
Benchmark_useReverseDepthRegex-6          	16172973	        74.05 ns/op

I'll gladly merge it. BTW, there are some commits in PR with broken tests

Yes, some commits with broken tests - for view what changes planned. It's a tests for clean-up glob/select index.
At this commits all work as later - but tests for final state.