Documentation for setting up a hyperbahn router

Forgive me if I have completely missed the underlying concepts of this, as I have yet to fully understand how the Hyperbahn routing mesh works. It appears to me as though Hyperbahn runs on a number of servers which work to form a consistent hash ring in much the same way as ringpop to handle or forward requests to the server which has information on how to connect to the relevant servers which advertise the given servicename. The problem I am having is twofold; firstly, there seem to be several references to Hyperbahn routers, yet little to no documentation or examples detailing what is required to set up a Hyperbahn router that other services can advertise on (such as this in the docs: https://github.com/uber/tchannel-node/blob/master/docs/GUIDE.md#setting-up-hyperbahn). The second problem I am having, which is very much an individual problem, is that I am not entirely clear on how TChannel/Ringpop/Hyperbahn are related. From what I understand, TChannel is the transport mechanism used by Ringpop to essentially handleorproxy requests to the correct app instance, and return the response. Hyperbahn is built upon Ringpop, and extends it to essentially create a list of ip:ports that correspond to a given servicename. Based on the documentation, in order to use a service on Hyperbahn, one instantiates a server that advertises with servicename on Hyperbahn, and makes a request to other servicenames using the getClientChannel() method, such as  (ie. Hyperbahn cannot be a client alone, it must also advertise a service). What I would be extremely grateful for would be for someone to point me in the direction of the documentation/example that demonstrates making a request over Hyperbahn and receiving the response, setting up a (barebones) Hyperbahn router, and a little detail on how tchannel subchannels and Hyperbahn interact given hyperbahn takes the root tchannel and advertises a service, how does one call different tchannel subchannels over Hyperbahn, or is this not possible/intended? And finally for someone to tell me if what I have said is utterly wrong and why. Whoever clarifies this for me shall have a beer bought for them in the event that we meet someday :)

@sashahilton00
I shared some of our internal docs:

Hyperbahn overview google
Hyperbahn overview github

there seem to be several references to Hyperbahn routers, yet little to no documentation or examples detailing what is required to set up a Hyperbahn router that other services can advertise

I exported the "running locally" document:

https://github.com/uber/hyperbahn/blob/master/docs/running-locally.md

As for how to run it in production:

Set up a ringpop bootstrap file and configure it ( see config/production.json )
Set up a remote config file and configure it ( see config/production.json )
Bootstrap your applications with a file of all the hyperbahn hosts.

You can deploy the application, As you can see in the Makefile, it needs a host, port & bootstrapFile ( which is used by ringpop )

What I would be extremely grateful for would be for someone to point me in the direction of the documentation/example that demonstrates making a request over Hyperbahn and receiving the response, setting up a (barebones) Hyperbahn router

I just updated the tchannel-node guide. The missing piece was the running-locally document.

little detail on how tchannel subchannels and Hyperbahn interact given hyperbahn takes the root tchannel and advertises a service. how does one call different tchannel subchannels over Hyperbahn, or is this not possible/intended?

Hyperbahn takes the rootChannel so it can call channel.address() and advertise the host & port to hyperbahn.
For each service you want to call you call getClientChannel({ serviceName: '...' }) and use that to make outbound requests.
So each process will advertise with hyperbahn and create N client channels to make outbound requests. The root channel is used to handle inbound requests.

And finally for someone to tell me if what I have said is utterly wrong and why. Whoever clarifies this for me shall have a beer bought for them in the event that we meet someday :)

As a bonus check out:

Hyperbahn & tchannel video
Ringpop talk

I'm afraid there are more internal docs that are harder too share.
My personal experience with running ~300k million QPS through hyperbahn @ uber has been that setting it up isn't too difficult, we were able to deploy and get ~10k QPS on it in a few weeks with confidence but scaling it out is more difficult.
The hyperbahn system was designed for "easy service discovery" and "tight coupling with tchannel RPC". Once hyperbahn is running, services are self service and you don't have to do any configuration.
To aid with scaling out we have an internal hyperbahn forks that contains some "useful" additions like dashboards and nagios alerts that is closed source. With these it will be a lot easier to monitor and have confidence in hyperbahn. We also have internal kafka -> elk integration.

@sashahilton00
I should also give a warning, the hyperbahn service discovery & routing solution is built by Uber, for Uber.
The open source project is poorly documented and running hyperbahn is non-trivial. I do not recommend using hyperbahn as an off the shelf solution for service discovery & routing.
There are better open source solutions available.
However, since the hyperbahn project is open source, if anyone wishes to implement their own router or service discovery system in nodejs or any language, then this project is a great reference implementation to study. Especially the integration tests explain a lot of scenarios in detail.

@Raynos Thanks very much for taking the time to lay that out for me, much appreciated. Just on the points you mentioned:

Set up a ringpop bootstrap file and configure it ( see config/production.json )
Set up a remote config file and configure it ( see config/production.json )
Bootstrap your applications with a file of all the hyperbahn hosts.

I assume the ringpop bootstrap file is a a JSON array of the host:ports acting as hyperbahn routers. If this is the case, what is included in the remote-config.file seen referenced here: https://github.com/uber/hyperbahn/blob/master/config/production.json#L12

Bootstrap your applications with a file of all the hyperbahn hosts.

I assume you mean passing a seedlist of the hyperbahn routers to tchannel as mentioned in the document you linked to. Will tchannel connect to all hyperbahn routers or just connect to a random few? Can one just pass a file with an array of all the Hyperbahn router host:ports?

Hyperbahn takes the rootChannel so it can call channel.address() and advertise the host & port to hyperbahn.

I was discussing an issue related to this here: https://github.com/uber/ringpop-node/pull/290#issuecomment-237719803 (from this comment to the bottom). I was essentially asking the question of how does HyperbahnClient.advertise() work when running in docker, given that docker maps an ephemeral port to the internal listening port, the ephemeral port of which is unknown to the node application.

For each service you want to call you call getClientChannel({ serviceName: '...' }) and use that to make outbound requests.

I assume that using this will just open a line of communication to one of the servers advertising on that serviceName. If so, I assume I need to run Ringpop on each of the servers advertising on a Hyperbahn serviceName to handleOrProxy to the correct server within that particular service (I've had a few beers, sorry if that is unclear).
Thanks for the links to the video, I have actually watched them (multiple times, kudos to @mranney for that presentation, refreshingly concise) to get a better understanding of how Hyperbahn/Ringpop works, as the concept of decentralised service discovery based on hashing is much more appealing when compared to having a centralised service registry. We are just running an etcd registry at the moment whilst we build the services.
Also, thanks for the warning, I was thinking that I was just being thick and that the answer lay in the documentation in plain sight. We are currently in the process of testing possible architectures for rewriting our prototype for production, and hyperbahn looks like a pretty good starting point for service discovery on an architecture that is mostly node.js at the moment, albeit it comes with a relatively steep learning curve. If you know of any better service discovery systems that are aimed at node.js, please drop a link, we have been looking at the likes of eureka, serf and so on, though if you have any in mind, I'm listening. Other than that, thanks for the help you have provided thus far.


Out of bound error

This error occurs while training. Is this error with train.txt file?
If so, how can i print out the image path that is being fed?

xmax < w and ymax < h

train loss: 736.09:  15%|████████████████████▎                                                                                | 1375/9029 [06:02<32:22,  3.94it/s][64 32 16]
Traceback (most recent call last):
File "train.py", line 184, in
if name == 'main': YoloTrain().train()
File "train.py", line 144, in train
for train_data in pbar:
File "/home/mousumi/anaconda3/envs/Tensorflow/lib/python3.7/site-packages/tqdm/_tqdm.py", line 1022, in iter
for obj in iterable:
File "/home/mousumi/PycharmProjects/tensorflow-yolov3-master/core/dataset.py", line 90, in next
label_sbbox, label_mbbox, label_lbbox, sbboxes, mbboxes, lbboxes = self.preprocess_true_boxes(bboxes)
File "/home/mousumi/PycharmProjects/tensorflow-yolov3-master/core/dataset.py", line 235, in preprocess_true_boxes
label[i][yind, xind, iou_mask, :] = 0.0
IndexError: index 64 is out of bounds for axis 1 with size 64
After training for sometime, I got this error. why? and I am also confused about the anchor size. I used k-means algorithm to find out clusters for my dataset using your code but i am not sure it will work or not. The input image size is 512 and I am using VEDAI dataset for single class problem. I am interested to know that when I get clusters using k-means algorithm, they are actually normalized clusters with the scale of my dataset. So when I put the clusters in baseline_anchors.txt , should I use the normalized clusters or I need to multiply it with the scale of my input image.

I guess it happens when the bboxes coordinates are parsed from the '_train.txt' file, the values of those coordinates must be in between some given values, but it would be great to get to know those values

xmax < w and ymax < h

I've already made sure that my xmax, xmin, ymax, ymin are in this range, but it still throw this error. I have to minus 1 manually. LOL

xmax < w and ymax < h

I've already made sure that my xmax, xmin, ymax, ymin are in this range, but it still throw this error. I have to minus 1 manually. LOL

Does this method work for you? I have similar problem , sometimes it throw error after two iterations , sometimes after 7 , and so on.

xmax < w and ymax < h

I've already made sure that my xmax, xmin, ymax, ymin are in this range, but it still throw this error. I have to minus 1 manually. LOL

Does this method work for you? I have similar problem , sometimes it throw error after two iterations , sometimes after 7 , and so on.

yes, it works for me.

xmax < w and ymax < h

I've already made sure that my xmax, xmin, ymax, ymin are in this range, but it still throw this error. I have to minus 1 manually. LOL

Does this method work for you? I have similar problem , sometimes it throw error after two iterations , sometimes after 7 , and so on.

yes, it works for me.

Hi,   -1 from w or from h? or from both?   Can you share the script please


Fix handling of essential DOFs in GeometricMultigrid

This fixes a bug in the multigrid solver by representing the prolongations by RectangularConstrainedOperator. If the prolongation operators are not constrained, then their transpose will have nontrivial action on the essential DOFs, resulting in degraded convergence.
Resolves #4002.

PR
Author
Editor
Reviewers
Assignment
Approval
Merge

#4003
@pazner
@v-dobrev
@sebastiangrimberg + @tarikdzanic
12/1/23
⌛due 12/15/23
⌛due 12/22/23

PR Checklist

[ ] Code builds.
[ ] Code passes make style.
[ ] Update CHANGELOG:

[ ] Is this a new feature users need to be aware of? New or updated example or miniapp?
[ ] Does it make sense to create a new section in the CHANGELOG to group with other related features?

[ ] Update INSTALL:

[ ] Had a new optional library been added? If so, what range of versions of this library are required? (Make sure the external library is compatible with our BSD license, e.g. it is not licensed under GPL!)
[ ] Have the version ranges for any required or optional libraries changed?
[ ] Does make or cmake have a new target?
[ ] Did the requirements or the installation process change? (rare)

[ ] Update continuous integration server configurations if necessary (e.g. with new version requirements for each of MFEM's dependencies)

[ ] .github
[ ] .appveyor.yml

[ ] Update .gitignore:

[ ] Check if make distclean; git status shows any files that were generated from the source by the project (not an IDE) but we don't want to track in the repository.
[ ] Add new patterns (just for the new files above) and re-run the above test.

[ ] New examples:

[ ] All sample runs at the top of the example source file work.
[ ] Update examples/makefile:

[ ] Add the example code to the appropriate SEQ_EXAMPLES and PAR_EXAMPLES variables.
[ ] Add any files generated by it to the clean target.
[ ] Add the example binary and any files generated by it to the top-level .gitignore file.

[ ] Update examples/CMakeLists.txt:

[ ] Add the example code to the ALL_EXE_SRCS variable.
[ ] Make sure THIS_TEST_OPTIONS is set correctly for the new example.

[ ] List the new example in doc/CodeDocumentation.dox.
[ ] If new examples directory (e.g.examples/pumi), list it in doc/CodeDocumentation.conf.in
[ ] Companion pull request for documentation in mfem/web repo:

[ ] Update or add example-specific documentation, see e.g. the src/examples.md.
[ ] Add the description, labels and screenshots in src/examples.md and src/img.
[ ] In examples.md, list the example under the appropriate categories, add new categories if necessary.
[ ] Add a short description of the example in the "Extensive Examples" section of features.md.

[ ] New miniapps:

[ ] All sample runs at the top of the miniapp source file work.
[ ] Update top-level makefile and makefile in corresponding miniapp directory.
[ ] Add the miniapp binary and any files generated by it to the top-level .gitignore file.
[ ] Update CMake build system:

[ ] Update the CMakeLists.txt file in the miniapps directory, if the new miniapp is in a new directory.
[ ] Add/update the CMakeLists.txt file in the new miniapp directory.
[ ] Consider adding a new test for the new miniapp.

[ ] List the new miniapp in doc/CodeDocumentation.dox
[ ] If new miniapps directory (e.g.miniapps/nurbs), add it to MINIAPP_SUBDIRS in the makefile.
[ ] If new miniapps directory (e.g.miniapps/nurbs), list it in doc/CodeDocumentation.conf.in
[ ] Companion pull request for documentation in mfem/web repo:

[ ] Update or add miniapp-specific documentation, see e.g. the src/meshing.md and src/electromagnetics.md files.
[ ] Add the description, labels and screenshots in src/examples.md and src/img.
[ ] The miniapps go at the end of the page, and are usually listed only under a specific "Application (PDE)" category.
[ ] Add a short description of the miniapp in the "Extensive Examples" section of features.md.

[ ] New capability:

[ ] All new public, protected, and private classes, methods, data members, and functions have full Doxygen-style documentation in source comments. Documentation should include descriptions of member data, function arguments and return values, template parameters, and prerequisites for calling new functions.
[ ] Pointer arguments and return values must specify whether ownership is being transferred or lent with the call.
[ ] Any new functions should include descriptions of their intended use e.g. for internal use only, user-facing, etc., along with references to example code whenever possible/appropriate.
[ ] Consider adding new sample runs in existing examples to highlight the new capability.
[ ] Consider saving cool simulation pictures with the new capability in the Confluence gallery (LLNL only) or submitting them, via pull request, to the gallery section of the mfem/web repo.
[ ] If this is a major new feature, consider mentioning it in the short summary inside README (rare).
[ ] List major new classes in doc/CodeDocumentation.dox (rare).

[ ] Update this checklist, if the new pull request affects it.
[ ] Run make unittest to make sure all unit tests pass.
[ ] Run the tests in tests/scripts.
[ ] (LLNL only) After merging:

[ ] Update internal tests to include the new features.

This PR is now under review (see the table in the PR description). To help with the review process, please do not force push to the branch.

@pazner , @v-dobrev  -
Thank you for the quick fix!
When we run this with our elasticity parameters, we get the following
E=1, E>>1 -- good convergence ~10-18 iterations
E << 1, -- poor convergence rates 50-300 iterations
So I'm wondering why things head south when E << 1?
We are also trying cases where Emax/Emin > 1e3 and we see similar behavior to when E << 1
Thanks
Joe

@jtristano, since you are solving elasticity, could it be that for $E \ll 1$ your material is nearly incompressible? In that case, I would expect worse convergence.
I also think that if the ratio $E_{\max} / E_{\min}$ is very large, then you will probably needed a stronger smoother to get robust convergence.

ping: @tarikdzanic

Merged in next for testing...

Re-merged in next for testing...

Re-merged in next for testing...


UnboundLocalError: local variable 'correct' referenced before assignment.

Let me start this by telling you guys that I am a complete noob with Discord bot development, so it might be something that I am doing wrong. I will appreciate it if you can provide me with any guidance. Thank you in advance.
Describe the bug
While testing the example(https://github.com/terabyte3/discord-ext-forms) that you guys have on the main page of this repo, I keep getting the error below:
Ignoring exception in command testform:
Traceback (most recent call last):
File "C:\Users\rosorio\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\discord\ext\commands\core.py", line 85, in wrapped
ret = await coro(*args, **kwargs)
File "C:\Users\rosorio\Documents\PersonalProject\Discord_Trade_Bot\bot.py", line 65, in testform
result = await form.start()
File "C:\Users\rosorio\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\discord\ext\forms\form.py", line 297, in start
if correct:
UnboundLocalError: local variable 'correct' referenced before assignment
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\rosorio\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\discord\ext\commands\bot.py", line 902, in invoke
await ctx.command.invoke(ctx)
File "C:\Users\rosorio\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\discord\ext\commands\core.py", line 864, in invoke
await injected(*ctx.args, **ctx.kwargs)
File "C:\Users\rosorio\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\discord\ext\commands\core.py", line 94, in wrapped
raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: UnboundLocalError: local variable 'correct' referenced before assignment
To Reproduce
I am using Python -V 3.9.7. Also, I installed these requirements:
discord.py
python-dotenv==0.13.0
discord.ext.forms
furo
Sphinx==3.5.4
numpydoc
`import os
import json
import re
import discord
import random
from os.path import join, dirname
from discord import message
from discord import client
from discord.ext.forms.helpers import channel
from dotenv import load_dotenv
from discord.utils import get
#from discord.ext.forms.form import Validator
from discord.ext.forms import Form
from discord.ext import commands
DISCORD_TOKEN = os.environ.get('DISCORD_TOKEN')
load_dotenv()
TOKEN = os.getenv('DISCORD_TOKEN')
bot = commands.Bot(command_prefix='!')
@bot.event
async def on_ready():
""" Runs once the bot has established a connection with Discord """
print(f'{bot.user.name} has connected to Discord')

# check if bot has connected to guilds
if len(bot.guilds) > 0:
    print('connected to the following guilds:')

    for guild in bot.guilds:
        print(f'* {guild.name}#{guild.id}')

@bot.command(name="test")
@commands.has_role("Admin")
async def on_test(ctx):
guild = ctx.guild
channel = ctx.channel
author = ctx.author
command = ctx.message.content

# log event on command line
print(f'* "{author.name}" issued "{command}" in "{channel}" of "{guild}"')

# post random GIF from the JSON file
await channel.send(f'* "{author.name}" issued "{command}" in "{channel}" of "{guild}"')

@bot.command()
async def testform(ctx):
form = Form(ctx,'Title')
form.add_question('Question 1','first')
form.add_question('Question 2','second')
form.add_question('Question 3','third')
result = await form.start()

embed=discord.Embed(title="details", description=f"1st: `{result.first}`\n 2nd: `{result.second}`\n 3rd: `{result.third}`")

await ctx.send(embed=embed)

if name == "main":
# launch bot
bot.run(TOKEN)`
Expected behavior
the same mechanic that you guys are displaying on the main page of this repo.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

Windows 10 pro
Visual Studio Code

Additional context
Add any other context about the problem here.

Thanks for opening an issue! Someone had just brought this up not too long ago, and I hadn't released the changes to PyPI. Try updating, it should work.

Any updates?

hey, sorry for the late response. So yeah, it works now, but not as it is described in your example. What I was expecting is eliminate the BOT asking me a new question on a new line, but the bot is asking me the second and third questions on the next lines.
Description of what I am seeing:

What I was expecting to see is the same mechanic that you have on the main page where basically when the user answers the first question, the bot seems to remove/use the same emb form to show the second question and so on. making the interaction very clean (instead of what You see in my screenshot. )
Here is my code:
`import os
import json
import re
import discord
import random
from os.path import join, dirname
from discord import message
from discord import client
from discord.ext.forms.helpers import channel
from dotenv import load_dotenv
from discord.utils import get
#from discord.ext.forms.form import Validator
from discord.ext.forms import Form
from discord.ext import commands
DISCORD_TOKEN = os.environ.get('DISCORD_TOKEN')
load_dotenv()
TOKEN = os.getenv('DISCORD_TOKEN')
bot = commands.Bot(command_prefix='!')
@bot.event
async def on_ready():
""" Runs once the bot has established a connection with Discord """
print(f'{bot.user.name} has connected to Discord')

# check if bot has connected to guilds
if len(bot.guilds) > 0:
    print('connected to the following guilds:')

    for guild in bot.guilds:
        print(f'* {guild.name}#{guild.id}')

@bot.command(name="test")
@commands.has_role("Admin")
async def on_test(ctx):
guild = ctx.guild
channel = ctx.channel
author = ctx.author
command = ctx.message.content

# log event on command line
print(f'* "{author.name}" issued "{command}" in "{channel}" of "{guild}"')

# post random GIF from the JSON file
await channel.send(f'* "{author.name}" issued "{command}" in "{channel}" of "{guild}"')

@bot.command()
async def testform(ctx):
form = Form(ctx,'Title')
form.add_question('Question 1','first')
form.add_question('Question 2','second')
form.add_question('Question 3','third')
result = await form.start()

embed=discord.Embed(title="details", description=f"1st: `{result.first}`\n 2nd: `{result.second}`\n 3rd: `{result.third}`")

await ctx.send(embed=embed)

if name == "main":
# launch bot
bot.run(TOKEN)`
I pretty much copy what you have on the main page. Not sure if I am missing anything else. Thank you very much for your help.
Additional question:
I am probably pushing it and this is off topic: but do you know or could you point me in the right direction. I am trying to DM a bot in my server and have it send a message to a specific channel in my server. I would like my admins to do the same but I have not been able to find out if it is possible. Anything I have tried tells me that action is not allowed from a DM.

when you init your form, set editanddelete=True:
form = Form(ctx,'Title', editanddelete)
form.add_question('Question 1','first')
form.add_question('Question 2','second')
form.add_question('Question 3','third')
result = await form.start()

embed=discord.Embed(title="details", description=f"1st: `{result.first}`\n 2nd: `{result.second}`\n 3rd: `{result.third}`")

await ctx.send(embed=embed)

Thank you very much for all your assistance. As I said before, I am new to developing Discord Bots, and this library is awesome. Now, I do have a follow-up question. I just found out that discord.py is not going to be active anymore so what does that mean for this library? could I still use it with something like Pycord? Also, if I continue to use Discord.py would my Bot eventually not work if Discord enforces their new standards? Sorry for asking so many questions, if this is too much trouble, would you mind suggesting a place where I can ask these questions? I am pretty new to the python community.  I have focused more on .NET development and do not know many people that are active in the python community. thank you in advance.

You should be able to use it with any wrapper, so long as it uses the import discord. If one doesn't, you could definitely fork it and just change all the discord imports to something else.
I just looked up pycord, and it should work, even though I haven't tested it.
Discord.py, assuming you aren't using the 2.0 version, will become obsolete in April 2022 (i think), when they will make it so that you have to explicitly request the ability to read message content.
I have no problem with you asking these questions, and if you want you can join the pycord server here


Tail of logs failed - logs only appear after the step

Component
web-ui
Describe the bug
Hello everyone !
First of all, thank you for the great work you do here!
We have a problem on our Woodpecker test instance; when running steps, the logs only appear after the step is completed. During execution, the log window remains black, which can be a problem when some tasks are very long.
As a quick diagnostic:

https://host/stream/logs/org/repo/34/6 returns event=error, data=eof during the step execution

: ping

event: error
data: eof

The woodpecker server logs:
{"level":"error","error":"stream: not found","time":"2021-12-08T14:45:36Z","message":"tail of logs failed"}

it is always the case. As soon as the step is finished (failed or successful), the logs are displayed as if by magic.
Any idea ?
System Info
{"source":"https://github.com/woodpecker-ci/woodpecker","version":"next-03a43567"}

Additional context
The woodpecker-server is on the same machine as the woodpecker-agent, both running with docker-compose.

A Caddy reverse-proxy is setup in front of the woodpecker-server (reverse of localhost:8000) and the WOODPECKER_HOST variable is setup.

The build works fine, just the logs don't appear.

Validations

[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A join our Discord Chat Server.

Seems that I have the same. During execution of step, no logs are presented. In Firefox development console only "eof" in the response presents.

Config: https://codeberg.org/argrento/termux-packages/src/branch/master/.woodpecker.yml
docker-compose:
# docker-compose.yml:
version: '3'

services:
  woodpecker-server:
    image: woodpeckerci/woodpecker-server:next
    restart: always
    ports:
      - 80:80
      - 443:443
      - 8000:8000
      - 9000:9000
    volumes:
      - woodpecker-server-data:/var/lib/woodpecker/
    environment:
      - WOODPECKER_OPEN=false
      - WOODPECKER_HOST={hidden}
      - WOODPECKER_LETS_ENCRYPT=true
      - WOODPECKER_LOG_LEVEL=error
      - WOODPECKER_ADMIN=argrento
      - WOODPECKER_DEFAULT_PIPELINE_TIMEOUT=120
      - WOODPECKER_MAX_PIPELINE_TIMEOUT=1440
      - WOODPECKER_GITEA=true
      - WOODPECKER_GITEA_URL=https://codeberg.org
      - WOODPECKER_GITEA_CLIENT={hidden}
      - WOODPECKER_GITEA_SECRET={hidden}
      - WOODPECKER_AGENT_SECRET={hidden}

  woodpecker-agent:
    image: woodpeckerci/woodpecker-agent:next
    command: agent
    restart: always
    depends_on:
      - woodpecker-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /mnt/data:/mnt/data
    environment:
      - WOODPECKER_SERVER=woodpecker-server:9000
      - WOODPECKER_AGENT_SECRET={hidden}

volumes:
  woodpecker-server-data:


add optional GSB v4 lookup URL argument to SafeBrowsing

Thanks for doing that, but is there anyone else than Google providing a Safe Browsing API ?

As a matter of fact, yes Threat Data Science offers a GSBv4 Lookup API compatible service: https://ts.threatdatascience.io/

Ah, I did not know that, thanks for sharing


CLI tools don't handle CRAM files correctly

The CLI tools like Cat/Sort need to be updated to properly handle CRAM files (once https://github.com/HadoopGenomics/Hadoop-BAM/issues/60) is done.

Not needed in light of #83


Dark tree leaves in experimental mode

1st picture: Caves and Cliffs enabled
2nd picture: Caves and Cliffs disabled
Heres the shadow map:

The tree leaves that appear dark no longer have color.a=0.0, instead they have color.a=1.0 now. As a result, the color.a method of tree leaves detection is no longer suitable.
An alternative tree leaves detection is needed to fix it.

The issue has been partially fixed in c6a51a0


Logs, Events, and Debugging (2.0)

Logs example PR: https://github.com/konveyor/mig-controller/pull/992

Merging as implementable. Will follow up with more updates to sections on events etc. as I continue to develop.


[video_player_android] PlatformException(channel-error, Unable to establish connection on channel., null, null)

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce
When running the flutter run --release command to run the app on OnePlus Ace phone, the following problem occurred:
[ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: PlatformException(channel-error, Unable to establish connection on channel., null, null)
#0      AndroidVideoPlayerApi.initialize (package:video_player_android/src/messages.g.dart:271)

Expected results
video can play
Actual results
the following problem occurred:
[ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: PlatformException(channel-error, Unable to establish connection on channel., null, null)
#0      AndroidVideoPlayerApi.initialize (package:video_player_android/src/messages.g.dart:271)

Code sample
Code sample

class Demo4Controller extends GetxController {
  final videoPlayerController = VideoPlayerController.network(
      'https://flutter.github.io/assets-for-api-docs/assets/videos/butterfly.mp4');

  bool _ready = false;

  bool get ready => _ready;

  late ChewieController chewieController;

  String videoUrl =
      "http://commondatastorage.flutter-io.cn/gtv-videos-bucket/sample/BigBuckBunny.mp4";

  @override
  void onInit() async {

    await videoPlayerController.initialize();

    chewieController = ChewieController(
      videoPlayerController: videoPlayerController,
      autoPlay: true,
      looping: true,
    );

    _ready = true;

    update();

    super.onInit();
  }

  @override
  void onReady() async {
    super.onReady();
  }

  @override
  void onClose() {
    chewieController.dispose();
    super.onClose();
  }
}

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
Logs
E/flutter ( 7797): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: PlatformException(channel-error, Unable to establish connection on channel., null, null)
E/flutter ( 7797): #0      AndroidVideoPlayerApi.initialize (package:video_player_android/src/messages.g.dart:271)
E/flutter ( 7797): <asynchronous suspension>
E/flutter ( 7797):
E/flutter ( 7797): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: PlatformException(channel-error, Unable to establish connection on channel., null, null)
E/flutter ( 7797): #0      AndroidVideoPlayerApi.create (package:video_player_android/src/messages.g.dart:293)
E/flutter ( 7797): <asynchronous suspension>
E/flutter ( 7797): #1      AndroidVideoPlayer.create (package:video_player_android/src/android_video_player.dart:66)
E/flutter ( 7797): <asynchronous suspension>
E/flutter ( 7797): #2      VideoPlayerController.initialize (package:video_player/video_player.dart:404)
E/flutter ( 7797): <asynchronous suspension>
E/flutter ( 7797): #3      Demo4Controller.onInit (package:meebox/pages/demo/demo4/demo_controller.dart:40)
E/flutter ( 7797): <asynchronous suspension>
E/flutter ( 7797):

Flutter Doctor output
Doctor output
$ flutter doctor -v
Flutter assets will be downloaded from https://storage.flutter-io.cn. Make sure you trust this source!
[✓] Flutter (Channel stable, 3.10.3, on Microsoft Windows [版本 10.0.22621.1848], locale zh-CN)
    • Flutter version 3.10.3 on channel stable at D:\apps\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision f92f44110e (3 weeks ago), 2023-06-01 18:17:33 -0500
    • Engine revision 2a3401c9bb
    • Dart version 3.0.3
    • DevTools version 2.23.1
    • Pub download mirror https://pub.flutter-io.cn
    • Flutter download mirror https://storage.flutter-io.cn

[✓] Windows Version (Installed version of Windows is version 10 or higher)

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.2)
    • Android SDK at D:\apps\Android\sdk
    • Platform android-TiramisuPrivacySandbox, build-tools 33.0.2
    • ANDROID_HOME = D:\apps\Android\sdk
    • Java binary at: D:\apps\Android Studio\jbr\bin\java
    • Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-9505619)
    • All Android licenses accepted.

[✗] Chrome - develop for the web (Cannot find Chrome executable at .\Google\Chrome\Application\chrome.exe)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[✓] Visual Studio - develop for Windows (Visual Studio 生成工具 2019 16.11.25)
    • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
    • Visual Studio 生成工具 2019 version 16.11.33423.256
    • Windows 10 SDK version 10.0.19041.0

[✓] Android Studio (version 2022.1)
    • Android Studio at D:\apps\Android Studio
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-9505619)

[✓] VS Code (version 1.79.2)
    • VS Code at C:\Users\lu\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.66.0

[!] Proxy Configuration
    • HTTP_PROXY is set
    ! NO_PROXY is not set

[✓] Connected device (3 available)
    • SEA AL10 (mobile) • 6HJ4C19713012657 • android-arm64  • Android 10 (API 29)
    • Windows (desktop) • windows          • windows-x64    • Microsoft Windows [版本 10.0.22621.1848]
    • Edge (web)        • edge             • web-javascript • Microsoft Edge 114.0.1823.51

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 2 categories.

Hi @tcdspring. Thanks for filing this issue. Are you currently on the latest version of the video_player package? If not kindly upgrade to the latest version. If issue persists on the latest versions, kindly provide a minimal complete reproducible code sample that doesn’t include 3rd party plugins or complex production code.
Thank you

@dam-ease I tried to create a minimal demo program that references only video_player one plugin, so that it works. But in my application, there is a problem, I suspect that there may be a problem with some plugin references, and I am still looking for the reason.


clean restart all

How to clean all docker images and related files to restart clean install ?  After update I cant login and cant ingest
ie:
r2r ingest-files  test.txt
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/httpx/_client.py", line 1674, in send
response = await self._send_handling_auth(
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/httpx/_client.py", line 1702, in _send_handling_auth
response = await self._send_handling_redirects(
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/httpx/_client.py", line 1739, in _send_handling_redirects
response = await self._send_single_request(request)
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/httpx/_client.py", line 1776, in _send_single_request
response = await transport.handle_async_request(request)
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/httpx/_transports/default.py", line 376, in handle_async_request
with map_httpcore_exceptions():
File "/usr/lib/python3.10/contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/httpx/_transports/default.py", line 89, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectError: All connection attempts failed
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/bc/Projects/ODS/R2R/venv2R2R/bin/r2r", line 8, in
sys.exit(main())
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/cli/cli.py", line 47, in main
cli()
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/click/decorators.py", line 45, in new_func
return f(get_current_context().obj, *args, **kwargs)
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/cli/commands/ingestion.py", line 32, in ingest_files
response = client.ingest_files(file_paths, metadatas, document_ids)
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/sdk/client.py", line 224, in sync_wrapper
result = asyncio.get_event_loop().run_until_complete(
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/nest_asyncio.py", line 98, in run_until_complete
return f.result()
File "/usr/lib/python3.10/asyncio/futures.py", line 201, in result
raise self._exception.with_traceback(self._exception_tb)
File "/usr/lib/python3.10/asyncio/tasks.py", line 232, in __step
result = coro.send(None)
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/sdk/ingestion.py", line 87, in ingest_files
return await client._make_request(
File "/home/bc/Projects/ODS/R2R/venv2R2R/lib/python3.10/site-packages/sdk/client.py", line 147, in _make_request
raise R2RException(
sdk.models.R2RException: Request failed: All connection attempts failed

Can you try deleting your related volumes and running docker system prune? Alternatively, if you have data in Docker that you do not want to put at risk, then you can try changing the value of POSTGRES_VECS_COLLECTION

I did but still cant login as normal start neither graph neo4j  start.
I suspect messed the install so need a verified scricpt that clean or update all to the current version of the scripts and docker.
is docker image updates sysnc with code?

It could be that you have a cached docker image, something we see happening to our users on occasion.
What version does the server output when you start it? The latest release should be 3.0.6. You may need to explicitly delete your R2R image and then re-fetch it to be certain that it updates.


Split NotebookControllerManager

For #10435
Split NotebookControllerManager into 5 parts as a prereq for redesigning some of this:

IControllerSelection - selected kernel of a notebook and different events
IControllerLoader - responsible for loading kernels on startup or during refresh events
IControllerRegistration - list of all of the controllers ever created
IControllerPreferredService - computes the preferred kernel for a notebook
IControllerDefaultService - computes a 'default' controller. This is the active interpreter in local and the 'default' Python 3 kernel in remote.

Codecov Report

Merging #10537 (6922e71) into main (8f5bc0e) will decrease coverage by 30%.
The diff coverage is 26%.

@@           Coverage Diff            @@
##            main   #10537     +/-   ##
========================================
- Coverage     71%      40%    -31%
========================================
  Files        464      472      +8
  Lines      27902    27991     +89
  Branches    4699     4690      -9
========================================
- Hits       19835    11305   -8530
- Misses      6199    15696   +9497
+ Partials    1868      990    -878

Impacted Files
Coverage Δ

src/interactive-window/interactiveWindow.ts
7% <0%> (-68%)
:arrow_down:

src/notebooks/debugger/debuggingManagerBase.ts
9% <0%> (-61%)
:arrow_down:

src/notebooks/types.ts
100% <ø> (ø)

src/standalone/import-export/exportCommands.ts
12% <0%> (-32%)
:arrow_down:

src/telemetry/types.ts
100% <ø> (ø)

src/standalone/api/kernelApi.ts
11% <7%> (-62%)
:arrow_down:

src/notebooks/controllers/liveKernelSwitcher.ts
21% <9%> (-65%)
:arrow_down:

...oks/controllers/installPythonControllerCommands.ts
19% <14%> (-33%)
:arrow_down:

...tebooks/controllers/kernelFilter/kernelFilterUI.ts
16% <14%> (-17%)
:arrow_down:

...otebooks/controllers/controllerPreferredService.ts
17% <17%> (ø)

... and 336 more

I think I'd rather leave the algorithm the way it is unless we can prove there's a bug. There's some subtle behavior here and I'd rather not change it if I can.


Make cloud-init style file less verbose

There are several things that can be eased out on the cloud-init supported syntax:

[x] First-class support for systemd commands (systemctl, etc.)
[x] // for sysctl commands
[x] // setup of users, and authorized_keys for ssh login
[x] // hostname
[x] // modules to modprobe
[x] ntp_server (?)
[ ] wifi/password(?)
[x] users/password ( see e.g. https://github.com/rancher/k3os#k3ospassword )
[ ] Ability to load github/gitlab repository's yaml files as would be with local directories

For inspiration, take a look at https://github.com/rancher/k3os#sample-configyaml
We can switch to a different format, or either change also the current implementation. For reference, at the moment we are using https://github.com/mudler/yip - which can be also tweaked towards that purpose as needed.

I've just cutted a 0.7.2 of yip including a lot more functionalities which are documented there. We can then re-use the same docs in cOS when we have a proper documentation in place ( See #49 ). The only missing points are setting up wireless and networking


Enhances a shader uniform limit warning

After this change, it will show a line where the limit was exceeded + this warning can now be correctly hidden.

Enchances a shader uniform limit warning -> Enhances a shader uniform limit warning

Thanks!


Removes more recent link from list element

This PR removes the show more button on the welcome page from a list element to a div for accessibility alignment.
This PR fixes #10622

i am curious why are there so many commits for this PR. @v-bbrady

i am curious why are there so many commits for this PR. @v-bbrady

@alanrenmsft I believe it's just because i created the branch off an older branch. I can submit the PR with just the one commit. I'll do that now.

i am curious why are there so many commits for this PR. @v-bbrady

@alanrenmsft I believe it's just because i created the branch off an older branch. I can submit the PR with just the one commit. I'll do that now.

yes, please.

Coverage decreased (-0.07%) to 35.314% when pulling dc928605b362bfc7122078f26982b56aae770013 on removes-more-recent-link-from-list-element into e540096e07ebbd3b98966da56aa73ce0bbd19a20 on main.


Register

@JuliaRegistrator register

@JuliaRegistrator register

Registration pull request created: JuliaRegistries/General/57775
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.1.0 -m "<description of version>" 5c550a95d748d30fc885d2b5bfa8da0e9815c533
git push origin v0.1.0

Registration pull request updated: JuliaRegistries/General/57775
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.1.0 -m "<description of version>" 5c550a95d748d30fc885d2b5bfa8da0e9815c533
git push origin v0.1.0

@JuliaRegistrator register

Registration pull request updated: JuliaRegistries/General/57775
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.1.0 -m "<description of version>" f59aba965eebdcd3de2c70386f1ea076a5e56a26
git push origin v0.1.0

@JuliaRegistrator register

Registration pull request updated: JuliaRegistries/General/57775
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.1.0 -m "<description of version>" 49cc75cc9c86f464a4391c8b03d9bfd29d1e2464
git push origin v0.1.0

@JuliaRegistrator register

Registration pull request created: JuliaRegistries/General/63885
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.2.0 -m "<description of version>" f1b4379d9ab058148e702e2251a623da131ca575
git push origin v0.2.0

@JuliaRegistrator register

Registration pull request created: JuliaRegistries/General/63886
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.2.1 -m "<description of version>" 591246db1f34762e9ff7fa96f4851338abe5cbdf
git push origin v0.2.1

@JuliaRegistrator register


Assertion failure error when minting FREEBI to FREEOS

22/6/22 freegov2 v0.9.25 iteration: 460109, account: vivvestin.

Login as vivvestin.
Run mintfreeos action-> user: vivvestin, input_quantity: 1.0000 FREEBI, mint_fee_currency: 4,XPR
Submit. Error: assertion failure with message: invalid currency for quantity

4/7/22 Retest. freeosgov2 v0.9.27, account: vivvestin. Failed with assertion failure with message: read.

6/7/22 Retest. freeosgov2 v0.9.28 iteration: 460295, account=vivvestin. Not fixec.
Mint FREEBI to FREEOS usingmintfeefree allowance + mint fee.
Convert 3 POINTs to FREEOS where mintfeefree allowance is < mintfreeos amount i.e. 2.0000 POINT.
Pay mintfee of 1 XPR for the remaining amount of 1 POINT.
After minting:
-freeos, freeostokens balance increased from 10 to 11 freeos. Correct.
-freeosgov2 bal remain the same at 1800.3716 POINT. Correct.
-freeosgov2 mintfeefree (freepoints) reduced from 89 POINT to 88 POINT. Expecting to remain as 89 POINT.
-freeosgov2 CREDIT balance remain as 1.0000FREEBI. Expecting None as CREDIT should have been used as mint fee.

I'm not sure why you are paying 1 XPR as the mint fee. vivvestin has mintfeefree = 88/89 points. The mint fee should not be necessary

7/7/22 Ooops! copy n paste botch up. I did not pay mintfee of 1 XPR but transferred 1.0000 FREEBI to freeosgov2 as a CREDIT before running mintfreeos action, Have updated desc accordingly.

11/7/22 Retest. freeosgov2 v0.9.29 iteration 460419. Fixed.


[rom] update sigverify mode behavior

We update the ROM to always read the OTP setting for what sigverify mode to use (SW vs OTBN), regardless of the LC state (previously, the mode setting was only read from OTP when the LC state was NOT TEST_UNLOCKED*).
Additionally, we default the sigverify mode to SW, if the CREATOR_SW_CFG_SIGVERIFY_RSA_MOD_EXP_IBEX_EN field OTP is uninitialized (i.e., all 0s).
We make these changes because we want to run GLS tests in the TEST_UNLOCKED* LC state using the real ROM, and not enabling the use of OTBN signature verification causes GLS runtimes to explode.

CC: @msfschaffner @arnonsha
This will require us to re-release the ROM, which we will do in a followup PR.

@sha-ron
Yes, sounds good

One thing to note: SPX signature verification is always disabled in test_unlocked* states. It can only be enabled in non test LC states. We could (in the future) homogenize the  behavior of that too, i.e., make SPX always disabled by default, and then only a specific OTP value turns it on.

Another thing to note with this change is when CREATOR_SW_CFG_SIGVERIFY_RSA_MOD_EXP_IBEX_EN is:

(hardened) True or anything else --> we use Ibex signature verification, and
(hardened) False --> we use OTBN signature verification.

This makes the logic a little awkward. Ideally, it may make more sense to rename CREATOR_SW_CFG_SIGVERIFY_RSA_MOD_EXP_IBEX_EN to CREATOR_SW_CFG_SIGVERIFY_RSA_MOD_EXP_OTBN_EN and have the logic do the following:

(hardened) False or anything else --> we use Ibex signature verification, and
(hardened) True --> we use OTBN signature verification.

Unfortunately, its late/risky to make this change above. So we opted to keep things as is with this PR.


Prevent PlatformNotSupportedException to be thrown

In order to have my macOS app to break at the location where an exception is thrown, I enabled a global Exception catchpoint:

Now this causes my app to break every time I call MsalCacheHelper.CreateAsyncbecause a PlatformNotSupportedException is being thrown internally (see https://github.com/AzureAD/microsoft-authentication-extensions-for-dotnet/issues/61).
Is there perhaps a way to prevent the exception to be thrown so my app is not switching into debugging mode?

Hi @tipa - I think the root cause may be https://github.com/AzureAD/microsoft-authentication-library-for-dotnet/issues/2251 ? Which we will fix with MSAL 4.24

No, this is a different issue. It has been present since I started using the library.
The exception is being caught (see below), but I am still being thrown into Debugging mode every time and I haven't found a way in Visual Studio for Mac to prevent that
https://github.com/AzureAD/microsoft-authentication-extensions-for-dotnet/blob/2ba33180761b8782757421741239b8b0967bdff1/src/Microsoft.Identity.Client.Extensions.Msal/MsalCacheHelper.cs#L232-L237
Based on this setting in Visual Studio it should not stop however.

If you think this is a VS issue, let me know and I will post this into the Visual Studio feedback forums


Failed input item releases don't produce error logs (hard to discover through logs when an item failed)

Target components
log, workitems
What is wrong
When handling exceptions (without raising explicitly) I don't see the in the logs. (maybe feature, not a bug)
But as soon as I log.exception() the caught error, I end up with two red error messages in the log (logging the error twice).

Not sure if this is really a bug, but this stemmed from the fact that .fail() Work Items didn't produce any error in the output log and you had to unfold all the processed input items until the leaf to see what was released as FAIL and what as DONE.
System info
Platform-agnostic, used the following conda.yaml (latest versions at the moment of test):
channels:
  - conda-forge

dependencies:
  - python=3.10.12                # https://pyreadiness.org/3.10
  - pip=23.2.1                    # https://pip.pypa.io/en/stable/news
  - robocorp-truststore=0.8.0     # https://pypi.org/project/robocorp-truststore/
  - pip:
    - rpaframework==28.2.0        # https://rpaframework.org/releasenotes.html
    - robocorp==1.6.2             # https://pypi.org/project/robocorp
    - robocorp-browser==2.2.2     # https://pypi.org/project/robocorp-browser

Example script
I just used our template-python-workitems to discover the problem, mainly under these lines:
try:
            name = item.payload["Name"]
            zipcode = item.payload["Zip"]
            product = item.payload["Product"]
            print(f"Processing order: {name}, {zipcode}, {product}")
            # assert 1000 <= zipcode <= 9999, "Invalid ZIP code"
            if not 1000 <= zipcode <= 9999:
                raise ValueError(f"Invalid ZIP code: {zipcode}")  # produces error log every time (because we raise explicitly)
            workitems.outputs.create(payload={"Order": f"{name} is complete"})
            item.done()
        except ValueError as err:
            item.fail("BUSINESS", code="INVALID_ORDER", message=str(err))
            log.warn(err)  # produces one warning as expected
        except KeyError as err:
            item.fail("APPLICATION", code="MISSING_FIELD", message=str(err))
            log.exception(err)  # logs twice the exception (and nothing in its absence)

So maybe the problem here is not with the logging, but with the fact that WorkItem.fail() doesn't produce any additional error log with it. And that might be the solution here, something we already have in RPA.Robocorp.WorkItems (logging the failure as well when it gets released with such state).

Slack thread
Additionally describing the cases through the images below:


[BUG] Rebase custom image based in Secureblue - rejected by policy

Describe the bug
Custom Bluebuild image based on Secureblue-Silverblue cannot be signed:
rpm-ostree rebase ostree-unverified-registry:ghcr.io...... works
 rpm-ostree rebase ostree-image-signed:docker://ghcr.io..... does not.
To Reproduce
Setting up own Bluebuild custom image. All private and public keys are set due to BlueBuild instructions.
Expected behavior
Should be able to rebase with signed image.
Actual behavior
After running  rpm-ostree rebase ostree-image-signed:docker://ghcr.io.....
error occur:
Creating importer: Failed to invoke skopeo proxy method OpenImage: remote error: Running image docker://ghcr.io/...... is rejected by policy.
My policy.json file contains paths to public key of my repo and others, but because you have hardened it and set all other settings to reject maybe I should modify it, but how, what to change?

Could you share your policy.json? I have a similar issue (also downstream from secureblue via bluebuild) but it's a little different.

@Bammede are you rebasing to the unsigned image before the signed?
If not then /usr/etc/pki/containers/custom-name.pub won't be present, so it will fail

@qoijjj so then the issue is rebasing from the unverified custom image to the verified custom image? You should be rebasing to unverified first anyways. I'm not sure how this pertains to secureblue if you're able to switch to your unverified custom image already.

I can actually repro this on my own custom image, and I think I understand what @Bammede is saying.
For me:
Install silverblue cleanly.
Rebase to the unverified image. This completes succesfully. rpm-ostree can layer things without any problem.
At this point, though, rpm-ostree upgrade will start failing with a "signature was required, but no signature was found" error. The same issue occurs when attempting to rebase from my unverified image to my verified one.
I haven't had the time to test if this is a problem with bluebuild. I plan to recreate my images using ublue as a base and see what happens.
@qoijjj I think this should be open until we confirm that it's not secureblue-specific.

Install silverblue cleanly.
Rebase to the unverified image. This completes succesfully. rpm-ostree can layer things without any problem.
At this point, though, rpm-ostree upgrade will start failing with a "signature was required, but no signature was found" error. The same issue occurs when attempting to rebase from my unverified image to my verified one.

This doesn't happen on secureblue. If it's happening on your custom image, then either your policy.json isn't being set properly in your build, or you have local overrides. Either way it's not related to secureblue as far as I can tell.

Just to wrap this up in case anyone comes looking: the same issue occurred when I moved my image to point to ublue's silverblue. I resolved everything by recreating my custom image with the latest template. My guess is that something changed a month or so ago that messed up repos created with the old one.


Open to small refactor of DirectX::SaveToDDS ?

Hello,
I'm using this library in a particular case. I'm working with a file format looking like this
custom_header
chunk1
chunk2
chunk3
chunk4

With chunk defined as
uint8_t* data;
uint8_t start_mip;
uint8_t end_mip;

Basically, the header contains texture information. And each chunk contains several mipmaps
Now, my issue is that DirectXTex does not provide way to deal with individual mipmaps.
My changes would be the following:

Make EncodeDDSHeader public
Write a new function, SaveDDSMipmapToMemory(bool fastpath, const DirectX::Image image, It &out, const DirectX::TexMetadata &metadata)
Make this function take an output iterator, instead of a Blob
Make SaveDDSToMemory use this function, and take an output iterator instead of a blob
Other required changes for this to work

Would you accept such a Pull Request ?
Currently, I have duplicated DirectXTex code, and customized it. But obviously, it's not ideal
Thanks

The ScratchImage structure is actually laid you exactly the way DDS files are laid out, so you can just binary write out the memory it creates. If you look at the implementation of SaveDDSTo* you'll see this pretty clearly.
So mostly this is a request to make the functionality of EncodeDDSHeader public...

Thanks for this quick answer. About ScratchImage, got it. This will make it easier, thanks

So mostly this is a request to make the functionality of EncodeDDSHeader public...
Exactly


support subDAO migration

instead of executing proposal hook to execute migrate, we execute proposal hook, to execute admin execute on subDAO, to execute migrate.

specifically, the configuration is:

parent DAO is contract admin of subDAO.
subDAO is admin of its own modules.

so, the core migrate for the subSAO needs to be execute by the parent DAO, and the rest of the messages should be executed by the subDAO via the admin execute method.

cc @Art3miX, does what i outline here make sense? this is required for juno subDAOs to upgrade.

@Art3miX did we ever address this? It's closed right?

@JakeHartnell i think so, dao-migrator need core to be v2 before it can start, so any dao can upgrade core separately and then execute dao-migrator in a separate proposal.
Or run migrator with the upgrade of the core, so it should cover all cases.


How to validate subschema (like a schema in openapi)?

Hi!
I've written an OpenApi Spec for my Rest API. I want to validate the payload inside the server and the client test suit. The idea was to validate against the existing schemas inside the openapi spec.
But how do I reference a subschema? In My example I would use openapi.json/components/schemas/Test, but the referenced TestId is inside the same file. So I need to read the whole file to parse the schema provided as json path. How can I do this? If this is not possible with this library, does anyone know an alternative (preferred in c#).
[...]
"components": {
    "schemas": {
        "Test": {
            "description": "Test Object",
            "type": "object",
            "additionalProperties": false,
            "properties": {
                "firmwareVersion": {
                    "type": "string",
                    "description": "Version of the Firmware",
                    "example": "1.1.4"
                },
                "id": {
                    "$ref": "#/components/schemas/TestId"
                }
            }
        },
        "TestId": {
            "type": "string",
            "format": "uuid",
            "description": "TestId"
        }
    }
}

Hi, Not sure I got this correctly, Is it possible to use NSwag.Core to validate a Json object? NJsonSchema helped me validate the object but it would be more convenient for me to have the data model written in OpenAPI. Thank you!

if you load an OpenAPI document with the NSwag.Core then all references are resolved (but not inlined!)
The NJsonSchema JSON Schema validator based on the JsonSchema instance can handle these references.

If I read a schema from the openapi doc as schema=document.Result.Components.Schemas["schema1"] and then try to do a schemaData=schema.ToJson(), it fails if there is a referenced schema.
Error:
System.InvalidOperationException: 'Could not find the JSON path of a referenced schema: Manually referenced schemas must be added to the 'Definitions' of a parent schema.'
I believe that then loading the openapi document using NSqag.Core does not resolve the references.

Found a way out. Apparently, it is not required to use ToJson(). The idea is to use
var document = OpenApiDocument.FromJsonAsync(spec);  var schema = document.Result.Components.Schemas["schema1"]; var test=sch.Validate(jobject);
Test is a collection of validation errors.

Yep, but keep in mind that Validate() uses JSON Schema semantics and not OpenAPI semantics which might lead to slightly different results than you expect... Ideally there would be an overload of Validate() with a JsonSchemaType parameter.


Expose compact JSON and XML format configuration

What would you like to be added:
Don't add indenting/spacing to any JSON output produced by syft (not just the syftjson format).
Why is this needed:
SBOM documents generated by syft can be very large (dozens of MB), there isn't much reason to include extra spacing for each entry, then we shouldn't. If a human-friendly output is needed then JQ can be used by the consumer.

Updating to include XML formats


do we need to add a route named 'block'?

UI Version: 0.X.X
Description:
I have questions and don't know where else to have them fielded but here. I hope opening an issue for this is not a problem, if it is, please point me in the direction of where I can ask people specfic questions:
I am working on an old issue:
I am in the process of writing a spec for the goToBlock method on TmModalSearch.vue:
    gotoBlock() {
      this.$router.push({
        name: `block`,
        params: { block: this.filters.blocks.search.query }
      })

In my spec I am doing this:
wrapper.vm.type = `blocks`
wrapper.vm.filters.blocks.search.query = `test input`
wrapper.vm.gotoBlock()
expect(wrapper.vm.$route.name).toEqual(`block`)

It seems that this should work. However, this route does not seem to exist:  [vue-router] Route with name 'block' does not exist
Am I missing something here, or do we need to add this route?

Hi @coreycosman, thanks for the proactivity, I'll mark this as duplicat(ish)ed of:
https://github.com/cosmos/voyager/issues/1866
Even if the question was related to TmModalSearch


purging tool wrappers that have officially  been moved to the IUC repos

Community in charge for development now!

Discussed the spades.pl script. Good to go. :+1:


Modded game not launching

After installing the mod it gives me this error:

I did everything right, following the instructions step by step, multiple times.
Tried it on the latest steam version. (July 3)

I’m not sure unfortunately I’m not able to test on the newest version of the game that just came out today (which also upgraded the engine to Godot 3.5.2 beyond my own upgrade of the engine version (3.4) for another week or so when I get back to my computer.  But it’s a good chance the newest upgrade (which is unexpected since the dev hadn’t worked on the game for over a year when I first made the mod) broke the VR mod.  Sorry.
If I can fix it I will when I can but it’s probably not something you are doing wrong.

No problem, thanks for the reply!

So, today I tried out all the quick hacks I had hoped would work to get at least some version of the VR mod up and running and unfortunately none worked.  Sorry.  Looks like it will take significant work to get the VR mod up and running again.  No timetable.    If you look on the main page you will see there's this steam depo downloader thing where maybe you can downgrade your game back to the version before July 2023 but I haven't tried it myself yet.

That's unfortunate... thanks, I'll try downgrading the game and see if it works.

Actually give me a few days.  I made some process tonight taking a new approach and have VR working for the newest version of the game in the Godot editor at least.  Have to test tomorrow in the modded game and see what else breaks but I think I may have found the offending code that was breaking the mod.

Hi can you please try completely deleting the vr mod folders, deleting the cruelty squad install and all files left in the cruelty squad install that were put there by the last VR modloader, reinstall Cruelty Squad, then use V.1.2 of the VR mod (found here: https://github.com/teddybear082/CrueltySquadVR-Modloader/releases), installing from scratch, and let me know if it works for you?    Thank you!

It works! Thank you!


Question: How can I copy a group from one file to another?

This is possible and easy to do using h5py, however, for me python is too slow so I am trying to use rust.
I seem to just be getting stuck with the issue that the types are not known at compile time. Is there are clean way of doing this or do I need to implement a copy function for each data type? which is very annoying it that is the case. Might be missing something, but it very hard to follow as the documentation is essentially non-existant. :(
I want to essentially just have a list of keys which get copied from one file to another. It really feels like it should not be hard to do.

We do not yet wrap H5Ocopy which would allow this functionality. PRs are welcome!


GGVPointer does not honour custom raycast provider

Overview
In our project we have registered a custom raycast provider in the input system, that has a number of special rules around what is considered a valid hit. After upgrading to MRTK 2.8, we now get unwanted focus events when using the GGV pointer, as it ignores our custom raycast provider and performs raycasts directly itself.
I assume the intent was not to prevent the use of custom raycast providers for the GGV pointer specifically, but it has certainly broken our use-case
Expected behavior
Pointers should all use the custom raycast provider configured in the input system
Actual behavior
Any pointers that implement 'IMixedRealityQueryablePointer' are ignoring the registered raycast provider
Steps to reproduce
Don't have one  to share, sorry. It is obvious, however, from the method 'FocusProvider:QuerySceneWithPointer()', which accepts the raycast provider and doesn't use it.
Unity editor version
2019.4
Mixed Reality Toolkit release version
2.8.1

@keith-phillips, if there is a straight forward fix, please feel free to open a pr :smile:

@davidkline-ms I feel like there should maybe an interface change to pass the registered IMixedRealityRaycastProvider through to the methods of IMixedRealityQueryablePointer, but I don't want to go making interface changes.
I've made the most minimal fix I can, which is to resolve and use the IMixedRealityRaycastProvider in the GGVPointer methods. I tried submitting a PR but messed it up a bit, I'll try again later if I can.
The ability to register a custom raycast provider is a hugely useful extension point in the MRTK, so it would be great to keep it functioning consistently.


serenity-next: update dependencies and fix things

Let me know if the examples should be kept as is.

just ran cargo update again.

Thanks!


chore: update release notes

Explanation
Update release notes.

/cherry-pick release-0.2

/cherry-pick release-0.2


Add Code of Conduct to the website.

Steps to compelete

Add a link to the Code of Conduct to the header and footer of the base.html template in site/templates/base.html (at time of writing the path is still app/templates/base.html but this will change in the near future.)

@kjaymiller  I would it :)


external: update objectstore with hostname id rgw-endpoint is hostname

fqdn is supported in with object store,
If the rgw-endpoint is passing the fqdn
Gateway should be updated with the hostname or fqdn name

/review @iamniting @BlaineEXE

The change looks good to me, Can you pls change the commit-msg a bit, Also pls add the bug in the PR description if there is bug else pls create one to backport the changes to 4.14.

I guess  we  cannot use the same rook BZ  Bug 2189727

thanks @malayparida2000 updated

@malayparida2000 please re-start the failing test

I can't rerun them, If PR looks good from all perspective @iamniting can rerun & approve

@iamniting ^^

/backport 4.14

/cherry-pick release-4.14

/cherry-pick release-4.12
/cherry-pick release-4.13

/cherry-pick release-4.13

@parth-gr pls, create manual PR for 4.12 branch

Created: https://github.com/red-hat-storage/ocs-operator/pull/2178


Broken CI target testing?

I just tried to update the Coq Nix Toolbox in hydra-battles, to take advantage of the associated nixpkgs update (mathcomp 1.13, gaia 1.13) and when I typed nix-shell, I got the following:
these derivations will be built:
  /nix/store/vgq2rqk59qcck4hv3pr7rscg9snfipsp-coq8.14-mathcomp-fingroup-1.13.0.drv
  /nix/store/35pc7g0m8z6ws1aqp1bifg4wn46590gr-coq8.14-mathcomp-algebra-1.13.0.drv
  /nix/store/acf5z6f8x1cr3zcg8g2gxy0r0h2z5r53-coq8.14-mathcomp1.13-zify-1.1.0+1.12+8.13.drv
  /nix/store/n9sdpyaw0il7cmbqgxz5dq3ljk97631w-coq8.14-serapi-8.14+rc1+0.14.0.drv
  /nix/store/s48ryl3iaa92xavc9ddjl7pkhhrcsi0z-coq8.14-gaia-1.13.drv

It felt strange since it should already have been in coq-community Cachix.
Thus, I looked at a GitHub Action run and I got this in the log of the "Checking presence of CI target gaia" step:
these derivations will be built:
  /nix/store/dvjbl77llk15hxyn2x5jvk7swzpimy8l-coq8.14-mathcomp-ssreflect-1.13.0.drv
  /nix/store/vgq2rqk59qcck4hv3pr7rscg9snfipsp-coq8.14-mathcomp-fingroup-1.13.0.drv
  /nix/store/35pc7g0m8z6ws1aqp1bifg4wn46590gr-coq8.14-mathcomp-algebra-1.13.0.drv
  /nix/store/s48ryl3iaa92xavc9ddjl7pkhhrcsi0z-coq8.14-gaia-1.13.drv

And yet the next steps were skipped: https://github.com/coq-community/coq-nix-toolbox/runs/4050720448?check_suite_focus=true
It looks like a bug in the step to detect the presence of the derivation build result in Cachix. Related to #76? (It looks like the latter fixes the problem, but didn't push to Cachix because it was opened from a fork.)

Closed by #76.


Persistent Disk mounts are not retried on failure in GCE (and perhaps others) - makes tests flakey.

After running the standard e2e tests repeatedly against a long-living standard e2e test cluster, the following tests started to fail consistently.  Before starting to fail, they succeeded 4 times.  I'm busy gathering the cluster logs and debugging.  I assume that one or both of the nodes in the test cluster got into a state where they were unable to schedule pods, or perhaps mount the required persistent disk.
Identified problems

PD should schedule a pod w/ a RW PD, remove it, then schedule it on another host

/go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/test/e2e/pd.go:122
Expected error:
    <*errors.errorString | 0xc2090e8de0>: {
        s: "gave up waiting for pod pd-test-767f4220-f599-11e4-929a-42010af01555 to be running after 300.00 seconds",
    }
    gave up waiting for pod pd-test-767f4220-f599-11e4-929a-42010af01555 to be running after 300.00 seconds
not to have occurred

PD should schedule a pod w/ a readonly PD on two hosts, then remove both.

/go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/test/e2e/pd.go:187
Expected error:
    <*errors.errorString | 0xc209066720>: {
        s: "gave up waiting for pod pd-test-3dcc937e-f59a-11e4-929a-42010af01555 to be running after 300.00 seconds",
    }
    gave up waiting for pod pd-test-3dcc937e-f59a-11e4-929a-42010af01555 to be running after 300.00 seconds
not to have occurred

On the bright side, it's worth noting that these are the only two tests that have failed against the long-living cluster thus far, after 24 runs of 36 e2e tests each (i.e. >800 tests).  So this failure is very specific to persistent disks.  In other respects the long-living "soak" test cluster still seems perfectly healthy.

Aah yes.  Found this in kubelet.log:
I0508 05:40:54.709496    2993 event.go:203] Event(api.ObjectReference{Kind:"Pod", Namespace:"default", Name:"pd-test-b972ba0f-f544-11e4-9f96-42010af01555", UID:"c56bbef3-f544-11e4-ac4f-42010af0727d", APIVersion:"v1beta3", ResourceVersion:"2789", FieldPath:""}): reason: 'failedMount' Unable to mount volumes for pod "pd-test-b972ba0f-f544-11e4-9f96-42010af01555_default": Could not attach disk: Timeout after 10s
E0508 05:40:54.733744    2993 pod_workers.go:102] Error syncing pod c56bbef3-f544-11e4-ac4f-42010af0727d, skipping: Could not attach disk: Timeout after 10s
I0508 05:40:54.734401    2993 event.go:203] Event(api.ObjectReference{Kind:"Pod", Namespace:"default", Name:"pd-test-b972ba0f-f544-11e4-9f96-42010af01555", UID:"c56bbef3-f544-11e4-ac4f-42010af0727d", APIVersion:"v1beta3", ResourceVersion:"2789", FieldPath:""}): reason: 'failedSync' Error syncing pod, skipping: Could not attach disk: Timeout after 10s
I0508 05:40:57.714562    2993 manager.go:1242] Need to restart pod infra container for "pd-test-b972ba0f-f544-11e4-9f96-42010af01555_default" because it is not found
I0508 05:40:57.717831    2993 event.go:203] Event(api.ObjectReference{Kind:"Pod", Namespace:"default", Name:"pd-test-b972ba0f-f544-11e4-9f96-42010af01555", UID:"c56bbef3-f544-11e4-ac4f-42010af0727d", APIVersion:"v1beta3", ResourceV

Here's where the failure is happening:
https://github.com/GoogleCloudPlatform/kubernetes/blob/4a6ab7fd3698c5cc7172c5bd63efab320d1f2d17/pkg/volume/gce_pd/gce_util.go#L61

In summary we try to attach the GCE PD here:
https://github.com/GoogleCloudPlatform/kubernetes/blob/06c2f4e3d50d35e50ecb4a7909bec62d5d366875/pkg/cloudprovider/gce/gce.go#L636
If that fails, we do a somewhat optimistic check here to see whether the PD is already attached to this node/instance (according to GCE's "Get Node"):
https://github.com/GoogleCloudPlatform/kubernetes/blob/06c2f4e3d50d35e50ecb4a7909bec62d5d366875/pkg/cloudprovider/gce/gce.go#L645
It's worth noting that if the attachment of the PD fails for any reason in this process, e.g.
https://github.com/GoogleCloudPlatform/kubernetes/blob/06c2f4e3d50d35e50ecb4a7909bec62d5d366875/pkg/cloudprovider/gce/gce.go#L640
... it is never retried, as far as I can tell.
https://github.com/GoogleCloudPlatform/kubernetes/blob/b52f8656342c59f583b403faba8e5d9c61dcd4f2/pkg/volume/gce_pd/gce_pd.go#L200
Also note this juicy TODO over here:
// TODO: handle failed mounts here.
https://github.com/GoogleCloudPlatform/kubernetes/blob/b52f8656342c59f583b403faba8e5d9c61dcd4f2/pkg/volume/gce_pd/gce_pd.go#L189
I have other stuff to do right now, so I'm going to leave this one for V1.0 triage on Monday.
In summary, to fix this, we need to restructure the code to handle attach and mount failures, as there appear to be absolutely no retries in the code.

It's worth noting that the above does not completely explain away why all subsequent attempts to attach and mount a PD on this node are failing. I would suggest addressing the requirement for retries mentioned above, and at the same time improving the logging so that we can see exactly what is failing and how.
Superficially it looks like all of the functions in the call stack to attach and mount the PD to the node and pod are reporting success, but subsequent attempts to stat the mounted PD fail here:
https://github.com/GoogleCloudPlatform/kubernetes/blob/4a6ab7fd3698c5cc7172c5bd63efab320d1f2d17/pkg/volume/gce_pd/gce_util.go#L61
Two suggested strategies here:

Double-check that no possible errors are being incorrectly suppressed in the call stack.
If we fail to stat the supposedly attached and mounted PD, then do a full unmount, detach cycle, and try again to attach and mount it (a few times).

cc: @thockin who wrote the original code by the looks of things, in case he has any suggestions.

I am guilty of no such crime :)  I just moved it around, so my fingerprints
are on it.
And I have no insight to offer except to debug harder.
On Fri, May 8, 2015 at 2:36 PM, Quinton Hoole<EMAIL_ADDRESS>wrote:

cc: @thockin https://github.com/thockin who wrote the original code by
the looks of things, in case he has any suggestions.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/7972#issuecomment-100374764
.

#6399 may well be related.

/cc @swagiaal

@quinton-hoole if the disk is failing to attach it is possible that the issue is outside of Kubernetes and in GCE. Look in the logs for something like "Error syncing pod [...] skipping: googleapi: [...]"
if there is nothing like that and the disk is attached then perhaps kube is failing to see the device. Try listing the device directory on the test machine 'ls /dev/disk/by-id/". Given error you pointed out that is likely what is happening but I cannot think of why :/
BTW even though AttachAndMountDisk does not have a tight retry loop it is called again in the next sync loop until the disk is attached and mounted

@quinton-hoole: Are you still looking into this issue? (Node team triage).
cc @saad-ali

I did a little bit of code archeology yesterday, but got interrupted before I got to the bottom of the problem.  I'll unassign myself for now, and update this bug if I find anything.  I definitely won't look at it again before tomorrow.

I'm looking in to this. I am able to repro on my own cluster by deploying a cluster, verifying that the PD test passes, leaving the cluster unused overnight, and then running PD test the following morning.

The test creates a PersistentDisk (PD) when it starts. It then creates a pod with the PD as a volume which causes Kubernetes to attach the PD. The Kubernetes code attaches the PD to the node that the pod is running apparently without issue. However, although the PD appears to attach properly (confirmed in dev console UI) it never shows up under /dev/disk/by-id/, so the test times out waiting for it. Following up internally with PD eng to hopefully figure out why.
PD
  should schedule a pod w/ a RW PD, remove it, then schedule it on another host
...
STEP: creating PD
INFO: Creating new PD. pdName="e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1"
....
e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1  Standard persistent disk  10 GB us-central1-b e2e-test-saadali-minion-689a
[PD IS SUCCESSFULLY  CREATED]
...
[PD IS SUCCESSFULLY  ATTACHED TO e2e-test-saadali-minion-689a]:
e2e-test-saadali-minion-689a  us-central1-b e2e-test-saadali-minion-689a, e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1
...
689a Logs filled with :
I0529 01:22:41.633896   32203 gce_util.go:74] os.Stat("/dev/disk/by-id/google-e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1") returned error: stat /dev/disk/by-id/google-e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1: no such file or directory
I0529 01:22:41.633913   32203 gce_util.go:74] os.Stat("/dev/disk/by-id/scsi-0Google_PersistentDisk_e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1") returned error: stat /dev/disk/by-id/scsi-0Google_PersistentDisk_e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1: no such file or directory

Thanks Saad.
On Thu, May 28, 2015 at 7:59 PM, Saad Ali<EMAIL_ADDRESS>wrote:

The test creates a PersistentDisk (PD) when it starts. It then creates a
pod with the PD as a volume which causes Kubernetes to attach the PD. The
Kubernetes code attaches the PD to the node that the pod is running
apparently without issue. However, although the PD appears to attach
properly (confirmed in dev console UI) it never shows up under
/dev/disk/by-id/, so the test times out waiting for it. Following up
internally with PD eng to hopefully figure out why.
PD
should schedule a pod w/ a RW PD, remove it, then schedule it on another host
...
STEP: creating PD
INFO: Creating new PD. pdName="e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1"
....
e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1  Standard persistent disk  10 GB us-central1-b e2e-test-saadali-minion-689a
[PD IS SUCCESSFULLY  CREATED]
...
[PD IS SUCCESSFULLY  ATTACHED TO e2e-test-saadali-minion-689a]:
e2e-test-saadali-minion-689a  us-central1-b e2e-test-saadali-minion-689a, e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1
...
689a Logs filled with :
I0529 01:22:41.633896   32203 gce_util.go:74] os.Stat("/dev/disk/by-id/google-e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1") returned error: stat /dev/disk/by-id/google-e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1: no such file or directory
I0529 01:22:41.633913   32203 gce_util.go:74] os.Stat("/dev/disk/by-id/scsi-0Google_PersistentDisk_e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1") returned error: stat /dev/disk/by-id/scsi-0Google_PersistentDisk_e2e-c35e8b41-05a0-11e5-8b46-f0921cde18c1: no such file or directory
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/7972#issuecomment-106666552
.

Tracking internally with b/21597665

@saad-ali Are you still working on this?

Yep, I'll follow up with you offline.

b/21597665  (Google-internal reference)

FYI @markturansky

Saad, can you update this if you get a second?

Sure. I implemented the suggested work around in PR #9929 but we decided that was too broadscope (hitting all the devices on the system with a udevadm trigger). So I closed that and created follow-up PR #10169 which tries to be much finer scoped and also implements retries on detach which should also help with issue #9994.

These tests are still failing in our continuous soak tests.  Re-opening...
e.g. jenkins:  /job/kubernetes-soak-continuous-e2e/1845/

Looks like these failures are caused by: Pod cannot be started due to exceeded capacity
Which is not the same problem as the bugs identified and fixed in this issue or issue #9994 (namely disks not getting attached or detached).
So let's track this in a separate issue: I opened #10899


Closes #1605 Eventual proxy to send acks to server

Addressed review comments

unrelated test failure

Rebased

comments taken care of

I've confirmed that this PR fixes the memory leak


warm up

May I ask why warmup only increased the number of epochs trained, rather than increasing it with a small learning rate (similar to MASK RCNN)?

We did the linear warmup learning rate schedule in warmup epochs, please see LinearWarmupScheduler for details.

Closed due to inactivity.


[BUG] Can make transactions but, transactions are not getting synced with local blockchain

Seid version
name: sei
server_name:
version: 3.0.1-108-g3ffd72ad
commit: 3ffd72adce56b7b430f737f1f403bcbbc0531b27
build_tags: netgo ledger,
go: go version go1.18.1 linux/amd64
build_deps:

<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>=><EMAIL_ADDRESS>
Chain ID
sei-chain
Describe the bug

I have cloned sei-chain then
initialize a local node using scripts of initialize_local_chain.sh
then I haved copied and placed seid binary into /usr/bin
there, i started executing seid cli commands
the problem is, i can make transaction but chain state is not syncing

Error shows as

A few Qs:

What mode did you broadcast the tx with? sync or async? Or did you not specify?
Is the node you broadcasted against caught up to the tip of the chain? Or was it still syncing?

What mode did you broadcast the tx with? sync or async? Or did you not specify?
I just run the scripts/initialize_local_chain.sh
Is the node you broadcasted against caught up to the tip of the chain? Or was it still syncing?
I was running a local blockchain with in my system i believe using scripts/initialize_local_chain.sh

@lhtpavankumar what exact command did you execute for this transaction?

@lhtpavankumar You need to add -b block -y to your command, for example in this case:
seid tx bank send sei1pn5th7kylq47nyut0p5lcazr50rrm8rkavddeh sei1cgtvr45yyyx44kmtaq27srqru78nwcf93qu78s 1usei --chain-id sei-chain -b block


Try to reproduce but some issues occur

I run the command "./train.sh 0 se model_se"
The issue is
"""""""""""""""""""""""""""""""""
Preprocessing:   0%|                                                                                | 0/11572 [00:00<?, ?it/s]
Traceback (most recent call last):
File "src/cdiffuse/preprocess.py", line 140, in
main(parser.parse_args())
File "src/cdiffuse/preprocess.py", line 120, in main
list(tqdm(executor.map(spec_transform, filenames, repeat(args.dir), repeat(args.outdir)), desc='Preprocessing', total=len(filenames)))
File "/home/tiger/.local/lib/python3.7/site-packages/tqdm/std.py", line 1195, in iter
for obj in iterable:
File "/usr/lib/python3.7/concurrent/futures/process.py", line 476, in _chain_from_iterable_of_lists
for element in iterable:
File "/usr/lib/python3.7/concurrent/futures/_base.py", line 586, in result_iterator
yield fs.pop().result()
File "/usr/lib/python3.7/concurrent/futures/_base.py", line 432, in result
return self.__get_result()
File "/usr/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
"""""""""""""""""""""""""""""""""
How to solve this?
Although "se_pre" mode can run, with the dataset provided by your link, I MUST change the sample_rate to 48000 in params.py, otherwise this code will throw a wrong information. Does this correct for the reproduce?
Also, I run for 12 hours with 4 GPU at step 156600 for "se_pre" mode. How long(how much epoch) do we need to train your model?

Hi @yizhidamiaomiao, thanks for sharing your experience!
I've replaced torchaudio.load_wav() with the torchaudio.load() function in the new commit. This may fix some errors, as torchaudio.load_wav has been removed in newer versions of torchaudio.
For the second question, can you share a link to the data, and is the sample rate of the data you are using 48000?
Also, the "se_pre" step is no longer needed, as the randomly initialized CDiffuSE performs as well as the one initialized from pre-trained parameters. The model with step 507600 (no pre-training) in my experiments slightly exceeded our paper's results.
Please try the new code and let me know if you have any further questions!

Hi, thank you for your response!
By your instructions, I do not train the "se_pre" now. I tried to directly train your model by the command: "./train.sh 0 se model_se" and evaluate at step 600075 by the command "./inference.sh 0 600075 se model_se".
However, the training result seems different from your folder 'Sample Files'. Here is the link of the generated speech by the trained model 'weights-600075.pt' https://drive.google.com/drive/folders/1aK0zzC1wDToWIAoEq2dNSsdwSo9n9rWd?usp=sharing, which may not competitive with the SOTA model. Could you please help us find out what should we do in order to reproduce your result in 'Sample Files' ?

Hi @yizhidamiaomiao, thanks for sharing the audio file!
The command you are using seems to be from a previous commit. I've updated the command style and torchaudio functions in this commit: https://github.com/neillu23/CDiffuSE/commit/7e13e6e44294eb0d020dc63108aeb24bb39b29e0
The new command would be ". /train.sh 0 model_se" and ". /inference.sh 0 model_se 600075". Here are the results I got from my trained model 'weights-54000.pt' https://drive.google.com/drive/folders/1EIh-ZwokHcRacv20Umk9MMkld9ETdBSQ?usp=sharing.
The environment I used was torchaudio 0.9.0/ pytorch 1.9.0.
If this doesn't work for you, please let me know; thanks again!

Thank you for reporting the following results!
I think a possible reason could be the difference between our training data. You mentioned the data you used with a 48000 sampling rate but the data I used are with a 16000 sample rate. Could you share your training data and model with me so I can try if your data/model works in my environment?
Thank you again, and sorry for the inconvenience!

Thank you for reporting the following results!
I think a possible reason could be the difference between our training data. You mentioned the data you used with a 48000 sampling rate but the data I used are with a 16000 sample rate. Could you share your training data and model with me so I can try if your data/model works in my environment?
Thank you again, and sorry for the inconvenience!

I use the data directly from your link "https://datashare.ed.ac.uk/handle/10283/2791" given in the sentence "The default dataset is VOICEBANK-DEMAND dataset. You can download them from VOICEBANK-DEMAND)" in the README.md file. Actually the audio downloaded in the given website are 48k audio, and I need to write a torchaudio.resample(48k, 16k) in the function "transform" in your preprocess file to train the code.

The data I'm using is already at a 16k sample rate, which may be different from the one in the link. Could you try adding a torchaudio.resample(48k, 16k) for both "signal" and "noisy_signal" in the __getitem__ function here in NumpyDataset? If this works, I will change the description in the README. Sorry again about this issue.


[RVT-2691] Return clearer error & document when namespace_id is not provided for KV operations

Context
https://discord.com/channels/822914074136018994/1093217973197615155/1128537451758092371
From SyncLinear.com | RVT-2691

Out of sync with Linear


Allow users to import the module

This PR allows a user to import the types using
import { ig, sc } from 'ultimate-crosscode-typedefs';

This also means that you no longer need to modify your tsconfig.json when installing ultimate-crosscode-typedefs.

Btw, import 'ultimate-crosscode-typedefs' does the exact same thing but is more magic


chore(deps): upgrade dev dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-dev-dependencies-1.x" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 2330

Questions ?
Please refer to the Backport tool documentation


FIX Provide clearer credit to Simon Erkelens for his contributions

Whilst @firesphere is referenced as an author in the Composer configuration and license for this module, the 'forked' status of the repository was recently removed to reduce toil when raising PRs, which had the unfortunate side effect of masking his original implementation and contributions.
To remedy this, I've added a section to the README describing his involvement and a short description of the differences  between the implementations, and including a link to his original repository.

Isn't the correct repo for this actually https://github.com/Firesphere/silverstripe-bootstrapmfa?

The correct repo for this is actually https://github.com/Firesphere/silverstripe-bootstrapmfa.

I did not even notice that myself....


cant create Instance inside conditional component

i have another initte bug i think and i cant figure it out.
i have a component, inside page i am calling this component.  It has v-if so that component conditionally renders.
inside this component i have this command

and this is console log
instance is null as you can see

if i remove v-if it works i guess but i need the if.
i am using nuxt3 btw

Hi. If I understand correctly the v-if directive is beeing added to the sidenav component right? At the moment of calling the initTE method the sidenav does not exist yet and so does it's initializing attributes. You can try calling the initTE method after the sidenav mounts - for example with use of a watcher. Maybe using a computed value to determine the visibility of the sidenav could also work (you would have to call the initTE inside the callback, before returning a value) but I think using watcher makes a lot more sense. You can also add a if statement to check whether the component was already initialized so it won't happen twice (unless needed).
Or if thats not the case (since now I see that the getElementById returns an element), you can try delaying the initTE method inside a setTimeout method by for example 200ms and see what happens.

yes thats not the case like you said.
initte method works inside the conditionally rendered component. When v-if is true the mounted event works. that i screenshotted.  This component is not rendering when the user first go to page. It renders according to axios fetch data.

Actually i discovered something i have a menu field on the left side (this is another sidenav i have used the same example in twelements page) it is sidenav too. And it also has initte method.

Did you try to add the allowReinits option? Maybe the initTE did see the init for sidenav before and is blocking this one.
initTE({ Sidenav, Input }, { allowReinits: true });
Or you can try initializing the sidenav via JS:
new Sidenav(document.getElementById("sidenav"));

you know that second code that you wrote works perfectly. Is it have negative effect?

It should not have any negative effects. The initTE method does basically the same thing - it just looks for the data attribute inside your app that is beeing used for autoinitialization.
Good thing would be to make sure you wont initialize the sidenav twice - remove the Sidenav component from initTE method or instead of new Select call Select.getOrCreateInstance.

hi, can i get a recommendation from you?
i have multiple collapses in multiple components in 1 page
should i
async mounted() {
const { Collapse, initTE } = await import("tw-elements");
initTE({ Collapse });
},
in every component or is it better way to do that. (except slot if it is possible please)

Hi @Ischafak . The best thing would be to call the initTE({Collapse}) method inside some Parent component so that you won't have to call it in multiple places and won't risk initializing the component more than once. But if that won't work (for example, inside the parent element we do not have the element with data attribute used for initialization yet) and the component is working fine then using the initTE inside multiple component should be ok.


Add DLR native support - part 1

Channel for questions
Add DLR native jni
Description
(Brief description of what this PR is about)
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[ ] Changes are complete (i.e. I finished coding on this PR)
[ ] All changes have test coverage:
[ ] Code is well-documented:

For user-facing API changes, Java doc has been updated.
For new examples, README.md is added to explain the what the example does.

[ ] To the my best knowledge, examples and jupyter notebooks are either not affected by this change, or have been fixed to be compatible with this change

Changes

[ ] Feature1, tests, (and when applicable, Java doc)
[ ] Feature2, tests, (and when applicable, Java doc)

Comments

If this change is a backward incompatible change, why must this change be made?
Interesting edge cases to note here

Since DLR built binary is tiny, we can place the native along with the engine like SentencePiece. So we only publish the engine package.

@lanking520 talked to @frankfliu offline we still want to separate the native jar and engine jar consider we would have lots of platforms to publish.


Make entity name optional

Closes #57

With recent updates to the construction pipeline, is this PR still useful?

Before merging this, I'll change the code that accesses the name to use the CURIE as a fallback, for display purposes.


Add link to bookshelf example app

It doesn't look like there's that much documentation on how to run the app yet, so we can choose to wait on adding it to the list if that's preferable.