Make GitHub environment Scaffolder action use auth to resolve reviewers

Hey, I just made a Pull Request!
Changes the github:environment:create Scaffolder action to request and use a backend auth token when resolving the reviewer entityRefs from the Backstage catalog.
This is because previously it would throw a 401 error when backend auth was not disabled.
The logic for requesting the token is copied from the existing catalog:fetch action, which needs to do a similar thing.
Also slightly clarifies that Backstage entityRefs are expected in the reviewers list for this action.
Fixes #27515.
:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[x] Added or updated documentation
[x] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

I think that you will also need to pass through the AuthService here: https://github.com/backstage/backstage/blob/8ce0c4cd5174541cf59c20138dc6e267e5dd62d5/plugins/scaffolder-backend-module-github/src/module.ts#L75


feat: copy edits on content addressing concept guide

This PR introduces minor copy and clarity edits, plus feeds the filecorgi some kibble to keep it happy.

Image optimization came back clean!
Vuepress build was successful!

docs/concepts/content-addressing.md:

This phrase is duplicated. You should probably leave only “Content addressing”.# Content addressing Content addressing is a technique for organizing and locat...
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Use ‘own’ only if you want to emphasise that something belongs to a person or thing or was done by a person or thing. Consider writing “their”....s or even millions of people, each with their own domain in the key space. T...
                                           ^^^^^^^^^

Use ‘own’ only if you want to emphasise that something belongs to a person or thing or was done by a person or thing. Consider writing “your”....n exchange for not being able to choose your own keys, you get some valuable...
                                           ^^^^^^^^

If a new sentence starts here, add a space and start with an uppercase letter....er. To use IPFS with your NFTs, try nft.storage  (opens new window). It make...
                                           ^^^^^^^

If a new sentence starts here, add a space and start with an uppercase letter....ural artifacts that NFTs represent, nft.storage provides free storage and ba...
                                           ^^^^^^^

If a new sentence starts here, add a space and start with an uppercase letter....ore information, try the following: nft.storage documentation  (opens new wi...
                                           ^^^^^^^

Image optimization came back clean!
Vuepress build was successful!

docs/concepts/content-addressing.md:

This phrase is duplicated. You should probably leave only “Content addressing”.# Content addressing Content addressing is a technique for organizing and locat...
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Use ‘own’ only if you want to emphasise that something belongs to a person or thing or was done by a person or thing. Consider writing “their”....s or even millions of people, each with their own domain in the key space. T...
                                           ^^^^^^^^^

Use ‘own’ only if you want to emphasise that something belongs to a person or thing or was done by a person or thing. Consider writing “your”....n exchange for not being able to choose your own keys, you get some valuable...
                                           ^^^^^^^^

If a new sentence starts here, add a space and start with an uppercase letter....er. To use IPFS with your NFTs, try nft.storage  (opens new window). It make...
                                           ^^^^^^^

If a new sentence starts here, add a space and start with an uppercase letter....ural artifacts that NFTs represent, nft.storage provides free storage and ba...
                                           ^^^^^^^

If a new sentence starts here, add a space and start with an uppercase letter....mation, try the following: Official nft.storage documentation  (opens new wi...
                                           ^^^^^^^


Use Ansible Async to speed up playbook execution

https://docs.ansible.com/ansible/latest/user_guide/playbooks_async.html

Tried the following to try to install the asdf languages in parallel, but fails with "ERROR! 'poll' is not a valid attribute for a TaskInclud"
diff --git a/dotfiles/main.yml b/dotfiles/main.yml
index ba38f87..dce2fcc 100644
--- a/dotfiles/main.yml
+++ b/dotfiles/main.yml
@@ -178,11 +178,24 @@

     - name: Install ASDF language versions
       include_tasks: asdf_install.yml
-      tags: ['lang']
+      tags: ['lang', 'test']
       loop: "{{ languages | dict2items | map(attribute='key') | list }}"
+      async: 1800 # in seconds. So 30 minutes should be enough
+      poll: 0
+      register: installed_languages
       loop_control:
         loop_var: language

+    - name: Wait for language installation to complete
+      tags: ['lang', 'test']
+      async_status:
+        jid: "{{ item.ansible_job_id }}"
+      register: installing_languages_jobs
+      until: installing_languages_jobs.finished
+      retries: 360 # 360 retries with a 5 second delay, results in 1800 seconds (30 minutes)
+      delay: 5
+      loop: "{{ installed_languages.results }}"
+
     - name: Set ASDF global versions
       command: "asdf global {{ item }} {{ languages[item].global }}"
       tags: ['lang']


New version: pizzaboxer.Bloxstrap version 2.5.4

Pull request has been created with WinGet Releaser v2 :rocket:
Microsoft Reviewers: Open in CodeFlow

@wingetbot waivers Add Validation-Unattended-Failed


[5.0] Calling release on a Beanstalkd job that was buried throws an exception

I am trying to use Queued Commands/Jobs to handle delayed actions in an application. We are using Beanstalkd, and if an error happens we would like to bury the job so we can investigate and go from there.
What I have noticed is that if I bury the job and not throw an exception, the job is automatically deleted all together. I decided to solve this by burying the job and then re-throwing an exception I am catching to log errors and bury the job. This works, except for the fact the following exception is thrown:
exception 'Pheanstalk\Exception\ServerException' with message 'Job 13 NOT_FOUND: does not exist or is not reserved by client' in /home/vagrant/project/vendor/pda/pheanstalk/src/Command/ReleaseCommand.php:63

Now this doesn't break anything, but would like to remove the exception from happening altogether if possible.
Full stack trace:
exception 'Pheanstalk\Exception\ServerException' with message 'Job 13 NOT_FOUND: does not exist or is not reserved by client' in /home/vagrant/project/vendor/pda/pheanstalk/src/Command/ReleaseCommand.php:63
Stack trace:
#0 /home/vagrant/project/vendor/pda/pheanstalk/src/Connection.php(121): Pheanstalk\Command\ReleaseCommand->parseResponse('NOT_FOUND', NULL)
#1 /home/vagrant/project/vendor/pda/pheanstalk/src/Pheanstalk.php(384): Pheanstalk\Connection->dispatchCommand(Object(Pheanstalk\Command\ReleaseCommand))
#2 /home/vagrant/project/vendor/pda/pheanstalk/src/Pheanstalk.php(259): Pheanstalk\Pheanstalk->_dispatch(Object(Pheanstalk\Command\ReleaseCommand))
#3 /home/vagrant/project/vendor/laravel/framework/src/Illuminate/Queue/Jobs/BeanstalkdJob.php(88): Pheanstalk\Pheanstalk->release(Object(Pheanstalk\Job), 1024, '0')
#4 /home/vagrant/project/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(217): Illuminate\Queue\Jobs\BeanstalkdJob->release('0')
#5 /home/vagrant/project/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(159): Illuminate\Queue\Worker->process('beanstalkd', Object(Illuminate\Queue\Jobs\BeanstalkdJob), '0', '0')
#6 /home/vagrant/project/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(107): Illuminate\Queue\Worker->pop(NULL, 'release-points', '0', '3', '0')
#7 /home/vagrant/project/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(67): Illuminate\Queue\Console\WorkCommand->runWorker(NULL, 'release-points', '0', '128', false)
#8 [internal function]: Illuminate\Queue\Console\WorkCommand->fire()
#9 /home/vagrant/project/vendor/laravel/framework/src/Illuminate/Container/Container.php(523): call_user_func_array(Array, Array)
#10 /home/vagrant/project/vendor/laravel/framework/src/Illuminate/Console/Command.php(115): Illuminate\Container\Container->call(Array)
#11 /home/vagrant/project/vendor/symfony/console/Symfony/Component/Console/Command/Command.php(257): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /home/vagrant/project/vendor/laravel/framework/src/Illuminate/Console/Command.php(101): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /home/vagrant/project/vendor/symfony/console/Symfony/Component/Console/Application.php(874): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /home/vagrant/project/vendor/symfony/console/Symfony/Component/Console/Application.php(195): Symfony\Component\Console\Application->doRunCommand(Object(Illuminate\Queue\Console\WorkCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /home/vagrant/project/vendor/symfony/console/Symfony/Component/Console/Application.php(126): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /home/vagrant/project/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(94): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /home/vagrant/project/artisan(36): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 {main}

  [Pheanstalk\Exception\ServerException]
  Job 13 NOT_FOUND: does not exist or is not reserved by client

Not sure how we can fix this. It doesn't look broken to me?

I guess what I'm trying to find out is, is there anyway I can bury the job and not have the worker delete or release the job?

Or well, manually talk to beanstalkd instead of using what Laravel provides, but that would be a bummer :/

Not sure. @taylorotwell probably knows more about this than me. :)

Help me @taylorotwell, you're my only hope :P. Thanks for chiming in though @GrahamCampbell

+1

Ping @taylorotwell.

:+1:

I believe I have figured that out.
On your queue.php config file, set the ttl for something larger than the default 60 value.
		'beanstalkd' => [
			'driver' => 'beanstalkd',
			'host'   => env('BEANSTALKD_HOST'),
			'queue'  => 'default',
			'ttr'    => 600, //Example
		],

Do not forget to listen to your queue specifying a timeout equal
php artisan queue:listen --timeout=600
And in case you are executing some shell command using something like Symfony\Component\Process class, verify for timeout options when creating the object.
In my situation, I've specified it through the constructor
 $this->process = new Process($s_COMMAND, null, null, null, 600); //600 seconds for timeout
 $this->process->run();


Add a ser_json_datetime config option for configuring how datetimes are serialized.

Initial Checks

[X] I have searched Google & GitHub for similar requests and couldn't find anything
[X] I have read and followed the docs and still think this feature is missing

Description
We have a use case where we want to have datetime objects in our pydantic models, but when serialized they become unix timestamps/ millisecond unix timestamps. Heres a rough way that we currently do it:
from typing import Annotated
import datetime as dt
from pydantic import BaseModel, PlainSerializer

T_SerializedDatetime = Annotated[
    dt.datetime,
    PlainSerializer(
        lambda x: x.timestamp() * 1000,
        return_type=int,
        when_used="json",
    ),
]

class Value(BaseModel):
    series: dict[T_SerializedDatetime, float]

This works fine, however we have noticed that when we're serializing a large data series we see a lot of time spent computing the x.timestamp() * 1000 value within python, which leads to some performance issues.
What would be really great is if we could do the following:
class Value(BaseModel):
    series: dict[dt.datetime, float]

    model_config = ConfigDict(ser_json_datetime="milliseconds_float")

And in this case pydantic would do the computation of the millisecond timestamp/second timestamp within rust, thus increasing performance for us.
I would invision the modes being similar to the current state of ser_json_timedelta, being able to take iso8601, seconds_float, milliseconds_float as options, and in order to keep existing behaviour defaulting to iso8601.
Is this something that the pydantic team would support having as a feature?
Affected Components

[ ] Compatibility between releases
[ ] Data validation/parsing
[X] Data serialization - .model_dump() and .model_dump_json()
[ ] JSON Schema
[ ] Dataclasses
[X] Model Config
[ ] Field Types - adding or changing a particular data type
[ ] Function validation decorator
[ ] Generic Models
[ ] Other Model behaviour - model_construct(), pickling, private attributes, ORM mode
[ ] Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.

As a note, I'm happy to work on this feature, though may need a bit of guidance in this (i know bits and bobs now from the work on ser_json_timedelta, but as this would be a new config option im guessing theres more to consider!!)

Some kind of option on this line seems very reasonable to me, thanks!
FWIW we currently infer the seconds / microseconds split for timestamps - see https://github.com/pydantic/pydantic/issues/7940 and https://github.com/pydantic/speedate/issues/14
I think any implementation / design here strongly wants that complementary problem solved so that you don't fall into edge cases when deserializing. Perhaps there's room for a pair of options ser_json_datetime='float' and then a config mode for timestamp_unit='milliseconds' (with the config affecting the unit for both directions)? Open to opinions🤔

Hmm, given that we just added milliseconds_float, seconds_float options for ser_json_timedelta, perhaps we should just use a consistent pattern here?

Thanks, glad its something worth considering!
I think i'd agree with the milliseconds_float, seconds_float (probably int in this case?) to keep a consistent pattern

I've done a very rough implementation in a draft PR for pydantic core which uses this method anyway so somewhat bias, but happy to change depending on what you guys feel

I've set the PR i have done with this implementation as ready to review, but welcome any design feedback if we want to change this 👍🏻

I've set the PR i have done with this implementation as ready to review, but welcome any design feedback if we want to change this 👍🏻

Awesome! Will review tomorrow :)

Hi @sydney-runkle , did you get a chance to have a look?

Alright, so we've discussed a bit of a switch up here re the API. This piggy backs off of @davidhewitt's comment above.
The new API we're proposing
Config settings:

val_timestamp_unit: 'seconds', 'milliseconds', or 'infer', defaults to 'infer'
ser_timestamp_unit: 'seconds', 'milliseconds', or 'infer', defaults to 'infer'
ser_json_timedelta = 'float' or 'iso8601'
ser_json_datetime = 'float' or 'iso8601'

Unfortunately, this will make it such that some of the work from your previous PR is reverted 😞, at least in terms of config setting changes. We can likely reuse a lot of the internal logic, and likewise for your current pydantic-core PR.
This change will also make it such that we resolve https://github.com/pydantic/pydantic/issues/7940. You can see some related work already done here: https://github.com/pydantic/pydantic-core/pull/1052.
Additionally, this should fix some issues re round trips for cases around our current watershed.
Apologies for this change up after you've done some of the work to implement support. We would of course love your continued help with development here, if you're still interested.
Would love to hear your thoughts on this API given that you've requested support for customization like the above!

@sydney-runkle  Makes sense, i may be a little constrained for time soon but i'm hoping I can get this through for you guys for the 2.10 release (Or at least, I would love to see it in that release!!)
I may have some follow up questions on this once, i dive into the details. I'll work on the pydantic core side first ofc. Do we envisage any changes needed to speeddate?

@ollz272,
Great! We're super excited to have you contributing :). Looking forward to some more collaboration.

Hey @ollz272, hope you're having a good weekend. Any estimate as to when you might be able to pick this up?

@sydney-runkle apologies, i have a feeling that with new work and life commitments im not going to be able to get round to this any time soon

@ollz272,
No worries. I'll pick up these changes. Thanks for your help on this front - definitely kickstarted our movement to a better API + more extensive functionality.


How to Predict 3D Faces Using Models?

I'm glad to see your work. I'm just interested in 3D face detection. Under the model folder, I saw some of the models you provided. I want to achieve the following effects:

But I don't know how to use the model you provide to build prediction code. Can you give me some guidance? Thank you very much.

The 3D face model is significantly more difficult than the facedetector_front.tflite we have in the model folder. We recommend that you read the publication for Face Mesh
What is the use case you are looking at for 3D face detection?

@mgyong
Sorry, maybe the meaning of the expression is wrong. I actually want to use the model you provided to do face detection. I want to get the following detection effect:

But I don't know how to use your model and build prediction code. Can you help me?

@zhouhao-learning Have you gone through the Face detection for Android running GPU example?
If you have issues with understanding some of the Android APIs for MediaPipe, you can start with simpler example

No, I didn't run the Face detection for Android running GPU example. I don't want to run on Android right now. I just want to write a predictive code that can achieve the face alignment effect, but I don't know how to build the prediction code. Can you help me? Thank you.

@zhouhao-learning This 3D face mesh issue is out of scope for project currently

@mgyong
No, if it is beyond the scope of the project, how do you achieve the following effects?

@zhouhao-learning If you are interested in getting the effects by any means, you might find it useful to know that this 3D face mesh technology is exposed via the Augmented Faces API in ARCore.

@kartynnik
Thank you for sharing. I'm still looking for ways to implement Python


Create new Lambda repo: spacecat-content-import-worker

This task is a request for a new Github repo to be created as spacecat-content-import-worker at https://github.com/adobe/spacecat-content-import-worker

@blefebvre created repo, added you as admin and spacecat team as collaborators. suggest to create a spacecat-importer team with your team for additional collaboration assignment.


Fix doxygen-related documentation issues

JerryScript-DCO-1.0-Signed-off-by: Akos Kiss<EMAIL_ADDRESS>

LGTM


Improve the SEO of the project site

The project needs to have updated tags and descriptions so that it improves the SEO of the project.

I would like to work on it

@kumarshobhit The official coding period starts from 1st Dec 2020 i.e tomorrow. I'm assigning to you for now, but remember to raise the PR tomorrow, else it will stand against the rules & regulations of NWoC 2020 & won't be counted!
Best of Luck :+1:

Thank you :)


EV target SOC	not updated

v1.5.2
Describe the bug
The EV target SOC (attibute of EV Smart Charging Charging) is not updated when i change the number.ev_smart_charging_minimum_ev_soc

The EV target SOC and the number.ev_smart_charging_minimum_ev_soc are not the same.
The EV target SOC is the highest SOC level that charging ever should reach.
The number.ev_smart_charging_minimum_ev_soc is used in combination with switch.ev_smart_charging_apply_price_limit to make sure that a certain minimum SOC level is reached even if the prices are high.


segfault with custom input device

I developed a new module according to SDL_mouse.c, which is used to implement input devices through the network. When I am running, they are all normal. The button can be touched and the normal touch state changes, but when I pass the button through btn.set_event_cb (event_handle) After binding the event, click the button again, micropython has a segfault, what is the cause? (The same mpy code runs normally with the SDL module, and the event_handle runs successfully). Thank you for your help.
New module link
modnid.c

Please run with debugger and attach stack trace.

Closing due to inactivity.
Please reopen if you have new inputs.


Get location button removes other pins

Figure out what's doing this and remove it. Probably an option within leaflet.locator

Fixed https://github.com/Project-OSRM/osrm-frontend-v2/commit/bca3cf84bd76c044a78661eb2c83d87390c8f6d8


Compiling non-nearest neighbor 2-qubit gate

For instance, if we want to compile a CNOT gate between qubit (0, 0) and qubit (1, 1) to Foxtail.
import cirq
circuit = cirq.Circuit()
circuit.append([cirq.CNOT.on(cirq.GridQubit(0,0), cirq.GridQubit(1,1))])
cirq.google.optimized_for_xmon(circuit, new_device=cirq.google.Foxtail)

It raises an error ValueError: Non-local interaction: GateOperation(Exp11Gate(half_turns=1.0), (GridQubit(0, 0), GridQubit(1, 1))).
Is there a way to automatically add SWAP gates to the circuit, or other automatic methods, to compile the non-nearest neighbor 2-qubit gate?

Is there a way to automatically add SWAP gates to the circuit, or other automatic methods, to compile the non-nearest neighbor 2-qubit gate?

Yes. But if we are talking about NISQ, this is an expensive operation. What we could do is "fake" CNOT((0,0), (1,1)) by compiling it qubits that are local  CNOT((0,0), (1,1)) compiles to CNOT((0,1), (1,1)) . Maybe add a flag for it to make it explicit.

Yes. But if we are talking about NISQ, this is an expensive operation. What we could do is "fake" CNOT((0,0), (1,1)) by compiling it qubits that are local CNOT((0,0), (1,1)) compiles to CNOT((0,1), (1,1)) . Maybe add a flag for it to make it explicit.

I agree with you that those SWAP operations are expensive, and mapping the qubits alternatively is definitely a better solution for the simple example I quoted.
Nonetheless, I am encountering situations with multiple 2-qubit gates and simply re-mapping cannot solve the problem of nonlocal 2-qubit gates. SWAP gates are unavoidable in those situations, and it would be nice if the compiler can adapt to those cases. The compiled circuits may be beyond the reach of near-term devices, but it is still useful to get some ideas on the gate count and circuit depth.

You might be interested in the cirq.contrib.acquaintance module. It doesn't automatically insert swaps to implement non-local gates, but rather gives a way of specifying an "acquaintance strategy" that swaps qubits around so that they're locally adjacent when you need them to be. If you know both the structure of the circuit and hardware, you can often do this in a way that doesn't introduce too much overhead, i.e. could be feasible on NISQ devices.

@bryano Thanks! Will try the module.

At the moment, we do not ever insert swap operations for the user. The reasoning is that this drastically changes the cost of the circuit, and we wouldn't be able to do a good job of it anyways. If you want to test non-local circuits, the current way to do that is by not specifying a device.
All that being said, I guess we could add something like a "RemoteCzViaSwaps" gate. Hmm..

For reference, here is a code to apply a gate across a path of qubits. If the path has length L, this circuit has depth L+7 (not accounting for CZ adjacency constraints).
def apply_gate_across_path(path: Sequence[cirq.QubitId],
                           gate: cirq.Gate) -> cirq.OP_TREE:
    k = len(path) // 2
    left, right = path[:k], path[k:][::-1]

    left_pairs = list(zip(left, left[1:]))
    right_pairs = list(zip(right, right[1:]))
    pairs = left_pairs + right_pairs
    kick_forward = [cirq.CNOT(a, b) for a, b in pairs],
    kick_backward = [cirq.CNOT(b, a) for a, b in pairs]
    matrix = cirq.unitary(gate, None)
    is_diagonal = matrix is not None and cirq.is_diagonal(
        matrix, tolerance=cirq.Tolerance(atol=0.00001))
    meetup = ([kick_backward, kick_forward]
              if is_diagonal
              else [kick_forward, kick_backward, kick_forward])

    return cirq.Circuit.from_ops(
        meetup,
        gate.on(left[-1], right[-1]),
        cirq.inverse(meetup),
        strategy=cirq.InsertStrategy.EARLIEST,
    ).all_operations()

c = cirq.Circuit.from_ops(apply_gate_across_path(cirq.LineQubit.range(10), cirq.SWAP))
print(c)

0: ───@───────X───────@───────────────────────────────@───────X───────@───
      │       │       │                               │       │       │
1: ───X───@───@───X───X───@───────────────────────@───X───X───@───@───X───
          │       │       │                       │       │       │
2: ───────X───@───@───X───X───@───────────────@───X───X───@───@───X───────
              │       │       │               │       │       │
3: ───────────X───@───@───X───X───@───────@───X───X───@───@───X───────────
                  │       │       │       │       │       │
4: ───────────────X───────@───────X───×───X───────@───────X───────────────
                                      │
5: ───────────────X───────@───────X───×───X───────@───────X───────────────
                  │       │       │       │       │       │
6: ───────────X───@───@───X───X───@───────@───X───X───@───@───X───────────
              │       │       │               │       │       │
7: ───────X───@───@───X───X───@───────────────@───X───X───@───@───X───────
          │       │       │                       │       │       │
8: ───X───@───@───X───X───@───────────────────────@───X───X───@───@───X───
      │       │       │                               │       │       │
9: ───@───────X───────@───────────────────────────────@───────X───────@───

d = cirq.Circuit.from_ops(apply_gate_across_path(cirq.LineQubit.range(10), cirq.CZ**0.5))
print(d)

0: ───X───────@───────────────────────────────────@───────X───
      │       │                                   │       │
1: ───@───X───X───@───────────────────────────@───X───X───@───
          │       │                           │       │
2: ───────@───X───X───@───────────────────@───X───X───@───────
              │       │                   │       │
3: ───────────@───X───X───@───────────@───X───X───@───────────
                  │       │           │       │
4: ───────────────@───────X───@───────X───────@───────────────
                              │
5: ───────────────@───────X───@^0.5───X───────@───────────────
                  │       │           │       │
6: ───────────@───X───X───@───────────@───X───X───@───────────
              │       │                   │       │
7: ───────@───X───X───@───────────────────@───X───X───@───────
          │       │                           │       │
8: ───@───X───X───@───────────────────────────@───X───X───@───
      │       │                                   │       │
9: ───X───────@───────────────────────────────────@───────X───

Understand. Thanks for the explanation and the sample code 👍

Placement is challenging, and the acquaintance module in contrib is one answer to this.  The question is whether we want other simple solutions for this for uses who want to build these circuits.  I think @Strilanc suggestion is good and don't see a reason why this basic primitive should be put into device.

Paging @MichaelBroughton and @tanujkhattar: I feel like between Devices and Transformers this should have been resolved.

Discussed offline: qubit placement is now the responsibility of Feature Testbed. At lower levels, specific qubits chosen do not matter.


Prefix permission constraints are bypassed when using the available-prefixes API

Environment

Nautobot version (Docker tag too if applicable): 2.3.12
Python version: 3.11
Database platform, version: PG16
Middleware(s):

Steps to Reproduce

Create a Prefix, e.g <IP_ADDRESS>/24 with description ThisDescriptionIsMandatory.
Create a new user and set his permissions to view all prefixes
Create a custom add/edit/delete Prefix permissions for this user with the constraint {"description": "ThisDescriptionIsMandatory"}
As this new user, query the available-prefixes API endpoint on the newly created prefix <IP_ADDRESS>/24 and request a new child prefix with any random description, e.g. test
Verify the Prefix is created successfully

Expected Behavior
The new user should have gotten an object-level permission error, since the newly created prefix (child of <IP_ADDRESS>/24) does not respect the Prefix add/edit/delete permission constraint assigned to the user - i.e. the description of the newly created prefix is NOT ThisDescriptionIsMandatory.

Observed Behavior
The new child prefix was created not respecting the permission constraints.
This same user is NOT allowed to create that same prefix in the GUI, but the available-prefixes endpoint allows it.
Besides, by browsing this new child prefix in the GUI, the user can't delete or edit it - which is expected since the description doesn't match the constraints. But if this is the case, the user shouldn't have been able to create it.

Same issue likely applies to available-ips and available-vlans endpoints.


docs: Clarify availability of analytics

This commit tweaks the language in Analytics.md to make it clear
that summaries of install/error data are publicly available,
while only Homebrew maintainers can access the full analytics
(as specified above in the document).

Thanks @woodruffw!

This is an important clarification to make. Good work @woodruffw! 👍


[Somebody interested?] Bigger battery on board

It's just an idea, not yet as sophisticated as your work ;)
But fixing an empty battery within 5 seconds replacing it, putting the empty one into the charger and use it for weeks/months before you need to repeat? Sounds good to me.
https://www.amazon.de/-18650-battery-holder/dp/B081HW4TGR
Pinta crashed while painting mockup :'( So here's the text-only-edition:
I'd place the battery right above the first thumb key with enough space for an OLED, extend the board and screw the holder to it.
What do you think?

Better solutions are appreciated.

Hey I did this! With my placement it fills the space underneath the palm, and stays out of the way of the thumb. I didn't want it on the inside between the boards, because I prefer to use that space for a mouse.

It doesn't look very nice and I made a 3d model for doing it properly, which I can upload when I find it.

Looks promising :) thanks for sharing

I've used 3000mAh flat one.
At the top back that give some tenting 😉

(The keyboard is a sofle choc but i've done the same thing on my dailly hillside)


Mp4 Bug

Describe your bug here.

Command Prompt/Terminal logs (if existing)
vlc/MP4Handler.hx (line 81)
vlc/bitmap/VlcBitmap.hx (line 480)
vlc/bitmap/VlcBitmap.hx (line 246)
vlc/bitmap/VlcBitmap.hx (line 331)
openfl/events/EventDispatcher.hx (line 402)
openfl/display/DisplayObject.hx (line 1399)

Uncaught Error: VLC caught an error!
Please report this error to the GitHub page: https://github.com/ShadowMario/FNF-PsychEngine

> Crash Handler written by: sqirra-rng

Are you modding a build from source or with Lua?

Windows
Did you edit anything in this build? If so, mention or summarize your changes.
Idk

Send the video file on what's causing it

Send the video file on what's causing it

All mp4 videos won't play in Cutscene.

Send the video file on what's causing it

My PC doesn't crash, but other people's PCs crash.


More fixes for 1.0

Some additional fixes on top of #68. Passes tests locally. Thanks again, @miguelraz.

Womderful!

I am liking more and more the style of explicitly overloading methods from other modules.
Thanks very much for cleaning this package up. This should put us in a good place.


Documentation updates for .NET 9

.NET 9 will be released for general availability during November 12-14, 2024. These are the readme changes required to match the new version.

[x] Test .NET 9 Runtime on Raspbian 10, 11, and maybe 12 (at least on ARM32, where the libc6 requirement is higher)

.NET 8 and 9 require a higher version (2.35) of ARM32 libc6 than Debian 10 and 11 provide (2.28 and 2.31). However, I've been running .NET 8 on multiple Debian 11 ARM32 installations for about a year and it's been perfectly stable and compatible, so this requirement may be overblown and merits testing.

[x] Add .NET 9 to Compatible versions > Operating systems and .NET releases

.NET 9 removes support for Debian 11, so update the table and add a footnote
Maybe also clarify footnotes to illustrate which packages this repository actually contains, because "Incompatible" doesn't mean "Nonexistent."

[x] Add .NET 9 to example version tables

Installation > Install package > package name format
Installation > Install package > specific minor version
To find these locations in Readme.md, you can search for <!-- Add new releases here -->

libc6 dependencies

.NET version
ARM32 min libc6
ARM64 min libc6

6
⚠ 2.23
2.23

7
⚠ 2.27
2.23

8
2.35
2.23

9
2.35
2.23

Raspbian
ARM32 libc6
ARM64 libc6

10
⚠ 2.28
2.28

11
⚠ 2.31
2.31

12
2.36
2.36

Committed to new net9.0 branch, to be merged into master when .NET 9 packages are available.

.NET Runtime 9.0 RC2 fails to run on Debian 10 armhf with an error that it requires glibc 2.34, but 2.28 is installed. This means .NET 9 armhf packages should not be listed for Debian 10. However, .NET 8 runs fine on the same computer, so the stated system requirements are all lies.

This just updates the documentation for the new .NET 9 system requirements. The packages themselves will be generated differently in #28.


Fixed android 11+ permission issues

Added permission check and asking feature if required, on scan click which fixes the permission issues

Since permissions will be added through a different PR (and the current PR contains personal project files), I'll close the PR for now.


ValueError: buffer is smaller than requested size

I'm trying to convert this bodypix model (resnet50, stride 32). It was already mentioned in #1 but this seems to be a different error:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tfjs_graph_converter/converter.py", line 137, in main
    convert(argv[0].split(' '))
  File "/usr/local/lib/python3.6/dist-packages/tfjs_graph_converter/converter.py", line 107, in convert
    api.graph_model_to_frozen_graph(args.input_path, args.output_path)
  File "/usr/local/lib/python3.6/dist-packages/tfjs_graph_converter/api.py", line 217, in graph_model_to_frozen_graph
    graph = load_graph_model(model_dir)
  File "/usr/local/lib/python3.6/dist-packages/tfjs_graph_converter/api.py", line 204, in load_graph_model
    return _convert_graph_model_to_graph(model_json, model_dir)
  File "/usr/local/lib/python3.6/dist-packages/tfjs_graph_converter/api.py", line 177, in _convert_graph_model_to_graph
    weight_list = read_weights(weights_manifest, base_path, flatten=True)
  File "/usr/local/lib/python3.6/dist-packages/tensorflowjs/read_weights.py", line 74, in read_weights
    return decode_weights(weights_manifest, data_buffers, flatten=flatten)
  File "/usr/local/lib/python3.6/dist-packages/tensorflowjs/read_weights.py", line 189, in decode_weights
    value = _deserialize_numeric_array(data_buffer, offset, dtype, shape)
  File "/usr/local/lib/python3.6/dist-packages/tensorflowjs/read_weights.py", line 123, in _deserialize_numeric_array
    offset=offset).reshape(shape)
ValueError: buffer is smaller than requested size

I've downloaded the weights with
for i in {1..23}; do wget https://storage.googleapis.com/tfjs-models/savedmodel/bodypix/resnet50/float/group1-shard${i}of23.bin; done

Versions used are
tfjs_graph_converter 0.2.0
Dependency versions:
    tensorflow 1.15.0
    tensorflowjs 1.3.2

Please check the sizes of the downloaded weight files.
You'll get this error if the weight files are invalid.
The files should have a size of exactly 4MB (4096kB), except for the last file (group1-shard23of23,bin), which should be a little smaller.
The problem stems from wget not decompressing the downloaded files (the server delivers the files gzip-compressed). I don't know the correct switch for wget, but this curl command should do the trick:
curl -O --compressed https://storage.googleapis.com/tfjs-models/savedmodel/bodypix/resnet50/float/group1-shard[1-23]of23.bin

Ah, yes, you are correct! wget messed up. It works now. Thank you


Update versions to 1.0.0 in environment scripts -- master branch

arm64_env.sh
env.sh

Images pointed to 0.7.0 need to be pointed to 1.0.0
Also the scheduler image name needs to be corrected:

arm64_env = docker-support-scheduler-go-arm64
env = docker-support-scheduler-go

This needs to be done soon as the PR to update edgex-go has been created.

@FelixTing please take care of this issue

1.0.0 images doesn't appear in the repository

PR to rev edgex-go to 1.0.0 was merged today. Images have been pushed to nexus staging
https://nexus3.edgexfoundry.org/#browse/browse:docker.staging:v2%2Fdocker-core-data-go%2Ftags%2F1.0.0

@tsconn23 will we have v1.0 for current support-rulesengine?  if not, we don't need to change its version to v1.0.


Propagate OpenTelemetry context in tasks

When using OpenTelemetry to create trace spans, the context is not automatically propagated to child tasks. This results in spans that have no correlation with their parent, which makes it worthless to trace.
One could wrap everything by hand (for instance, saving the original OTel context in a field of a source, and attaching to it in its run/1 protocol implementation) but it is repetitive and error-prone.
This change automatically attaches the context to every spawned task, so it is already available for creating spans with
OpenTelemetry.Tracer.with_span("My span") do
  # inside a span
end

or to open and close spans in :telemetry handlers (which are run in the child processes)
See also the following PR that adds the same support to absinthe.

absinthe-graphql/absinthe#1227

@benwilson512 the type issues are solved. Do you think it can be merged?

This looks great, thanks!


onClick no longer works in firefox?

Notification shows, but clicks do nothing. Cannot achieve anything in the onClick callback, not even simple logging.  FF 67.0.4 mac

Ended up needing to revoke notification permission and re-add. No idea why.


mucommander: 1.2.0-1 -> 1.3.0-1

Description of changes

https://github.com/mucommander/mucommander/releases/tag/1.3.0-1
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 246596 run on aarch64-darwin 1

1 package built:

mucommander


Strange behavior within transition-group

Version
2.6.11
Reproduction link
https://jsfiddle.net/yrqt02kj/1/
Steps to reproduce
transition-group  renders the component twice (and the second rendering is done without parsed "{{variable}}").
If we just remove transition-group parent at all, it works as expected and there is no duplicated content.
What is expected?
there should not be the duplicated content (with unrendered variable tag)
What is actually happening?
it shows the content twice

Hi, thanks for your interest but Github issues are for bug reports and feature requests only. You can ask questions on the forum, the Discord server or StackOverflow.

You must but testt's template outside of #app so Vue doesn't use it

Hi.
I've asked that on discord channel but got no response, I thought it was bug and opened here. ok, thanks , just your suggestion is not actually direct answer to the problem i described.


Update README.md

Added hyphens to bullet points to make them match the 'Special Thanks' style and a proper comma

Please update this PR to target the develop branch, thanks!

@ajmeese7 Can you submit this against the develop branch?  If not, before v19 I'll do it :) But I'd be happy to have your name get the credit for helping out the library.


[Event Request] Codeunit 5817 "Undo Posting Management" - OnUpdateServLineAfterInitQtyToShip

Hi, would it be possible to get an event OnUpdateServLineAfterInitQtyToShip in codeunit 5817(Undo Posting Management), like this:
procedure UpdateServLine(ServLine: Record "Service Line"; UndoQty: Decimal; UndoQtyBase: Decimal; var TempUndoneItemLedgEntry: Record "Item Ledger Entry" temporary)
    var
        xServLine: Record "Service Line";
        ReserveServLine: Codeunit "Service Line-Reserve";
    begin
        with ServLine do begin
            xServLine := ServLine;
            case "Document Type" of
                "Document Type"::Order:
                    begin
                        "Quantity Shipped" := "Quantity Shipped" - UndoQty;
                        "Qty. Shipped (Base)" := "Qty. Shipped (Base)" - UndoQtyBase;
                        "Qty. to Consume" := 0;
                        "Qty. to Consume (Base)" := 0;
                        InitOutstanding;
                        InitQtyToShip;
                        OnUpdateServLineAfterInitQtyToShip(ServLine);
                    end;
                else
                    FieldError("Document Type");
            end;
            Modify;
            RevertPostedItemTracking(TempUndoneItemLedgEntry, "Posting Date");
            xServLine."Quantity (Base)" := 0;
            ReserveServLine.VerifyQuantity(ServLine, xServLine);

            UpdateWarehouseRequest(DATABASE::"Service Line", "Document Type", "Document No.", "Location Code");

            OnAfterUpdateServLine(ServLine);
        end;
    end;

[IntegrationEvent(false, false)]
local procedure OnInsertGLInvtReportEntryBeforeCalculateCostAmount(var ServLine: Record "Service Line")
begin
end;

Thanks

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


[HUDI-1552] Improve performance of key lookups from base file in Metadata Table.

What is the purpose of the pull request
Improves the performance of key lookups from Metadata Table.
In my scale testing with 150 partitions and 100K+ files on HDFS, the time to read the key was reduced (100ms avg -> 10ms) and the total data read from the HFile was reduced (85MB -> 3MB). The size of the base file was 3MB so this means that the in-memory HFile block caching was also working.
Brief change log

Cache the KeyScanner across lookups so that the HFile index does not have to be read for each lookup.
Enable block caching in KeyScanner.
Move the lock to a limited scope of the code to reduce lock contention.

Verify this pull request
This pull request is already covered by existing tests, such as (please describe tests).
mvn test -pl  hudi-client/hudi-spark-client -Dtest=TestHoodieBackedMetadata
Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Codecov Report

Merging #2494 (19894f6) into master (c8ee40f) will decrease coverage by 40.49%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master   #2494       +/-   ##
============================================
- Coverage     50.18%   9.68%   -40.50%
+ Complexity     3051      48     -3003
============================================
  Files           419      53      -366
  Lines         18931    1930    -17001
  Branches       1948     230     -1718
============================================
- Hits           9501     187     -9314
+ Misses         8656    1730     -6926
+ Partials        774      13      -761

Flag
Coverage Δ
Complexity Δ

hudicli
?
?

hudiclient
?
?

hudicommon
?
?

hudiflink
?
?

hudihadoopmr
?
?

hudisparkdatasource
?
?

hudisync
?
?

huditimelineservice
?
?

hudiutilities
9.68% <ø> (-59.80%)
0.00 <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

The diff coverage is n/a.

@@             Coverage Diff              @@
##             master   #2494       +/-   ##
============================================
- Coverage     50.18%   9.68%   -40.50%
+ Complexity     3051      48     -3003
============================================
  Files           419      53      -366
  Lines         18931    1930    -17001
  Branches       1948     230     -1718
============================================
- Hits           9501     187     -9314
+ Misses         8656    1730     -6926
+ Partials        774      13      -761

Flag
Coverage Δ
Complexity Δ

hudicli
?
?

hudiclient
?
?

hudicommon
?
?

hudiflink
?
?

hudihadoopmr
?
?

hudisparkdatasource
?
?

hudisync
?
?

huditimelineservice
?
?

hudiutilities
9.68% <ø> (-59.80%)
0.00 <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Trying to understand this part. Was the workload, trying to fetch all the keys out of the HFile or just 1?

The size of the base file was 3MB so this means that the in-memory HFile block caching was also working.

Trying to understand this part. Was the workload, trying to fetch all the keys out of the HFile or just 1?

The size of the base file was 3MB so this means that the in-memory HFile block caching was also working.

Trying to understand this part. Was the workload, trying to fetch all the keys out of the HFile or just 1?

The workload was a commit followed by a Clean operation with num_versions_retained=1 so it will clean all partitions. Hence, number of key lookups should be equal to number of partitions and all the keys should have been read from the HFile.

The size of the base file was 3MB so this means that the in-memory HFile block caching was also working.

Trying to understand this part. Was the workload, trying to fetch all the keys out of the HFile or just 1?

The workload was a commit followed by a Clean operation with num_versions_retained=1 so it will clean all partitions. Hence, number of key lookups should be equal to number of partitions and all the keys should have been read from the HFile.

With enableReuse=false, the caching of readers needs special handling because:

Multiple threads may call into HoodieBackedTableMetadata.getRecordByKeyFromMetadata() to read their respective keys
If enableReuse=false, then each of these threads will try to close the readers after reading the key

Hence, we essentially have two codepaths:

enableReuse=false then  readers cannot be cached
enableReuse=true then the readers can be cached.

I have updated the patch to handle both these cases by modifying the openFileSliceIfNeeded function (renamed to getReader) which returns either:

cached readers when enableReuse=true
newly opened readers when enableReuse=false

With enableReuse=false, the caching of readers needs special handling because:

Multiple threads may call into HoodieBackedTableMetadata.getRecordByKeyFromMetadata() to read their respective keys
If enableReuse=false, then each of these threads will try to close the readers after reading the key

Hence, we essentially have two codepaths:

enableReuse=false then  readers cannot be cached
enableReuse=true then the readers can be cached.

I have updated the patch to handle both these cases by modifying the openFileSliceIfNeeded function (renamed to getReader) which returns either:

cached readers when enableReuse=true
newly opened readers when enableReuse=false

@vinothchandar  PTAL.

@prashantwason We can remove the reuse configuration - i.e no need to have this behavior be user controlled.
but ultimately, we still need to close everything out,  where metadata table is opened from executors. I am going to just introduce a boolean variable within HoodieBackedTableMetadata

Looks good @vinothchandar


Failed to deploy testwebapp

Hello experts,
I created WebLogic domain on pv and got error while deploying testwebapp.
Below is information of steps:
The WebLogic Operator version: v3.0.0
I used script in kubernetes\samples\scripts\create-weblogic-domain\domain-home-on-pv\create-domain.sh, the domain was created successfully with the follow output:
Creating weblogic domain domain1
Input parameters being used
export version="create-weblogic-sample-domain-inputs-v1"
export adminPort="7001"
export adminServerName="admin-server"
export domainUID="domain1"
export domainHome="/shared/domains/domain1"
export serverStartPolicy="IF_NEEDED"
export clusterName="cluster-1"
export configuredManagedServerCount="5"
export initialManagedServerReplicas="2"
export managedServerNameBase="managed-server"
export managedServerPort="8001"
export image="store/oracle/weblogic:<IP_ADDRESS>"
export imagePullPolicy="IfNotPresent"
export imagePullSecretName="haicheregcred"
export productionModeEnabled="true"
export weblogicCredentialsSecretName="domain1-weblogic-credentials"
export includeServerOutInPodLog="true"
export logHome="/shared/logs/domain1"
export httpAccessLogInLogHome="true"
export t3ChannelPort="30012"
export exposeAdminT3Channel="false"
export adminNodePort="30701"
export exposeAdminNodePort="true"
export namespace="default"
javaOptions=-Dweblogic.StdoutDebugEnabled=false
export persistentVolumeClaimName="haicheazurefile"
export domainPVMountPath="/shared"
export createDomainScriptsMountPath="/u01/weblogic"
export createDomainScriptName="create-domain-job.sh"
export createDomainFilesDir="wlst"
export serverPodMemoryRequest="768Mi"
export serverPodCpuRequest="250m"
export istioEnabled="false"
export istioReadinessPort="8888"

Generating /home/haiche/azure/weblogic-domains/domain1/create-domain-job.yaml
Generating /home/haiche/azure/weblogic-domains/domain1/delete-domain-job.yaml
Generating /home/haiche/azure/weblogic-domains/domain1/domain.yaml
Checking to see if the secret domain1-weblogic-credentials exists in namespace default
Checking if the persistent volume claim haicheazurefile in NameSpace default exists
The persistent volume claim haicheazurefile already exists in NameSpace default
W0810 09:51:07.364919    4841 helpers.go:535] --dry-run is deprecated and can be replaced with --dry-run=client.
configmap/domain1-create-weblogic-sample-domain-job-cm created
Checking the configmap domain1-create-weblogic-sample-domain-job-cm was created
configmap/domain1-create-weblogic-sample-domain-job-cm labeled
Checking if object type job with name domain1-create-weblogic-sample-domain-job exists
No resources found in default namespace.
Creating the domain by creating the job /home/haiche/azure/weblogic-domains/domain1/create-domain-job.yaml
job.batch/domain1-create-weblogic-sample-domain-job created
Waiting for the job to complete...
Error from server (BadRequest): container "create-weblogic-sample-domain-job" in pod "domain1-create-weblogic-sample-domain-job-j5bgz" is waiting to start: PodInitializing
status on iteration 1 of 20
pod domain1-create-weblogic-sample-domain-job-j5bgz status is Init:0/1
Error from server (BadRequest): container "create-weblogic-sample-domain-job" in pod "domain1-create-weblogic-sample-domain-job-j5bgz" is waiting to start: PodInitializing
status on iteration 2 of 20
pod domain1-create-weblogic-sample-domain-job-j5bgz status is Init:0/1
status on iteration 3 of 20
pod domain1-create-weblogic-sample-domain-job-j5bgz status is Running
status on iteration 4 of 20
pod domain1-create-weblogic-sample-domain-job-j5bgz status is Completed
domain.weblogic.oracle/domain1 created

Administration console access is available at http://haicheaksc-haicheresourcegr-685ba0-1eda3c35.hcp.eastus.azmk8s.io:30701/console
The following files were generated:
  /home/haiche/azure/weblogic-domains/domain1/create-domain-inputs.yaml
  /home/haiche/azure/weblogic-domains/domain1/create-domain-job.yaml
  /home/haiche/azure/weblogic-domains/domain1/domain.yaml

Completed

and then I create load balancer with the configuration:
Load balancer for admin server:
apiVersion: v1
kind: Service
metadata:
  name: domain1-admin-server-external-lb
  namespace: default
spec:
  ports:
  - name: default
    port: 7001
    protocol: TCP
    targetPort: 7001
  selector:
    weblogic.domainUID: domain1
    weblogic.serverName: admin-server
  sessionAffinity: None
  type: LoadBalancer

Load balancer for cluster:
apiVersion: v1
kind: Service
metadata:
  name: domain1-cluster-1-external-lb
  namespace: default
spec:
  ports:
  - name: default
    port: 8001
    protocol: TCP
    targetPort: 8001
  selector:
    weblogic.domainUID: domain1
    weblogic.clusterName: cluster-1
  sessionAffinity: None
  type: LoadBalancer

Then I deployed the testwebapp, and got the following error:
[partition-name: DOMAIN] > <BEA-149078> <Stack trace for message 149004
weblogic.management.DeploymentException: [Deployer:149189]An attempt was made to execute the "distribute" operation on an application named "testwebapp" that is not currently available. The application may have been created after non-dynamic configuration changes were activated. If so, the operation cannot be performed until the server is restarted so that the application will be available.
	at weblogic.deploy.internal.targetserver.DeploymentManager.assertDeploymentMBeanIsNonNull(DeploymentManager.java:1598)
	at weblogic.deploy.internal.targetserver.DeploymentManager.findDeploymentMBean(DeploymentManager.java:1643)
	at weblogic.deploy.internal.targetserver.DeploymentManager.createOperation(DeploymentManager.java:1172)
	at weblogic.deploy.internal.targetserver.DeploymentManager.createOperations(DeploymentManager.java:1690)
	at weblogic.deploy.internal.targetserver.DeploymentManager.handleUpdateDeploymentContext(DeploymentManager.java:182)
	at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.updateDeploymentContext(DeploymentServiceDispatcher.java:173)
	at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doUpdateDeploymentContextCallback(DeploymentReceiverCallbackDeliverer.java:155)
	at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.updateDeploymentContext(DeploymentReceiverCallbackDeliverer.java:29)
	at weblogic.deploy.service.internal.statemachines.targetserver.ReceivedPrepare.callDeploymentReceivers(ReceivedPrepare.java:208)
	at weblogic.deploy.service.internal.statemachines.targetserver.ReceivedPrepare.handlePrepare(ReceivedPrepare.java:112)
	at weblogic.deploy.service.internal.statemachines.targetserver.ReceivedPrepare.receivedPrepare(ReceivedPrepare.java:52)
	at weblogic.deploy.service.internal.targetserver.TargetRequestImpl.run(TargetRequestImpl.java:214)
	at weblogic.deploy.service.internal.transport.CommonMessageReceiver$1.run(CommonMessageReceiver.java:614)
	at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:670)
	at weblogic.invocation.ComponentInvocationContextManager._runAs(ComponentInvocationContextManager.java:352)
	at weblogic.invocation.ComponentInvocationContextManager.runAs(ComponentInvocationContextManager.java:337)
	at weblogic.work.LivePartitionUtility.doRunWorkUnderContext(LivePartitionUtility.java:57)
	at weblogic.work.PartitionUtility.runWorkUnderContext(PartitionUtility.java:41)
	at weblogic.work.SelfTuningWorkManagerImpl.runWorkUnderContext(SelfTuningWorkManagerImpl.java:644)
	at weblogic.work.ExecuteThread.execute(ExecuteThread.java:415)
	at weblogic.work.ExecuteThread.run(ExecuteThread.java:355)
>

could you help to check? Thank you

I got the answer from @edburns, should use the image from Oracle Container Registry https://oracle.github.io/weblogic-kubernetes-operator/quickstart/get-images/.

The error still happens to container-registry.oracle.com/middleware/weblogic:<IP_ADDRESS>, reopen this issue.
Input parameters being used
export version="create-weblogic-sample-domain-inputs-v1"
export adminPort="7001"
export adminServerName="admin-server"
export domainUID="domain1"
export domainHome="/shared/domains/domain1"
export serverStartPolicy="IF_NEEDED"
export clusterName="cluster-1"
export configuredManagedServerCount="5"
export initialManagedServerReplicas="2"
export managedServerNameBase="managed-server"
export managedServerPort="8001"
export image="container-registry.oracle.com/middleware/weblogic:<IP_ADDRESS>"
export imagePullPolicy="IfNotPresent"
export imagePullSecretName="galia10regcred"
export productionModeEnabled="true"
export weblogicCredentialsSecretName="domain1-weblogic-credentials"
export includeServerOutInPodLog="true"
export logHome="/shared/logs/domain1"
export httpAccessLogInLogHome="true"
export t3ChannelPort="30012"
export exposeAdminT3Channel="false"
export adminNodePort="30701"
export exposeAdminNodePort="true"
export namespace="default"
javaOptions=-Dweblogic.StdoutDebugEnabled=false
export persistentVolumeClaimName="galia10azurefile"
export domainPVMountPath="/shared"
export createDomainScriptsMountPath="/u01/weblogic"
export createDomainScriptName="create-domain-job.sh"
export createDomainFilesDir="wlst"
export serverPodMemoryRequest="768Mi"
export serverPodCpuRequest="250m"
export istioEnabled="false"
export istioReadinessPort="8888"

Generating /home/haixia/azure/weblogic-domains/domain1/create-domain-job.yaml
Generating /home/haixia/azure/weblogic-domains/domain1/delete-domain-job.yaml
Generating /home/haixia/azure/weblogic-domains/domain1/domain.yaml
Checking to see if the secret domain1-weblogic-credentials exists in namespace default
Checking if the persistent volume claim galia10azurefile in NameSpace default exists
The persistent volume claim galia10azurefile already exists in NameSpace default
configmap/domain1-create-weblogic-sample-domain-job-cm configured
Checking the configmap domain1-create-weblogic-sample-domain-job-cm was created
'weblogic.domainUID' already has a value (domain1), and --overwrite is false
'weblogic.domainName' already has a value (domain1), and --overwrite is false
Checking if object type job with name domain1-create-weblogic-sample-domain-job exists
Deleting domain1-create-weblogic-sample-domain-job using /home/haixia/azure/weblogic-domains/domain1/create-domain-job.yaml
job.batch "domain1-create-weblogic-sample-domain-job" deleted
Creating the domain by creating the job /home/haixia/azure/weblogic-domains/domain1/create-domain-job.yaml
job.batch/domain1-create-weblogic-sample-domain-job created
Waiting for the job to complete...
Error from server (BadRequest): container "create-weblogic-sample-domain-job" in pod "domain1-create-weblogic-sample-domain-job-2zcvb" is waiting to start: PodInitializing
status on iteration 1 of 20
pod domain1-create-weblogic-sample-domain-job-2zcvb status is Init:0/1
Error from server (BadRequest): container "create-weblogic-sample-domain-job" in pod "domain1-create-weblogic-sample-domain-job-2zcvb" is waiting to start: PodInitializing
status on iteration 2 of 20
pod domain1-create-weblogic-sample-domain-job-2zcvb status is Init:0/1
status on iteration 3 of 20
pod domain1-create-weblogic-sample-domain-job-2zcvb status is Completed
domain.weblogic.oracle/domain1 created

Domain domain1 was created and will be started by the WebLogic Kubernetes Operator

Administration console access is available at http://galia10aks-galia10resourceg-058876-4ae5f0a2.hcp.eastus.azmk8s.io:30701/console
The following files were generated:
  /home/haixia/azure/weblogic-domains/domain1/create-domain-inputs.yaml
  /home/haixia/azure/weblogic-domains/domain1/create-domain-job.yaml
  /home/haixia/azure/weblogic-domains/domain1/domain.yaml

Completed

What is your base image based on,  we have seen similar issue when using the weblogic image from docker hub (which is out of date).  If you are, please switch to use the image from https://container-registry.oracle.com  (under middleware, you will need to sign in to the browser, select the image and accept license agreement first)

What is your base image based on, we have seen similar issue when using the weblogic image from docker hub (which is out of date). If you are, please switch to use the image from https://container-registry.oracle.com (under middleware, you will need to sign in to the browser, select the image and accept license agreement first)

Hello @jshum2479  I got the same error running with images from https://container-registry.oracle.com/, my steps were:

open https://container-registry.oracle.com/, search "weblogic"
click "weblogic", sign in, and accept license agreement.
set image path image="container-registry.oracle.com/middleware/weblogic:<IP_ADDRESS>"

I am running the image on Azure Kubernetes Service.

How do you change the image field?  Did you change it in the domain.yaml  and reapply the yaml or you just change the input.yaml?

Do you have time for a zoom session?

Do you have time for a zoom session?
I created a new AKS instance, cloned operator source code, changed kubernetes\samples\scripts\create-soa-domain\domain-home-on-pv\create-domain-inputs.yaml,  and ran kubernetes\samples\scripts\create-soa-domain\domain-home-on-pv\create-domain.sh.

I am glad to have a zoom session, thank you!
My working time is UTC+8, I am available now, please let me know when you are available.

Hi @galiacheng ,
Are you able to make further progress in this issue or is it still an issue?
I think the main difference between my environment and your environment is the PV, other than I use a specific resource group

I added the persistentVolumeReclaimPolicy:  Retain
Removed the mountOptions
I pointed the right testwebapp.war to use, but you can compare the md5sum between the one in
src/integration-tests/apps and yours

This issue is solved by applying pv/pvc config from @jshum2479
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolume
metadata:
  name: fileshare-pv
  labels:
    usage: fileshare-pv
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  storageClassName: azurefile
  persistentVolumeReclaimPolicy: Retain
  azureFile:
    secretName: azure-secret
    shareName: fileshare1
    readOnly: false
  mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=1000
  - gid=1000
  - mfsymlinks
EOF

cat <<EOF | kubectl apply -f -
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: fileshare-pvc
spec:
  storageClassName: azurefile
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  selector:
    matchLabels:
      usage: fileshare-pv
EOF


CompleteCheckpoint(true) never completes for CheckpointStrategy.FoldOver

I've noticed that sometimes CompleteCheckpoint(true) is stuck in WAIT_FLUSH when FoldOver strategy is used. The most consistent way for me to reproduce was:

Start session
Upsert a key/value
Take full checkpoint, CompleteCheckpoint(true)
Stop session
Exit app
Start app
Recover
Start session
Read a key/value
Stop session
Start session
TakeFullCheckpoint and CompleteCheckpoint(true)

It seems as if there were no changes in the session then there is nothing to write to checkpoint but it still tries to wait for flush which never happens. This had happened to me (a few times though inconsistently) even for cases where other sessions were writing key/values.

Is this an issue with 'variable length structs' as well?

I am sure that I encountered this with both variable length structs and when using object log, I don't remember if I tested fixed length key/values.

Created gists that reproduces the issue:

Fixed size - https://gist.github.com/marius-klimantavicius/a9ae5b9a1649ad314bd9b4abac96bb0f
Using object log - https://gist.github.com/marius-klimantavicius/11aed3f5e63b36b16f928c98ca245c2a

I ran the repro for fixed size (x64, debug/release, net46) around 10 times, no error. Can you try the same config and see?

I can reliably reproduce this with x64/release/(net46,net472). The first time the app is run it finishes within a second, in case of the second and subsequent times - I give up after 10-15 secs and kill the process (when I first encountered it I waited for 15 minutes [went for lunch] it did not complete).
This is what I see in trace window:
********* Primary Recovery Information ********
Index Checkpoint: e86357a9-0461-46e7-a2e2-d22e4bf7523c
HybridLog Checkpoint: 36c44720-c0a4-4569-8051-456a0987b7e5
******** Index Checkpoint Info for e86357a9-0461-46e7-a2e2-d22e4bf7523c ********
Table Size: 2097152
Main Table Size (in GB): 0.134217728
Overflow Table Size (in GB): 1.024E-06
Num Buckets: 16
Start Logical Address: 88
Final Logical Address: 88
******** HybridLog Checkpoint Info for 36c44720-c0a4-4569-8051-456a0987b7e5 ********
Version: 1
Is Snapshot?: False
Flushed LogicalAddress: 0
Start Logical Address: 88
Final Logical Address: 88
Num sessions recovered: 1
Recovered sessions:
693a141d-289b-437d-a340-73c3d3d977be: 0
******* Recovered HybridLog Stats *******
Head Address: 64
Safe Head Address: 64
ReadOnly Address: 88
Safe ReadOnly Address: 88
Tail Address: 88
Moved to INTERMEDIATE, 2
Moved to PREP_INDEX_CHECKPOINT, 2
Moved to INTERMEDIATE, 2
Moved to PREPARE, 2
Moved to INTERMEDIATE, 2
Moved to IN_PROGRESS, 3
Moved to INTERMEDIATE, 3
Moved to WAIT_PENDING, 3
Moved to INTERMEDIATE, 3
Moved to WAIT_FLUSH, 3

My little investigation shows the following:

We are moving from WAIT_PENDING to WAIT_FlUSH
We check that strategy is FoldOverSnapshot

if (FoldOverSnapshot)
{
    hlog.ShiftReadOnlyToTail(out long tailAddress);
    _hybridLogCheckpoint.info.finalLogicalAddress = tailAddress;
}

ShiftReadOnlyToTail tries to monotonically update ReadOnlyAddress, but fails because ReadOnlyAddress and tailAddress are the same (in my case 88)
Because monotonic update failed the method OnPagesMarkedReadOnly is not 'called' thus in turn AsyncFlushPages is not called.
As flush was not called we are forever stuch in WAIT_FLUSH state.

I've just checked that this seems to have been fixed in master branch - all of my tests were done with latest release (2<IP_ADDRESS>).


Users should be able to mix delete + insert for mutations

The "easy" way to create mutations is to do either a set of inserts (aka SetCell) or a set of deletes (of columns or families).  Users should be allowed to create a combination of SetCell and deletes for a single row.
The implementation is in #161.  This is a backwards incompatible change only for those users that explicitly use gaxOptions.

This was fixed by #161.


@IfBundleAvailable does not support nested tests classes

The JUnit condition BundleAvailableCondition is not working with the nested test classes. Seems this is only used for gRPC test at the moment. I guess it was added once grpc modules were not in the default profile.
We can delete it or fix as this can be a valuable test util in the future.

While not against fix, I think maybe we should delete this usage altogether if and when this is OSGi-specific functionality -- for SGv2 we would not be using this anyway, correct? And we are not planning on adding new services on SGv1 that would be needing this, now that grpc is included by default.

Let s fix now and remove later on. I am also pro removing.

SGTM!


thingbuf::mpsc::Sender hanging up for parallel try_send_ref and send / send_ref from sync thread and async tokio::task

Hi! We are using thingbuf in a performance-sensitive application for its speed while supporting sync and async interaction on the same sender type. Recently I ran into an issue where thingbuf seems to hang up our async application completely in busy loops. The profiling in the degraded state showed almost only calls to thingbuf.
I was able to reproduce the issue in a minimum example. You can run several variations of it yourself by checking out this repo:
https://github.com/sgasse/thingbuf_hangup/
The initial setup (binary thingbuf_sendref) to get into the hangup was this:

One std::thread sends with try_send_ref in a loop every 10ms.
One tokio::task receives on the channel with recv_ref().await in a loop. After 10s, there is a delay of 10s introduced between receive calls. This simulates badly handled backpressure from a downstream task.
One tokio::task starts sending with send(..).await in a loop after 20s.
One tokio::task logs an alive message every second.

Once the second sender becomes active, we no longer see any alive logs. Introducing logs to thingbuf shows that two threads (one tokio worker and the self-spawned thread) are both stuck in this loop in push_ref.
Here is some log output with line numbers from src/lib.rs from thingbuf for the hang-up scenario, this is infinitely:
...
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 202: push_ref loop starts
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 294: if
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 227: idx: 77, state: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 301: tail at the end of the loop: 7757
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 286: head: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 202: push_ref loop starts
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 294: if
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 227: idx: 77, state: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 301: tail at the end of the loop: 7757
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 286: head: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 202: push_ref loop starts
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 294: if
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 227: idx: 77, state: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 301: tail at the end of the loop: 7757
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 286: head: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 202: push_ref loop starts
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 294: if
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(2) line 227: idx: 77, state: 7246
[2024-03-06T08:16:20Z DEBUG thingbuf] thread ThreadId(3) line 301: tail at the end of the loop: 7757
...

The initial setup mimics the behavior of a part of our real application. However I varied the setup in other examples, here are some findings:

Whether we try_send_ref() (sync), send_ref().await() or send(..).await (async) does not seem to matter, see examples thingbuf_sendref, thingbuf_sendref_pure, thingbuf_send_recvref and thingbuf_send_no_try_recvref which all hang-up.
When using recv().await instead of recv_ref().await, there is no hangup, see example thingbuf_send.
Switching from 1 worker thread to e.g. 3 sometimes leads to a few more logs printed immediately after the second sender starts but then, the example also hangs up.
My system is x86_64-unknown-linux-gnu, but I initially found it on aarch64-linux-android so I guess it does not depend on the platform.
The issue is reproducible with both rustc in version 1.76 and nightly, so probably not related to the compiler.
To check that there is nothing obviously wrong with the example setup, I replaced thingbuf::mpsc with tokio::mpsc in one example, which works as expected: It still logs the alive messages and does not hang up.

I would have expected the same behavior as I see with tokio::mpsc, so I guess it is a bug. But please let me know if there is a limitation which I overlooked or if I can provide further info.

I encountered the same problem.

We are also using StaticThingbuf in the DragonOS kernel and have encountered the exact same problem. The issue appears randomly and is difficult to reproduce after the system kernel is restarted.
In our use case:

https://code.dragonos.org.cn/xref/DragonOS/kernel/src/debug/klog/mm.rs?r=b8ed38251dc255b0c525801b5dbf37d3b0d0d61e#48
https://code.dragonos.org.cn/xref/DragonOS/kernel/crates/klog_types/src/lib.rs?r=840045af94ea3391f29e87e968db5d9c48316981#189

Once the buffer is full, there is a very sluggish phenomenon. After a period of time, the performance drops off a cliff. Ultimately, it freezes at the push_ref.

Okay, v0.1.5 has been published to crates.io, including @tukan's changes from PR #81, which should fix this.
Let me know if you continue seeing performance issues after updating to v0.1.5!

After updating the repository, the performance issue still remains unresolved. Below is the backtrace of the program execution:

The push_ref() method eventually hangs at:

Hmm, that's unfortunate. Thanks for following up --- I'll keep looking.

It looks like, after 11089327 iterations under loom, the tests added in PR #81 seem to enter into an infinite spin loop.

 INFO iter{11089327}:thread{id=1}: loom::rt::execution: ~~~~~~~~ THREAD 1 ~~~~~~~~
[ThreadId(1)                     src/lib.rs:201] push_ref
[ThreadId(1)                     src/lib.rs:203] self.tail.load(Relaxed) = 0
[ThreadId(1)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(1)                     src/lib.rs:209] idx = 0
[ThreadId(1)                     src/lib.rs:210] gen = 0
[ThreadId(1)                     src/lib.rs:228] slot.state.load(SeqCst) = 0
[ThreadId(1)                     src/lib.rs:229] clear_has_reader(raw_state) = 0
[ThreadId(1)                     src/lib.rs:231] state == tail = true
[ThreadId(1)                     src/lib.rs:234] self.tail.compare_exchange_weak(tail, next_tail, SeqCst, Acquire) = Ok(0)
[ThreadId(1)                     src/lib.rs:238] check_has_reader(raw_state) = false
[ThreadId(1)                     src/lib.rs:257] advanced tail 0 to 1; claimed slot [0]
[ThreadId(1)                     src/lib.rs:272] -> initialized
[ThreadId(1)                     src/lib.rs:560] drop Ref<usize> (push), new_state = 1
[ThreadId(1)                     src/lib.rs:565] self.new_state = 1
[ThreadId(1)                     src/lib.rs:565] self.slot.state.store(test_dbg!(self.new_state), Release) = ()
[ThreadId(1)                    src/mpsc.rs:562] notifying rx (loom::thread::Thread)
[ThreadId(1)               src/wait/cell.rs:161] notifying; close=WAITING;
[ThreadId(1)               src/wait/cell.rs:163] bits = NOTIFYING
[ThreadId(1)               src/wait/cell.rs:164] self.fetch_or(bits, AcqRel) = WAITING
[ThreadId(1)               src/wait/cell.rs:168] self.fetch_and(!State::NOTIFYING, AcqRel) = NOTIFYING
[ThreadId(1)               src/wait/cell.rs:170] waiter = None
 INFO iter{11089327}:thread{id=2}: loom::rt::execution: ~~~~~~~~ THREAD 2 ~~~~~~~~
[ThreadId(2)                     src/lib.rs:201] push_ref
[ThreadId(2)                     src/lib.rs:203] self.tail.load(Relaxed) = 0
[ThreadId(2)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(2)                     src/lib.rs:209] idx = 0
[ThreadId(2)                     src/lib.rs:210] gen = 0
[ThreadId(2)                     src/lib.rs:228] slot.state.load(SeqCst) = 0
[ThreadId(2)                     src/lib.rs:229] clear_has_reader(raw_state) = 0
[ThreadId(2)                     src/lib.rs:231] state == tail = true
[ThreadId(2)                     src/lib.rs:234] self.tail.compare_exchange_weak(tail, next_tail, SeqCst, Acquire) = Err(1)
[ThreadId(2)                     src/lib.rs:290] failed to advance tail 0 to 1
[ThreadId(2)                    src/util.rs:40 ] hint::spin_loop() (x1)
 INFO iter{11089327}:thread{id=0}: loom::rt::execution: ~~~~~~~~ THREAD 0 ~~~~~~~~
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 0
[ThreadId(0)                     src/lib.rs:326] idx = 0
[ThreadId(0)                     src/lib.rs:327] gen = 0
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 1
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = true
[ThreadId(0)                     src/lib.rs:352] self.head.compare_exchange_weak(head, next_head, SeqCst, Acquire) = Ok(0)
[ThreadId(0)                     src/lib.rs:357] advanced head 0 to 1
[ThreadId(0)                     src/lib.rs:358] claimed slot [0]
[ThreadId(0)                     src/lib.rs:361] new_state =<PHONE_NUMBER>854775816
[ThreadId(0)                     src/lib.rs:361] slot.state.store(test_dbg!(new_state), SeqCst) = ()
[ThreadId(0)                     src/lib.rs:557] drop Ref<usize> (pop)
 INFO iter{11089327}:thread{id=2}: loom::rt::execution: ~~~~~~~~ THREAD 2 ~~~~~~~~
[ThreadId(2)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(2)                     src/lib.rs:209] idx = 1
[ThreadId(2)                     src/lib.rs:210] gen = 0
[ThreadId(2)                     src/lib.rs:228] slot.state.load(SeqCst) = 1
[ThreadId(2)                     src/lib.rs:229] clear_has_reader(raw_state) = 1
[ThreadId(2)                     src/lib.rs:231] state == tail = true
[ThreadId(2)                     src/lib.rs:234] self.tail.compare_exchange_weak(tail, next_tail, SeqCst, Acquire) = Ok(1)
[ThreadId(2)                     src/lib.rs:238] check_has_reader(raw_state) = false
[ThreadId(2)                     src/lib.rs:257] advanced tail 1 to 8; claimed slot [1]
[ThreadId(2)                     src/lib.rs:272] -> initialized
[ThreadId(2)                     src/lib.rs:560] drop Ref<usize> (push), new_state = 2
[ThreadId(2)                     src/lib.rs:565] self.new_state = 2
[ThreadId(2)                     src/lib.rs:565] self.slot.state.store(test_dbg!(self.new_state), Release) = ()
[ThreadId(2)                    src/mpsc.rs:562] notifying rx (loom::thread::Thread)
[ThreadId(2)               src/wait/cell.rs:161] notifying; close=WAITING;
[ThreadId(2)               src/wait/cell.rs:163] bits = NOTIFYING
[ThreadId(2)               src/wait/cell.rs:164] self.fetch_or(bits, AcqRel) = WAITING
[ThreadId(2)               src/wait/cell.rs:168] self.fetch_and(!State::NOTIFYING, AcqRel) = NOTIFYING
[ThreadId(2)               src/wait/cell.rs:170] waiter = None
 INFO iter{11089327}:thread{id=1}: loom::rt::execution: ~~~~~~~~ THREAD 1 ~~~~~~~~
[ThreadId(1)                     src/lib.rs:201] push_ref
[ThreadId(1)                     src/lib.rs:203] self.tail.load(Relaxed) = 8
[ThreadId(1)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(1)                     src/lib.rs:209] idx = 0
[ThreadId(1)                     src/lib.rs:210] gen = 8
[ThreadId(1)                     src/lib.rs:228] slot.state.load(SeqCst) =<PHONE_NUMBER>854775816
[ThreadId(1)                     src/lib.rs:229] clear_has_reader(raw_state) = 8
[ThreadId(1)                     src/lib.rs:231] state == tail = true
[ThreadId(1)                     src/lib.rs:234] self.tail.compare_exchange_weak(tail, next_tail, SeqCst, Acquire) = Ok(8)
[ThreadId(1)                     src/lib.rs:238] check_has_reader(raw_state) = true
[ThreadId(1)                     src/lib.rs:239] advanced tail 8 to 9; has an active reader, skipping slot [0]
[ThreadId(1)                     src/lib.rs:246] slot.state.fetch_update(SeqCst, SeqCst,
        |state|
            {
                Some(state & HAS_READER | next_state)
            }).unwrap_or_else(|_| unreachable!()) =<PHONE_NUMBER>854775816
[ThreadId(1)                    src/util.rs:40 ] hint::spin_loop() (x1)
 INFO iter{11089327}:thread{id=0}: loom::rt::execution: ~~~~~~~~ THREAD 0 ~~~~~~~~
[ThreadId(0)                     src/lib.rs:558] self.slot.state.fetch_and(!HAS_READER, SeqCst) =<PHONE_NUMBER>854775824
[ThreadId(0)                    src/mpsc.rs:570] notifying tx (loom::thread::Thread)
[ThreadId(0)              src/wait/queue.rs:371] WaitQueue::notify()
[ThreadId(0)              src/wait/queue.rs:377] state = 0
[ThreadId(0)              src/wait/queue.rs:378] self.state.compare_exchange_weak(state, WAKING, SeqCst, SeqCst) = Ok(0)
 INFO iter{11089327}:thread{id=1}: loom::rt::execution: ~~~~~~~~ THREAD 1 ~~~~~~~~
[ThreadId(1)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(1)                     src/lib.rs:209] idx = 0
[ThreadId(1)                     src/lib.rs:210] gen = 8
[ThreadId(1)                     src/lib.rs:228] slot.state.load(SeqCst) = 16
[ThreadId(1)                     src/lib.rs:229] clear_has_reader(raw_state) = 16
[ThreadId(1)                     src/lib.rs:231] state == tail = false
[ThreadId(1)                     src/lib.rs:306] self.head.fetch_or(0, SeqCst) = 1
[ThreadId(1)                     src/lib.rs:307] wrapping_add(head, self.gen) == tail = false
[ThreadId(1)                    src/util.rs:57 ] hint::spin_loop() (x2)
[ThreadId(1)                    src/util.rs:60 ] thread::yield_now()
 INFO iter{11089327}:thread{id=2}: loom::rt::execution: ~~~~~~~~ THREAD 2 ~~~~~~~~
[ThreadId(2)                     src/lib.rs:201] push_ref
[ThreadId(2)                     src/lib.rs:203] self.tail.load(Relaxed) = 9
[ThreadId(2)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(2)                     src/lib.rs:209] idx = 1
[ThreadId(2)                     src/lib.rs:210] gen = 8
[ThreadId(2)                     src/lib.rs:228] slot.state.load(SeqCst) = 2
[ThreadId(2)                     src/lib.rs:229] clear_has_reader(raw_state) = 2
[ThreadId(2)                     src/lib.rs:231] state == tail = false
[ThreadId(2)                     src/lib.rs:306] self.head.fetch_or(0, SeqCst) = 1
[ThreadId(2)                     src/lib.rs:307] wrapping_add(head, self.gen) == tail = true
[ThreadId(2)                     src/lib.rs:308] channel full
[ThreadId(2)              src/wait/queue.rs:189] WaitQueue::start_wait(0x7f3475749900)
[ThreadId(2)              src/wait/queue.rs:193] self.state.compare_exchange(WAKING, EMPTY, SeqCst, SeqCst) = Ok(2)
[ThreadId(2)           src/mpsc/blocking.rs:2163] core.tx_wait.start_wait(node, &thread) = Notified
[ThreadId(2)                    src/util.rs:57 ] hint::spin_loop() (x1)
[ThreadId(2)                    src/util.rs:60 ] thread::yield_now()
 INFO iter{11089327}:thread{id=1}: loom::rt::execution: ~~~~~~~~ THREAD 1 ~~~~~~~~
[ThreadId(1)                     src/lib.rs:313] self.tail.load(Acquire) = 9
[ThreadId(1)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(1)                     src/lib.rs:209] idx = 1
[ThreadId(1)                     src/lib.rs:210] gen = 8
[ThreadId(1)                     src/lib.rs:228] slot.state.load(SeqCst) = 1
[ThreadId(1)                     src/lib.rs:229] clear_has_reader(raw_state) = 1
[ThreadId(1)                     src/lib.rs:231] state == tail = false
[ThreadId(1)                     src/lib.rs:306] self.head.fetch_or(0, SeqCst) = 1
[ThreadId(1)                     src/lib.rs:307] wrapping_add(head, self.gen) == tail = true
[ThreadId(1)                     src/lib.rs:308] channel full
[ThreadId(1)              src/wait/queue.rs:189] WaitQueue::start_wait(0x7f3475752900)
[ThreadId(1)              src/wait/queue.rs:193] self.state.compare_exchange(WAKING, EMPTY, SeqCst, SeqCst) = Err(0)
[ThreadId(1)              src/wait/queue.rs:209] WaitQueue::start_wait_slow(0x7f3475752900)
[ThreadId(1)    src/util/mutex/loom_impl.rs:15 ] locking thingbuf::wait::queue::List<loom::thread::Thread>...
[ThreadId(1)    src/util/mutex/loom_impl.rs:17 ] -> locked thingbuf::wait::queue::List<loom::thread::Thread>!
[ThreadId(1)              src/wait/queue.rs:221] state = 0
[ThreadId(1)              src/wait/queue.rs:225] self.state.compare_exchange_weak(EMPTY, WAITING, SeqCst, SeqCst) = Ok(0)
[ThreadId(1)              src/wait/queue.rs:280] node.state.swap(WAITING, Release) = 0
[ThreadId(1)              src/wait/queue.rs:549] List::enqueue(0x7f3475752900)
[ThreadId(1)           src/mpsc/blocking.rs:2163] core.tx_wait.start_wait(node, &thread) = Wait
 INFO iter{11089327}:thread{id=0}: loom::rt::execution: ~~~~~~~~ THREAD 0 ~~~~~~~~
[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 1
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 0
[ThreadId(0)                     src/lib.rs:346] slot.state.load(Acquire) = 2
[ThreadId(0)                     src/lib.rs:350] raw_state == head + 1 = true
 INFO iter{11089327}:thread{id=2}: loom::rt::execution: ~~~~~~~~ THREAD 2 ~~~~~~~~
[ThreadId(2)                     src/lib.rs:201] push_ref
[ThreadId(2)                     src/lib.rs:203] self.tail.load(Relaxed) = 9
[ThreadId(2)                     src/lib.rs:205] tail & self.closed != 0 = false
[ThreadId(2)                     src/lib.rs:209] idx = 1
[ThreadId(2)                     src/lib.rs:210] gen = 8
[ThreadId(2)                     src/lib.rs:228] slot.state.load(SeqCst) = 2
[ThreadId(2)                     src/lib.rs:229] clear_has_reader(raw_state) = 2
[ThreadId(2)                     src/lib.rs:231] state == tail = false
[ThreadId(2)                     src/lib.rs:306] self.head.fetch_or(0, SeqCst) = 1
[ThreadId(2)                     src/lib.rs:307] wrapping_add(head, self.gen) == tail = true
[ThreadId(2)                     src/lib.rs:308] channel full
[ThreadId(2)              src/wait/queue.rs:189] WaitQueue::start_wait(0x7f3475749900)
[ThreadId(2)              src/wait/queue.rs:193] self.state.compare_exchange(WAKING, EMPTY, SeqCst, SeqCst) = Err(1)
[ThreadId(2)              src/wait/queue.rs:209] WaitQueue::start_wait_slow(0x7f3475749900)
[ThreadId(2)    src/util/mutex/loom_impl.rs:15 ] locking thingbuf::wait::queue::List<loom::thread::Thread>...
[ThreadId(2)    src/util/mutex/loom_impl.rs:17 ] -> locked thingbuf::wait::queue::List<loom::thread::Thread>!

[ThreadId(0)                     src/lib.rs:319] pop_ref
[ThreadId(0)                     src/lib.rs:324] head = 9
[ThreadId(0)                     src/lib.rs:326] idx = 1
[ThreadId(0)                     src/lib.rs:327] gen = 8
thread 'mpsc::tests::mpsc_blocking::mpsc_test_skip_slot' panicked at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/path.rs:204:13:
Model exceeded maximum number of branches. This is often caused by an algorithm requiring the processor to make progress, e.g. spin locks.
stack backtrace:
   0: std::panicking::begin_panic
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:686:12
   1: loom::rt::path::Path::branch_thread
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/path.rs:204:13
   2: loom::rt::execution::Execution::schedule
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/execution.rs:188:20
   3: loom::rt::branch::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/mod.rs:120:22
   4: loom::rt::scheduler::Scheduler::with_execution::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/scheduler.rs:48:34
   5: loom::rt::scheduler::Scheduler::with_state::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/scheduler.rs:130:28
   6: scoped_tls::ScopedKey<T>::with
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/scoped-tls-1.0.1/src/lib.rs:171:13
   7: loom::rt::scheduler::Scheduler::with_state
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/scheduler.rs:130:9
   8: loom::rt::scheduler::Scheduler::with_execution
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/scheduler.rs:48:9
   9: loom::rt::execution
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/mod.rs:171:5
  10: loom::rt::branch
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/mod.rs:118:25
  11: loom::rt::object::Ref<T>::branch_action
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/object.rs:351:9
  12: loom::rt::atomic::Atomic<T>::branch
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/atomic.rs:389:11
  13: loom::rt::atomic::Atomic<T>::load
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/atomic.rs:229:9
  14: loom::sync::atomic::atomic::Atomic<T>::load
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/sync/atomic/atomic.rs:28:9
  15: loom::sync::atomic::int::AtomicUsize::load
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/sync/atomic/int.rs:58:17
  16: thingbuf::Core::pop_ref
             at ./src/lib.rs:346:39
  17: thingbuf::mpsc::ChannelCore<N>::try_recv_ref
             at ./src/mpsc.rs:383:9
  18: thingbuf::mpsc::blocking::Receiver<T,R>::try_recv_ref
             at ./src/mpsc/blocking.rs:1888:9
  19: thingbuf::mpsc::tests::mpsc_blocking::mpsc_test_skip_slot::{{closure}}
             at ./src/mpsc/tests/mpsc_blocking.rs:111:19
  20: thingbuf::loom::inner::run_builder::{{closure}}
             at ./src/loom.rs:125:13
  21: loom::model::Builder::check::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/model.rs:185:17
  22: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
  23: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/boxed.rs:2015:9
  24: loom::rt::scheduler::spawn_threads::{{closure}}::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/loom-0.5.6/src/rt/scheduler.rs:149:21
  25: generator::gen_impl::GeneratorImpl<A,T>::init_code::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/generator-0.7.5/src/gen_impl.rs:336:21
  26: generator::stack::StackBox<F>::call_once
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/generator-0.7.5/src/stack/mod.rs:139:13
  27: generator::stack::Func::call_once
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/generator-0.7.5/src/stack/mod.rs:121:9
  28: generator::gen_impl::gen_init::{{closure}}
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/generator-0.7.5/src/gen_impl.rs:552:9
  29: core::ops::function::FnOnce::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
  30: std::panicking::try::do_call
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
  31: std::panicking::try
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
  32: std::panic::catch_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
  33: generator::gen_impl::catch_unwind_filter
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/generator-0.7.5/src/gen_impl.rs:543:5
  34: generator::gen_impl::gen_init
             at /home/eliza/.cargo/registry/src/index.crates.io-6f17d22bba15001f/generator-0.7.5/src/gen_impl.rs:570:25
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
[PANIC                     src/lib.rs:557] drop Ref<usize> (pop)
[PANIC                     src/lib.rs:558] self.slot.state.fetch_and(!HAS_READER, SeqCst) =<PHONE_NUMBER>854775817
[PANIC                    src/mpsc.rs:570] notifying tx (loom::thread::Thread)
[PANIC              src/wait/queue.rs:371] WaitQueue::notify()
[PANIC              src/wait/queue.rs:377] state = 0
[PANIC              src/wait/queue.rs:378] self.state.compare_exchange_weak(state, WAKING, SeqCst, SeqCst) = Err(1)
[PANIC              src/wait/queue.rs:377] state = 1
[PANIC              src/wait/queue.rs:399] WaitQueue::notify_slow(state: 1)
[PANIC    src/util/mutex/loom_impl.rs:15 ] locking thingbuf::wait::queue::List<loom::thread::Thread>...
[PANIC    src/util/mutex/loom_impl.rs:17 ] -> locked thingbuf::wait::queue::List<loom::thread::Thread>!
[PANIC              src/wait/queue.rs:578] List::dequeue(2) -> 0x7f3475752900
[PANIC              src/wait/queue.rs:581] last.state.swap(new_state, Release) = 1
[PANIC              src/wait/queue.rs:419] list.is_empty() = false
[PANIC                    src/wait.rs:66 ] NOTIFYING Thread { id: ThreadId(1), name: None } (from Thread { id: ThreadId(0), name: None })
[PANIC                     src/lib.rs:185] Core::close
[PANIC              src/wait/queue.rs:302] WaitQueue::continue_wait(0x7f3475752900)
[PANIC              src/wait/queue.rs:305] node.state.load(Acquire) = 2
[PANIC           src/mpsc/blocking.rs:2165] core.tx_wait.continue_wait(node, &thread) = Notified
[PANIC                    src/util.rs:57 ] hint::spin_loop() (x1)
[PANIC                    src/util.rs:60 ] thread::yield_now()

It looks like, after 11089327 iterations under loom, the tests added in PR #81 seem to enter into an infinite spin loop.
Full logs from that iteration

I have an idea what's wrong with the test but I have to debug it. Will try to find some time this week to work on it.

I investigated the test and the code in the https://github.com/sgasse/thingbuf_hangup/.
The test fails because of an edge case, and my initial thought is to adjust the test itself.
Here's a breakdown of the issue: We have a buffer with two slots. Initially, we write to slot [0], then to slot [1], and start reading from slot [0]. If we try writing again, slot [0] is unavailable (as it's still being read), so we skip it. Once the reading from slot [0] is completed, the head pointer moves to slot [1]. We attempt to write to slot [1] but can't, since it's not yet read, and the head pointer is already on slot [1]. Consequently, we declare the buffer as full. The problem arises in my test when starting to read from slot [1] – it never gets released, making it perpetually empty for reading and full for writing.
However, I identified a genuine issue within the code at https://github.com/sgasse/thingbuf_hangup/, which I've temporarily fixed on my local setup.
I executed cargo run --bin thingbuf_sendref from the https://github.com/sgasse/thingbuf_hangup repository (using the latest thingbuf), and it revealed an issue in accurately determining when the buffer is full.
I will re-examine my findings and aim to submit a PR later this week.

Please check out the PR: https://github.com/hawkw/thingbuf/pull/85
cc @hawkw

I tested it locally and can confirm that the issue is resolved after updating to v0.1.6. Thank you @tukan for the fix and @hawkw for quickly releasing it! :slightly_smiling_face: :pray:


question: Are Start and End tags guarunteed to be ordered correctly?

I'm building a Markdown formatter using pullmark, and for the correctness of the reformatting I need to know whether or not the output of pullmark will always produce a correctly ordered tree of Starts and Ends - i.e. we never see a situation like this:
Start(x)
Start(y)
End(x)
End(y)

From the way commonmark is defined and from my observations of pullmark it seems like this is the case - but if there are any edge cases I'd like to know about them before getting too deep on the project...
Also, great library - it's saving me a lot of time and effort having to implement an efficient parser myself.  Thanks so much :1st_place_medal:!

Yes, it is a tree. It's remotely possible there is a bug because this is only enforced by convention, but internally the parse state is represented with stacks so each push should have a corresponding pop.
Thanks for the kind words, hope this helps.

Awesome! Thank you :).


Linux Hardened Kernels 2024-01-27

Description of changes
Init Hardened 6.7 at 6.7.2-hardened1
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Building 6.7.2-hardened1

Built 6.7.2-hardened1

You can ignore the pkgs/by-name check failure, that's a bug I'll fix this week.

You can ignore the pkgs/by-name check failure, that's a bug I'll fix this week.

https://github.com/NixOS/nixpkgs/pull/285089