IdP is missing when using MDQ as metadata source

I'm trying to configure an SSO login similar to EduGAIN, using an MDQ service as the source of the metadata, but I cannot manage to make it work.
It always ends up in the next exception:
Internal Server Error: /saml2/login/
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/dist-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.7/dist-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/django/views/generic/base.py", line 98, in dispatch
    return handler(request, *args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/djangosaml2/views.py", line 250, in get
    raise IdPConfigurationMissing("IdP is missing or its metadata is expired.")
djangosaml2.exceptions.IdPConfigurationMissing: IdP is missing or its metadata is expired.

The metadata part of the config:
    'metadata': {
        'mdq': [{
            "url": "https://mdx-2020.eduid.hu/",
            "cert": path.join(BASEDIR, 'saml_certs/href-metadata-signer-2020.crt')
        }]
    },

I have tried to look up an example IdP manually in the MDQ, which works fine, here's the minimal working example which shows that the MDQ configured above is working:
import copy

from django.conf import settings
from django.core.management.base import BaseCommand
from saml2.config import SPConfig

class Command(BaseCommand):
    def handle(self, *args, **options):
        conf = SPConfig()
        conf.load(copy.deepcopy(settings.SAML_CONFIG))
        for metadata in conf.metadata.metadata.values():
            mdx = metadata
            entity2check = 'https://idp.elte.hu/auth/saml2/idp/metadata.php'
            r = mdx.service(entity2check, 'idpsso_descriptor', 'single_sign_on_service')
            self.stdout.write("%s" % r)

which outputs:
{'urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect':
[{'__class__': 'urn:oasis:names:tc:SAML:2.0:metadata&SingleSignOnService',
'binding': 'urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect',
'location': 'https://idp.elte.hu/auth/saml2/idp/SSOService.php'}]}

where I can see that it actually did parse the IdP data.
What am I missing from the configuration? What else is needed to use the MDQ as an IdP metadata source?

Hi @cheoppy
Put a breakpoint here
https://github.com/IdentityPython/djangosaml2/blob/5f956aab8d262cba84f1c52d78b710e059d9007d/djangosaml2/utils.py#L41
you should be alble to get all the idps available (even through MDQ).
It seems that you didn0t get any of them, here
https://github.com/IdentityPython/djangosaml2/blob/5f956aab8d262cba84f1c52d78b710e059d9007d/djangosaml2/views.py#L249

Indeed I haven't got any available idps, here's the traceback provided by django at this point:

The configured_idps is empty. How can it be empty when I use MDQ?

I use djangosaml2 with a MDQ, so I'd suggest to put a breakpoint to check what's happens in your configuration

I did some debugging, and found the following concerning the metadata section of the config:
config.metadata:
type: <class 'saml2.mdstore.MetadataStore'>
value (as string): {https://mdx-2020.eduid.hu/: dict_items([])}

config.metadata.metadata:
type: dict
value (as string): {'https://mdx-2020.eduid.hu/': <saml2.mdstore.MetaDataMDX object at 0x7fa453d45ac8>}

config.metadata.metadata['https://mdx-2020.eduid.hu/']:
type: <class 'saml2.mdstore.MetaDataMDX'>:
value (as string): dict_items([])

result = metadata.any("idpsso_descriptor", "single_sign_on_service") (L48): empty dict
I seems to me that the saml2.mdstore.MetaDataMDX is picked up correctly from the config, but it looks empty and thus no idps are found when I initiate the login. What else should I look for?

@peppelinux I've been doing some tests and come up with a solution, see https://github.com/IdentityPython/djangosaml2/pull/327 . I found that the MetaDataMDX object will be always empty unless a lookup is made, which will fetch the appropriate idp and djangosaml2 will see it as a legit metadata source. My minimal working fix may not be the best solution, but it works when the only and primary metadata source is MDQ.

got it @cheoppy, thank you
I've just requested some easy changes before merging it and have a new release

not at least, can we converge also to the following issue?
https://github.com/IdentityPython/djangosaml2/issues/273
we can pre-select the supported idps and pass to the MDX.
Just to tell you if you find something usefull also for you

Done here
https://github.com/IdentityPython/djangosaml2/releases/tag/v1.4.0


Point ISO URL to Ubuntu old releases CDN

Ubuntu released 22.04.3 today 3 hours ago and they migrated version 22.04.2 to the old-releases CDN, hence 22.04 builds can't find the 22.04.2 ISO image to pull from the current URL. Hence configuring the iso_url to be the old releases URL.
$ curl -sIo /dev/null  -w "%{http_code}" https://releases.ubuntu.com/22.04/ubuntu-22.04.2-live-server-amd64.iso
404

$ curl -sIo /dev/null -w "%{http_code}" https://old-releases.ubuntu.com/releases/jammy/ubuntu-22.04.2-live-server-amd64.iso
200

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/cherrypick release-0.17

/approve


Configuration modified to work with certbot installed in host

This should close #50
However, the volume link to "/etc/letsencrypt:/etc/letsencrypt" is not very elegant, there should be a better way but for now it works.
Also "server_name perf.test.fedcloud.eu;" should not provide the direct dns name (although probably can be removed)
Therefore is not ready to close, but to keep as branch for now.

With the above implemented redirect, you may now renew certbot with the webapp running and it should automatically reload the new certificates without having to shut down the app.


[Iam] Change update policy for cluster Iam section from SUPPORTED to IGNORED so that the update policy depends on the nested fields.

Description of changes
Change update policy for cluster Iam section from SUPPORTED to IGNORED so that the update policy depends on the nested fields. In fact, some of them are updatable and some others are not.
This change is part of the ResourcePrefix feature as ResourcePrefix is the first not updatable field in Iam section.
Important Notes
Additional unit tests should be added to cover this change and protect it from future regressions.
Such unit tests have been intentionally considered out of scope of this PR as per internal team agreement and will be implemented in an immediate follow up activity.
Tests
Manual tests:

[OK] ResourcePrefix specified => Iam section cannot be removed
[OK] Iam section omitted => Iam.ResourcePrefix cannot be added
[OK] Iam section omitted => Iam.PermissionsBoundary and Iam.Roles.LambdaFunctionsRole can be added
[OK] Iam.PermissionsBoundary and Iam.Roles.LambdaFunctionsRole specified => Iam section can be removed
Existing unit tests to verify no regression has been introduced.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Signed-off-by: Giacomo Marciani<EMAIL_ADDRESS>

Good work!


Setting the quantity on the SubscriptionBuilder only works when the user has a mandate

When I create a new subscription it looks like I can set the quantity on the SubscriptionBuilder:
$user->newSubscription($name, $plan)->quantity(10)->create();
This does work fine when the user already has a mandate, but is doesn't work when the payment is done using iDEAL.

Hi @sebastiaands ,
That's unfortunate. Can you share the payment 1) metadata and 2) amount? It's in the Mollie Dashboard.

Hi @sandervanhooft ,
Thank you for your response.
Hereby an example of the metadata:
{
  "owner": {
    "type": "App\\Organisation",
    "id": 34
  },
  "actions": [
    {
      "handler": "Laravel\\Cashier\\FirstPayment\\Actions\\StartSubscription",
      "description": "Abonnement voor één expertise",
      "subtotal": {
        "currency": "EUR",
        "value": "175.00"
      },
      "taxPercentage": "21.0000",
      "plan": "one_expertise",
      "name": "partners",
      "quantity": 4
    }
  ]
}

Thanks!
Preparing a fix in #71

Just released a fix


Add missing .mvn folder

Signed-off-by: Federico Valeri<EMAIL_ADDRESS>

Why should this be committed in? I think it should not be present in the repo.

Are these supposed to be downloaded automatically? I'm getting the following error:
./mvnw compile quarkus:dev
./mvnw: line 276: /Users/fvaleri/Documents/code/drain-cleaner/.mvn/wrapper/maven-wrapper.properties: No such file or directory
/Users/fvaleri/Documents/code/drain-cleaner/.mvn/wrapper/maven-wrapper.jar: No such file or directory
Error: Could not find or load main class org.apache.maven.wrapper.MavenWrapperMain
Caused by: java.lang.ClassNotFoundException: org.apache.maven.wrapper.MavenWrapperMain

Hmm, maybe we should remove the mvnw files as well.

That's fine for me, as I always use my own Maven version. You mean both mvnw and mvnw.cmd right?


Parse the description text in course details page and display it with style- convert text in bold, links.

Describe the feature you'd like
Parse the description text in course details page and display it with style- convert text in bold, links.
Screenshots

I am outreachy applicant, please can i work on this?

Assigned to you.

Will like to know if you have any preferable styling for this task?

We need to use a markdown converter I think. The text we are receiving is in markdown.
** TEXT ** -> should be converted to bold,
text-> should be treated like heading, ...

@ujjwalagrawal17 Can I work on this issue?

Yes @ankitadhiman003


SDR-2398 | Adds Models and /meta vendor endpoint

The following resource is missing a Response and Request Model /employees/{employeeId}/personal-address.
Aside from that the security looks good and it was able to load successfully.

@RickyAmparoCloud
Yes that's true
Since I couldn't find the vendor API for that.
Can you share the link to that, so that I can add it quickly

@ChinnaBabuSadam I wasn't actually able to find the API docs for the personal address.add operation but rather only found the docs for the .change operation. This is a weird one, not sure if it's better to add the model for the legal address of leave it without a model.
Here's the API documentation: https://developers.adp.com/articles/api/workers---personal-communication-management-v2-api

@RickyAmparoCloud
exactly that's what I confused with
So, I just left it alone and forwarded to @kellytgold the same


on-run-start, on-run-end should run for seeds

Should they also run for archives? Tests? Regardless, definitely for seeds:
https://github.com/fishtown-analytics/dbt/blob/development/dbt/node_runners.py#L496
https://github.com/fishtown-analytics/dbt/blob/development/dbt/node_runners.py#L351

on-run-{start|end} hooks run for seeds as of dbt v0.11.x. The SeedRunner inherits from the Model Runner here. Just validated that the hooks run locally. Closing this.


fix: publish fix

:tada: This PR is included in version 1.1.5 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Question About check_validity Function in Floorplan Generation

Hello authors, thank you for sharing your excellent work.
I’m curious about the purpose of the check_validity function in floorplan generation. When I skipped running it, even with overlapping polygons or room vertices with angles ≤90°, the generation worked fine.
Could you clarify its role and necessity?

This check_validity function will ensure each room is square and does not overlap with other rooms. You may still continue the generation after skipping this function, but it will result in buggy output.


fix versioning in istio

Per helm docs, version needs to be semver, appversion can be whatever we want.
We should minimally match appversion with the folder version

I assume it wwas 1.1.5 because that is where we forked from upstream. Are we comfortable losing that connection?

I assume it wwas 1.1.5 because that is where we forked from upstream. Are we comfortable losing that connection?
I see that the app version remains 1.1.5, so maybe that is enough

LGTM


[W6.1e][W11-A1] Sashankh Chengavalli Kumar

[LO-JUnit]Updated JUnit tests for isAnyNull().
Observed PR #796  to enhance understanding of exercise.

@ChengSashankh Please update the title of the PR to reflect your phase B team name.


cppQML: Device reboots when trying to start the weston, on arm32

For the cppQML template,  when trying to start the weston on arm32, the suddenly reboots.
Container logs
 *  Executing task: docker logs --tail 1000 -f 392187d569c80bd3f7a745d47b1df0047ca8166a4b905cb206fdf5ac5405e872

Switching VT tty1 to text mode if currently in graphics mode
Switching to VT 7
Removing previously created '.X*-lock' entries under /tmp before starting Weston. Pass 'IGNORE_X_LOCKS=1' environment variable to Weston container to disable this behavior.
 *  Terminal will be reused by tasks, press any key to close i
``

I think this was fixed by 9585e89f3515211fa492531c40e326418367fe52


Replace logo svg and correct typo

Required items, please complete
Change(s):

Change Logo svg file as the old one not showing up
Correct typo

Reason for Change(s):

Logo not showing up when installing solution

Version Updated:

no

Testing Completed:

yes

Checked that the validations are passing and have addressed any issues that are present:

Yes

Hello @FortiNDR-Integration, Thanks for raising this PR.
We will investigate this PR and provide you an update by 28 February, 2024 EOD


Sequencer: handle the case where blocks go backwards

There is a limitation in our current hacky implementation of L1 blocks in HotShot block headers, where the leader doesn't know about the previous L1 block, and thus, if their ETH client is lagging for any reason, may propose an L1 block that is actually before the parent's L1 block. This breaks a fundamental assumption of the OP node. So we need to add a rule that says if the L1 origin has gone backwards, the sequencer can just use the L2 parent's L1 origin, and otherwise carry on as normal.
I also need to think more about what happens if a HotShot block's timestamp goes backwards.

I think HotShot timestamps decreasing occasionally is OK for demo purposes.
The way the query service endpoint is currently written (https://github.com/EspressoSystems/espresso-sequencer/pull/555), it will always return the first block sequenced with timestamp at least equal to the start of the requested window. This is the same behavior we would have if the timestamps were always increasing. Furthermore, since prev is by definition sequenced before the first block in the window, it will always have timestamp less than the first block in the window, which is a supposed invariant of the relationship between prev and the first block.
For the end of the window, the query service takes the first block after the start of the window whose timestamp is at least end. This becomes next and the block before it becomes the last block in the window. Since next is by definition the first block in sequence order with timestamp at least than end, next has timestamp less than end, and thus the two blocks that define the end of the window also have decreasing timestamps.
The OP node only really cares about those four blocks: two defining the start of the window and two defining the end. It assumes that the blocks in the window have timestamps between the start and end, but we don't actually check this, and nothing breaks if it is violated. Technically the sequencer does check the timestamps of the middle blocks, but only to sanity check the data returned by the query service, so we can just turn those checks from errors into warning logs.


Consider changing process.nextTick to queueMicrotask

Browserify/Webpack will replace process.nextTick with a very inefficient shim which uses setTimeout(..., 0). Have you considered using queueMicrotask instead?
If you're concerned about browser support, I have a shim here: https://github.com/feross/queue-microtask

Yea I’m moving to that in the next major which will also have promise support for the stream handlers :)


TypeScript wrong this syntax highlighting in particular (but valid) class-method parentheses styling

VSCode Version: 1.31.1
OS Version: Windows 10.0.17134 Build 17134

class C {

  f: (g: Function) => any
  = g => g();

  h: () => string
  = () =>'a';

  k: (g: Function) => any
  =
  this.f
  (
    this.h
  )

}

Actual behavior
See how this.h above is is not highlighted the same way as this.f.
Expected behavior
This is valid code. this.f and this.h should be highlighted the same way.

Does this issue occur when all extensions are disabled?: Yes
EDIT
Note that the roughly equivalent JavaScript code is property highlighted.  So this is a TypeScript syntax highlight problem. Apparently, not a JavaScript one (I submit the edit after vscodebot labeled the issue as "javascript".
class C {

  f() {return g()};

  h(){return 'a'};

  k(){ return (
      this.f
      (
        this.h
      )
    )
  }

}

This is limitation of tmLanguage syntax that doesn't allow multi line lookups. Class could contain constructor signature with parameters on new line and hence we prefer that over expression with parenthesis starting on new line.


fixed reading nexxtLine Scanner issue with in switch case within whil…

fixed reading nextLine() Scanner issue with in switch case within while loop in AddressBookApp.java

Great work!


Create interface for emailing registrants

This is the UI and front end part related to the work Ben is doing for #285

I need a way to send payment confirmations. I know you are working on an emailing system. In the interim is there a way I can run a report showing basic registration info (kind of like the confirmation email) that shows what they paid and if they still owe anything?


Bump protobuf to 29.0

Please don't merge this, there is so much dependency cruft with this upgrade that needs explaining on their side, Mockito, Picolocli, OSGi tooling (not OSGi compatibility itself). Do a bazel REPIN and see the following:
DEBUG: /private/var/tmp/_bazel_dan/4062fc18797f0c796822bf92f1458040/external/rules_jvm_external~/private/rules/coursier.bzl:752:18: Found duplicate artifact versions
    com.google.guava:guava has multiple versions 33.3.1-jre, 32.0.1-jre
    org.mockito:mockito-core has multiple versions 5.14.2, 4.3.1
    info.picocli:picocli has multiple versions 4.7.6, 4.6.3

so many other libraries regressed to old versions
I'm not 100% certain but this: biz.aQute.bnd:biz.aQute.bnd.util is osgi buildtime cruft that's made it's way onto the runtime dependency list. Stuff that you use to create osgi dependencies but itself isn't required on the classpath, I could be wrong.
For sure: what on earth is mockito-core doing on a prod classpath.
Polite grumble opened here: https://github.com/protocolbuffers/protobuf/issues/19451


Tracking Admin Accounts (Analysis/Requirements)

For the UML designs that we have in the Analysis doc and for the lists we have in the Requirements doc we don't have "Administrator" listed under User attributes.
Do we have another way of determining if a user account is an admin account or do we need to make these additions?

No current plans to add admin functionality, but it is definitely a possibility for a later feature.


[resoto][docs] Update core args and link all component READMEs to the corresponding docs

Description
Update core cli args. Also put a link to each components docs in the READMEs.
Code of Conduct
By submitting this pull request, I agree to follow the code of conduct.

Codecov Report

Merging #779 (5df4465) into main (635a782) will decrease coverage by 0.00%.
The diff coverage is 85.89%.

@@            Coverage Diff             @@
##             main     #779      +/-   ##
==========================================
- Coverage   89.75%   89.74%   -0.01%
==========================================
  Files          69       69
  Lines        9737     9822      +85
==========================================
+ Hits         8739     8815      +76
- Misses        998     1007       +9

Flag
Coverage Δ

resotocore
89.74% <85.89%> (-0.01%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

resotocore/resotocore/cli/cli.py
94.15% <ø> (ø)

resotocore/resotocore/web/api.py
66.97% <33.33%> (+0.20%)
:arrow_up:

resotocore/resotocore/core_config.py
95.59% <50.00%> (-1.20%)
:arrow_down:

resotocore/resotocore/model/model.py
92.56% <84.21%> (-0.95%)
:arrow_down:

resotocore/resotocore/cli/command.py
87.20% <97.14%> (+0.26%)
:arrow_up:

resotocore/resotocore/__init__.py
100.00% <100.00%> (ø)

resotocore/resotocore/__main__.py
87.68% <100.00%> (ø)

resotocore/resotocore/dependencies.py
87.60% <100.00%> (+1.65%)
:arrow_up:

resotocore/resotocore/parse_util.py
100.00% <100.00%> (ø)

resotocore/resotocore/web/certificate_handler.py
94.00% <100.00%> (+0.31%)
:arrow_up:

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 690c1fd...5df4465. Read the comment docs.


Make image proxy filter tunable.

Adds IMAGE_PROXY configuration setting to change image proxy filter behaviour:

0 = Never proxy.
1 = Proxy non-HTTPS (default).
2 = Always proxy.

Added additional unit tests.
Addresses #142.

If accepted, I'll do a PR for the documentation.

If you don't mind, I would prefer having the config values as text, for example "none/http-only/all" or the ones you suggested "never/http/always". Seems more explicit to me than the numeric approach.
Thanks

Okay. I'll update the code accordingly.


activities with instantaneous control path

Is your feature request related to a problem? Please describe.
Consider the following example code:
activity LCD_printCounter (line: nat8, num: nat32)
    if EXT_lcdQueueIsFull() then
        await not EXT_lcdQueueIsFull()
    end
    _ = EXT_lcdQueuePutPrintCmd(line, num)
    await not EXT_lcdQueueIsFull() // just added to satisfy the compiler
end

activity Receiver (sysTick: bool)(leds: LedStates)
    var cRx: nat32
    [...] // instantaneous code
    repeat
        [...] // instantaneous code
        res = run COM_receive()(data, len) // Await next packet.
        [...] // instantaneous code
        run LCD_printCounter(15, true, successCount)
        [...] // instantaneous code
    end
end

Basically, there is a receiver activity which continuously receives packets from a communication interface. After each reception, the reception counter is incremented an shall be printed to the LCD. The LCD has a command queue which is capable of storing up to N commands to be executed on the LCD. The LCD activity checks whether it can put a new command into that queue or not. If the queue is full the activity will wait until there is at least one free slot in the queue. Otherwise, the activity will immediately put the new command in the queue.
At this point, the LCD activity actually could terminate because it has already done its job. The Blech compiler, however, enforces that all control paths within an activity must contain at least one await statement. So in order to get the code compiled I have added another unconditional await statement which is actually not required from the applications point of view and unnecessarily suspends the termination of the LCD activity.
Describe the solution you'd like
To summarize it: In this scenario I would need the possibility to implement an activity which might await but is not forced to so. In this case it might be required to not enforce the await statement being present in the LCD activity but in the calling Receiver activity instead.
Do you think that this could be feasible? Or maybe do you have a suggestion on how to deal with that problem in a different way?

If you don't want the extra await in LCD_printCounter I would inline the code directly into Receiver.  This way its up to the Receiver to deal with and control the stepping.

@frameworklabs Yes, you are right. I also considered this workaround. But LCD_printCounter is called at different locations within my code – not only in the Receiver (which is not obvious from the simplified example code). This means that I have to inline this check manually every time I want to call LCD_printCounter. In particular, this means that, although this check is actually part of the internal logic of LCD_printCounter, I cannot encapsulate it in the same activity. Instead, the caller has to implicitly know that this check is mandatory and has to insert it on each call.

I see you point but I like that run <activity> like await <cond> clearly mark the end/start of instantaneous code.
Also, the approach with guarding an await with an if seems to be some kind of "imperative synchronous programming pattern" already:
if not condition then
    await condition
end
<do something now that condition is met>

I do not see how I could avoid the conditional await in this case. If there are already free slots in the queue then I do not want to unnecessarily postpone the LCD command execution into the future. The command should be executed as soon as possible. Thus, I have to check the condition first before I decide to wait for it to become true.

In the current compiler a run <activity> guarantees that an await is contained.
The necessary condition is that every thread/trail must stop at least at one await (must not be instantaneous). This is currently guaranteed by either an await or a run <activity>.
@[EntryPoint]
activity main()
    ... // no await necessary
    cobegin
        run <activity>  // at least one await
    with
        run <activity>  // at least one await
    end
    ... // no await necessary
end

The compiler keeps track of the complete run hierarchy from the entry point activity.
These conditions could be weakened.
If an activity has an instantaneous path, such an activity could be classifed, for example as immediate.  An immediate activity does not guarantee to stop at an await.
The following program would be wrong, because the first forked thread/trail might not stop at an await.
@[EntryPoint]
activity main()
    ... // no await necessary
    cobegin
        run <immediate activity>  // at least zero awaits
    with
        run <non-immediate activity>  // at least one await
    end
    ... // no await necessary
end

Adding an await in every path of the thread/trail would make the program correct. For example:
@[EntryPoint]
activity main()
    ... // no await necessary
    cobegin
        ...
        await <cond>
        ...
        run <immediate activity>  // at least zero awaits
    with
        run <non-immediate activity>  // at least one await
    end
    ... // no await necessary
end

The compiler could keep track and could deduce/infer this immediate property.
The programmer could define an activity as immediate (new keyword to be chosen), telling the compiler I plan to use an instantaneous path - even if the current implementation is not instantaneous.
immediate activity noAwait()
    // allows and instantaneous path
end

Problem solved. Should we do this?

I somehow don't like this approach for two reasons:

The caller of an immediate activity might now have to add an unconditional await to his code block (if there is no other stop already).
At the call-site you don't see if this is an instantaneous activity or not complicating human reasoning about your code.

Actually, I do not want to decide or vote whether to implement this feature or not. With the above example I just wanted to show the following:
Today, Blech follows a black-and-white approach. Code abstraction is provided by activities and functions. Functions must never await, activities always have to. This leads to a very sharp distinction between instantaneous and reactive code – a very reasonable and clear separation of concerns which I really like and support!
My example (LCD_printCounter), however, shows that there might be use cases where I need to implement code in Blech which might be either reactive or instantaneous and where the decision is not done / known (statically) during development time but (dynamically) during runtime. As it turns out, today, I basically have one option to deal with that:
I explicitly have to separate the conditional, instantaneous part from the one which is guaranteed to await. In my example, this would mean that the caller of LCD_printCounter has to check whether the queue is full or not. While this approach would be perfectly in line with the synchronous-reactive programming scheme, I see several drawbacks from a software engineering point of view:

Violates "do not repeat yourself".
Low cohesion between things that belong together (queue check and queue manipulation).
Lack encapsulation -> queue check and queue manipulation are distributed across different activities / functions.
No clear abstraction / interface -> the caller has to do some steps (the queue check) in advance in order to make the function (queue manipulation) work properly.

To sum it up: How to properly abstract code in Blech which might be either reactive or instantaneous?

Actually, your initial solution does not have any of the drawbacks you listed - it might just not be optimal from a timing perspective (as you might waste a step when the queue is not full).
Moving the queue check to the callers can be seen as an optimization but you pay for it with the  drawbacks you mentioned.
I like the clear separation between instantaneous and stepping code as currently present, but if we want to blur the line I could see an alternative to the immediate activity approach mentioned:
What if await <condition> can be instantaneous instead?
Your code could look like this then:
activity print(stuff)
  await not queueIsFull() // if queue is not full await is instantaneous
  enqueuStuffToPrint(stuff)
end

I don't know of the consequences to the semantics of the language resulting from such a change, but await to wait for a time-step even if the condition is true in the current one was something that surprised me when I first familiarized with the synchronous MoC.

Adding to the above instantaneous await idea:
To have a non-instantaneous await you might need to say something like this:
await next <condition>
So, to wait for the next tick you would have to say await next true.

The non-instantaneous await is necessary for 2 reasons.

It syntactically slices the control flow into reaction blocks (the code to be executed in one step).  With an instantaneous await, the compiler cannot slice the program into reaction blocks, because the "places" where the execution pauses are determined from runtime properties.
Due to the same reason, the compiler can prevent infinitely many statements within one reaction. With instantaneous await the following could loop forever

repeat
    ...
    <instantenous> await cond
    ...
end

Traditional synchronous languages usually have a pause statement, which slices the control flow into reaction blocks. Blechs await condition can be expressed with pause.
repeat
    pause
until condition end

await true is just pause.
The academic language Quartz, has pause, await condition and immediate await condition - which is instantaneous.
As pointed out before immediate await condition is just
if not condition then
    await condition
end

Which could be expressed with pause, by inlining the await condition substitution.
if not condition then
    repeat
        pause
    until condition end
end

All this is only valid if the evaluation of condition does not have side effects, which is guaranteed in Blech.
In Blech we decided against pause and against immediate await.

I somehow don't like this approach for two reasons:

The caller of an immediate activity might now have to add an unconditional await to his code block (if there is no other stop already).
At the call-site you don't see if this is an instantaneous activity or not complicating human reasoning about your code.

Both are very good arguments. Given that Blech is made for easier reasoning about your code - for example by having separate input and output parameter lists -  I will try to work out something else ...

Actually, your initial solution does not have any of the drawbacks you listed - it might just not be optimal from a timing perspective (as you might waste a step when the queue is not full).

Timing is one aspect. Another point is that you have to find a reasonable condition that you can await which might not always be as straight forward as in the above example. Using await true as a general workaround wouldn't be a good solution from my point of view. Depending on how your Blech code is triggered, e.g. entirely event-driven without any periodic ticks, it is unclear at which point in time the next reaction (await true) will be performed. So LCD_printCounter might be suspended for several seconds, minutes or even hours until it has the chance to detect that the condition is (already) true. By this, your system might become inoperable.
I completely agree with both of you and I also do not like the approach to blur the borders between instantaneous and stepping code. However, I think that the use case described above is not only some special corner case. To generalize it: What I basically want to do is to access a resource in the C environment which is either free or occupied. If is already free (in the same reaction in which I decide that I want to use the resource) then I want to use it instantaneously. If it is occupied then I have to wait for the resource to become free again. The pattern could look like this:
activity UseResource ()
  if not isFree() then
    await isFree()
  end
  useResource() // now the resource is occupied
end

This pattern, however, leads to the problem mentioned above. So to overcome this I could add:
activity UseResource ()
  if not isFree() then
    await isFree()
  end
  useResource()
  await isFree() // wait for the resource to become free again
end

In principle, this solves the problem. This approach, however, fails if using the resource is actually an instantaneous operation. E.g. "using" the queue just means to push a new element to it. This happens instantaneously so that it would be possible to push multiple elements to the same queue within the same reaction. But maybe exactly this is the conceptual flaw of the approach that I tried to implement. Maybe, in the "synchronous world", the correct approach would be to distribute the insertion of N queue elements across N consecutive reactions. Still, the question remains on how to reasonably suspend the current reaction after each push operation...

Would it be too much of a hack in an event-driven trigger mode to inform the runtime via some flag or method call to immediately start the next step when the current one is finished?
activity useResource
  if not isAvailable() then
    await isAvailable()
    useResource() // BTW do act and fun have  separate identifier sets?
  else
    useResource()
    notifyWantsImmediateNextStep()
    await true
  end
end

Would it be too much of a hack in an event-driven trigger mode to inform the runtime via some flag or method call to immediately start the next step when the current one is finished?

Would be perfectly fine, could be done with an external C function (e.g. a software interrupt).

useResource() // BTW do act and fun have  separate identifier sets?

No, currently not. Would be very unusual. Some languages separate identifier sets for types and modules from variables and subprograms. Variables and subprograms usually fall into one category.

Here is my 2 cents:
I'd make LCD_printCounter a boolean function
function LCD_printCounter (line: nat8, num: nat32)
    if EXT_lcdQueueIsFull() then
        return false
    else
        _ = EXT_lcdQueuePutPrintCmd(line, num)
        return true
    end
end

then its usage pattern is an immediate-await style call
activity Receiver (sysTick: bool)(leds: LedStates)
    var cRx: nat32
    [...] // instantaneous code
    repeat
        [...] // instantaneous code
        res = run COM_receive()(data, len) // Await next packet.
        [...] // instantaneous code
        if not LCD_printCounter(15, true, successCount) then
            await LCD_printCounter(15, true, successCount)
        end
        [...] // instantaneous code
    end
end

The trick is that the side-effect free function LCD_printCounter will be tried with every tick until the queue is not full and it succeeds, which corresponds to your desired behaviour. Well, actually it obviously does have a side-effect (adding to the queue) but Blech does not know that and therefore would allow its usage in the context of an await.
 You decide.
The annoying bit is that whenever you want to use it you have to follow this pattern
if not foo() then
    await foo()
end

Maybe, just maybe, the syntactic sugar immediate-await foo() is not that bad after all?


Local docker host duplicates

Rancher Version: v0.63.1
Docker Version: 1.9.1
OS: OS x 10.10.4
Steps to Reproduce: I've added a local docker host one time
Results: Somehow, after sometime spent I see my local host added twice and three times.
Expected: Just one host, which was really added.

At the bottom of this closed issue, it explains why this is occurring.
https://github.com/rancher/rancher/issues/936

@maZahaca Let me know if you have further questions.

@deniseschannon Did I understand this point correctly and on regular linux machines it will be fine? I consider Rancher to run in production. But unfortunately, we have to restart our hosts sometime.

Yes, on regular machines it should be fine.


Group by plugin categories

Via the discussion in getnikola/nikola#1840.

Use Nikola to build the plugins site out of .plugin files.

We have a scaffolding ready:

we could just parse .plugin and requirements*.txt files with post.compiler.read_metadata and grab the plugin metadata from there,
take README.md as the post content,
write Nikola plugins to create .zip and .json files,
tweak the templates to display everything nicely,

and we’ll have a functional plugin site generated for us.  We will lose the no-refresh part, but a refresh is just something normal on the Internet.  (besides, DISQUS does not like this)
@ralsina, Opinions?  I could happily get it done by July 12th if you agree to solution 2.  I could also rewrite the themes site to use the same infrastructure.


Acominer with Mesa fork causes RX 480 graphics to crash unrecoverably.

This has happened twice now in the past two weeks, and both times the GPU fails to recover with these four messages:
[442366.409593] amdgpu:
                 failed to send message 200 ret is 0
[442368.471903] amdgpu:
                 last message was failed ret is 0
[442370.534578] amdgpu:
                 failed to send message 201 ret is 0
[442372.609531] amdgpu:
                 last message was failed ret is 0

Over and over again, about two seconds apart.
It is impossible to recover from this this without a hard reset. GPU reset fails, shutdown does not commence, magic sysrq does not reboot, and even the reset button on my case does not work. I have to actually hold the power button to reset it manually. This is very problematic, as it causes downtime for my mining pool and several servers.
Apparently dmesg only sends lines up to a certain amount, so the start of the error the first time on January 15th is lost. The second time, however, I did manage to catch it:
[535735.177927] [drm:amdgpu_job_timedout] *ERROR* ring gfx timeout, signaled seq=46058293, emitted seq=46058295
[535735.177942] [drm:amdgpu_job_timedout] *ERROR* Process information: process blender-2.93 pid 437 thread blender-2.:cs0 pid 2249
[535735.177947] amdgpu 0000:03:00.0: amdgpu: GPU reset begin!
[535735.375188] amdgpu 0000:03:00.0: amdgpu: BACO reset
[535735.562144] amdgpu 0000:03:00.0: amdgpu: GPU reset succeeded, trying to resume
[535735.563373] [drm] PCIE GART of 256M enabled (table at 0x000000F400500000).
[535735.563389] [drm] VRAM is lost due to GPU reset!
[535737.604174] amdgpu:
                 failed to send message 200 ret is 0
[535740.071713] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[535741.728737] amdgpu:
                 last message was failed ret is 0
[535743.791821] amdgpu:
                 failed to send message 100 ret is 0
[535745.538355] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[535745.539055] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[535745.899508] amdgpu:
                 last message was failed ret is 0
[535745.902206] amdgpu: SMU Firmware start failed!
[535745.902209] amdgpu: Failed to load SMU ucode.
[535745.902210] amdgpu: fw load failed
[535745.902211] amdgpu: smu firmware loading failed
[535745.902214] [drm] Skip scheduling IBs!
[535745.902216] [drm] Skip scheduling IBs!
[535745.902225] [drm] Skip scheduling IBs!
[535745.902226] [drm] Skip scheduling IBs!
[535745.902242] amdgpu 0000:03:00.0: amdgpu: GPU reset(2) failed
[535745.902278] [drm] Skip scheduling IBs!
[535745.902288] [drm] Skip scheduling IBs!
[535745.902295] [drm] Skip scheduling IBs!
[535745.902299] [drm] Skip scheduling IBs!
[535745.902308] [drm] Skip scheduling IBs!
[535745.902310] [drm] Skip scheduling IBs!
[535745.902320] [drm] Skip scheduling IBs!
[535745.902331] [drm] Skip scheduling IBs!
[535745.902343] [drm] Skip scheduling IBs!
[535745.902349] [drm] Skip scheduling IBs!
[535745.902353] [drm] Skip scheduling IBs!
[535745.902357] [drm] Skip scheduling IBs!
[535745.902360] [drm] Skip scheduling IBs!
[535745.902366] [drm] Skip scheduling IBs!
[535745.902371] [drm] Skip scheduling IBs!
[535745.902375] [drm] Skip scheduling IBs!
[535745.902380] [drm] Skip scheduling IBs!
[535745.902384] [drm] Skip scheduling IBs!
[535745.902388] [drm] Skip scheduling IBs!
[535745.902391] [drm] Skip scheduling IBs!
[535745.902396] [drm] Skip scheduling IBs!
[535745.902401] [drm] Skip scheduling IBs!
[535745.902404] [drm] Skip scheduling IBs!
[535745.902408] [drm] Skip scheduling IBs!
[535745.902411] [drm] Skip scheduling IBs!
[535745.902417] [drm] Skip scheduling IBs!
[535745.902421] [drm] Skip scheduling IBs!
[535745.902426] [drm] Skip scheduling IBs!
[535745.902430] [drm] Skip scheduling IBs!
[535745.902433] [drm] Skip scheduling IBs!
[535745.902436] [drm] Skip scheduling IBs!
[535745.902440] [drm] Skip scheduling IBs!
[535745.902443] [drm] Skip scheduling IBs!
[535745.902446] [drm] Skip scheduling IBs!
[535745.902449] [drm] Skip scheduling IBs!
[535745.902453] [drm] Skip scheduling IBs!
[535745.902456] [drm] Skip scheduling IBs!
[535745.902459] [drm] Skip scheduling IBs!
[535745.902462] [drm] Skip scheduling IBs!
[535745.902466] [drm] Skip scheduling IBs!
[535745.902469] [drm] Skip scheduling IBs!
[535745.902472] [drm] Skip scheduling IBs!
[535745.902476] [drm] Skip scheduling IBs!
[535745.902480] [drm] Skip scheduling IBs!
[535745.902483] [drm] Skip scheduling IBs!
[535745.902488] [drm] Skip scheduling IBs!
[535745.902492] [drm] Skip scheduling IBs!
[535745.902495] [drm] Skip scheduling IBs!
[535745.902499] [drm] Skip scheduling IBs!
[535745.902503] [drm] Skip scheduling IBs!
[535745.902506] [drm] Skip scheduling IBs!
[535745.902738] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[535745.903705] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[535745.903871] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[535745.909776] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[535745.909788] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[535745.917157] amdgpu 0000:03:00.0: amdgpu: GPU reset end with ret = -22
[535747.965685] amdgpu:
                 failed to send message 201 ret is 0
[535752.092718] amdgpu:
                 last message was failed ret is 0
[535754.156165] amdgpu:
                 failed to send message 282 ret is 0
[535756.159935] [drm:amdgpu_job_timedout] *ERROR* ring sdma0 timeout, signaled seq=4008009, emitted seq=4008011
[535756.159947] [drm:amdgpu_job_timedout] *ERROR* Process information: process  pid 0 thread  pid 0
[535756.159953] amdgpu 0000:03:00.0: amdgpu: GPU reset begin!
[535756.220125] amdgpu:
                 last message was failed ret is 0
[535758.287806] amdgpu:
                 failed to send message 170 ret is 0
[535758.289144] [drm:amdgpu_job_timedout] *ERROR* ring sdma1 timeout, signaled seq=634699, emitted seq=634701
[535758.289152] [drm:amdgpu_job_timedout] *ERROR* Process information: process  pid 0 thread  pid 0
[535758.289157] amdgpu 0000:03:00.0: amdgpu: GPU reset begin!
[535758.289159] amdgpu 0000:03:00.0: amdgpu: Bailing on TDR for s_job:9af4b, as another already in progress
[535758.643706] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[535761.295930] amdgpu:
                 last message was failed ret is 0
[535763.362051] amdgpu:
                 failed to send message 171 ret is 0
[535765.436241] amdgpu:
                 last message was failed ret is 0
[535767.501770] amdgpu:
                 failed to send message 200 ret is 0
[535769.566274] amdgpu:
                 last message was failed ret is 0
[535771.630977] amdgpu:
                 failed to send message 201 ret is 0
[535775.758325] amdgpu:
                 last message was failed ret is 0
[535777.822903] amdgpu:
                 failed to send message 261 ret is 0
[535779.886624] amdgpu:
                 last message was failed ret is 0
[535781.950258] amdgpu:
                 failed to send message 200 ret is 0
[535784.014566] amdgpu:
                 last message was failed ret is 0
[535786.076908] amdgpu:
                 failed to send message 201 ret is 0
[535790.207115] amdgpu:
                 last message was failed ret is 0
[535792.270633] amdgpu:
                 failed to send message 261 ret is 0
[535794.334658] amdgpu:
                 last message was failed ret is 0
[535796.398011] amdgpu:
                 failed to send message 200 ret is 0
[535798.462346] amdgpu:
                 last message was failed ret is 0
[535800.529184] amdgpu:
                 failed to send message 201 ret is 0
[535804.656539] amdgpu:
                 last message was failed ret is 0
[535806.720778] amdgpu:
                 failed to send message 261 ret is 0
[535810.849788] amdgpu:
                 last message was failed ret is 0
[535812.912689] amdgpu:
                 failed to send message 261 ret is 0
[535814.976085] amdgpu:
                 last message was failed ret is 0

If you know of anywhere better to send this issue, I would really appreciate that as well. I can get similar issues with some other Ethereum miners and with SteamVR, but am not sure if I can put this in a bug report to Mesa, since it uses a custom fork rather than the official one.
This seems to happen more often when I am doing something else in 3D. The first time, was when I launched the miner before Cities: Skylines had completely closed, which is relatively understandable since CS uses a crazy amount of VRAM. The second time, however, I was only zooming in in Blender, with Google Earth running far in the background. I thought I could avoid the crashes by not running anything GPU-intensive, but it appears that this is not the case.

The error is a generic GPU timeout, meaning that something got corrupted and the GPU didn't return results as expected.
The typical response to this is a GPU reset, however it seems that in your case the reset has also failed.
Since you run an atypical PCIe configuration, I think maybe GPU reset isn't working at all for you?  It's possible that Mesa is depending on PCIe atomics in some way, which does not work in your configuration as you mentioned before. There's a small chance that this is an acominer bug, although I don't think so given it happens with other miners too and no other users have reported such hangs.

Thank you so much for the quick and knowledgeable response!
If I reset right now when nothing is happening, resetting works acceptably, with a few glitches at first but overall success. My dmesg contains the following:
[ 5258.602302] amdgpu 0000:03:00.0: amdgpu: GPU reset begin!
[ 5258.874321] amdgpu 0000:03:00.0: amdgpu: BACO reset
[ 5259.060660] amdgpu 0000:03:00.0: amdgpu: GPU reset succeeded, trying to resume
[ 5259.061938] [drm] PCIE GART of 256M enabled (table at 0x000000F400500000).
[ 5259.061955] [drm] VRAM is lost due to GPU reset!
[ 5259.125771] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5259.332365] [drm] UVD and UVD ENC initialized successfully.
[ 5259.433361] [drm] VCE initialized successfully.
[ 5259.439057] amdgpu 0000:03:00.0: amdgpu: recover vram bo from shadow start
[ 5259.439075] amdgpu 0000:03:00.0: amdgpu: recover vram bo from shadow done
[ 5259.439093] amdgpu 0000:03:00.0: amdgpu: GPU reset(1) succeeded!
[ 5259.439159] [drm] Skip scheduling IBs!
[ 5260.124776] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5260.147738] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5260.299254] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5261.125595] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5261.800112] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5262.125534] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5262.125609] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5263.125618] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5263.300595] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5264.801740] amdgpu_cs_ioctl: 1 callbacks suppressed
[ 5264.801744] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5265.125582] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5266.124915] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5266.303067] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5267.126019] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5267.803650] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5268.125449] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5268.125524] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5269.009588] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5269.143642] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5270.125148] amdgpu_cs_ioctl: 2 callbacks suppressed
[ 5270.125153] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5270.805663] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5271.125429] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5272.125532] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5272.306532] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5273.125071] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5273.807558] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5274.125146] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5275.125142] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5275.308223] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5276.125679] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5276.809169] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5277.125857] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5278.126085] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5278.310105] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5279.125354] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5279.125448] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5279.811053] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5280.125636] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5281.125751] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5281.312006] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5282.124939] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5282.812832] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5283.125471] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5284.124768] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5284.313666] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5285.124461] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5285.148289] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5285.814552] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5287.125286] amdgpu_cs_ioctl: 1 callbacks suppressed
[ 5287.125291] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5287.315461] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5288.125452] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5288.816122] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5289.125110] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5290.125597] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5290.316520] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5290.316635] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5291.125326] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5291.817888] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5293.125287] amdgpu_cs_ioctl: 1 callbacks suppressed
[ 5293.125291] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5293.318223] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5294.125071] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5294.818622] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5295.124570] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5296.125003] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5296.318969] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5297.124711] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5297.819455] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5298.124403] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5299.124951] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5299.319941] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5300.124731] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5300.820387] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5301.124320] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5302.125889] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5302.320854] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5302.320975] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5303.124802] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5303.821449] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5305.125874] amdgpu_cs_ioctl: 1 callbacks suppressed
[ 5305.125878] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5305.321793] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5306.124635] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5306.822309] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5307.124206] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5307.148012] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5308.125322] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5308.323363] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5309.125863] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5309.824585] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5311.124991] amdgpu_cs_ioctl: 1 callbacks suppressed
[ 5311.124995] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5311.324901] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5312.124817] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5312.825613] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5313.124527] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5314.125054] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5314.125105] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5314.325921] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5315.125015] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5315.826768] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5317.125248] amdgpu_cs_ioctl: 1 callbacks suppressed
[ 5317.125252] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5317.327114] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5318.125262] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5318.828042] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5319.124928] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5320.125559] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5320.328389] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5321.125513] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5321.829222] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5322.125106] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5323.125787] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5323.329650] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5324.124727] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5324.830579] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5325.124706] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5326.124244] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5326.124285] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5326.149110] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5326.330613] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!
[ 5327.124851] [drm:amdgpu_cs_ioctl] *ERROR* Failed to initialize parser -125!

There have also been one or two times when SteamVR crashed things and the manual reset worked successfully as well. I am aware of this command and usually try to invoke it when things go wrong, but although it does not work 80+% of the time, it does work occasionally. I have also had automatic resets work at least once.
Would you happen to know what amdgpu_gpu_recover printing "-11" might mean? When I tried to reset the GPU during the first acominer-related crash, it always printed that when I tried to recover but I can find no results about what this actually means online.

Hmm, thank you, and I see that GPU reset is working fine (Failed to initialize parser -125 just means that the applications need to be restarted because their context are lost).

Would you happen to know what amdgpu_gpu_recover printing "-11" might mean? When I tried to reset the GPU during the first acominer-related crash, it always printed that when I tried to recover but I can find no results about what this actually means online.

It probably means that a GPU reset is already in progress, I don't know if -11 is actually an errno but if it's errno then it would mean "Resource temporarily unavailable". Since the GPU reset fails it probably will never end, bringing the system into a hanging state.
I honestly don't have an idea how this can be solved, but here's an attempt anyway.
https://github.com/ishitatsuyuki/acominer/actions/runs/1733336306

@happysmash27 Did you have a chance to try out the experimental build (ishitatsuyuki/acominer/actions/runs/1733336306)? No need to hurry, just let me know if it didn't work.


What is the best way to debug "Maximum call stack size exceeded"?

Sometimes when I'm coding I got the error screen generated by SSR. Mostly of time, a hard reload solves the problems, but sometimes not. What is the best way to debug and try to find the cause when you get Maximum call stack size exceeded error?

Hi @gtso86
This is a known issue, please see #1354
If you could help us with a reproduction, it would be awesome!


Maximum Bipartite Matching in cpp

Please resolve the merge conflicts :)

Okay will try that..!!

Removed merge conflicts please check

Thanks for the PR :) Merged.


Environment doesn't recover properly, after going back in once my laptop has gone back to sleep / period of inactivity passed.

Environment doesn't recover properly, after going back in once my laptop has gone back to sleep / period of inactivity passed.  Its doesnt ask me to reauthenticate, but things are in a "broken" state.  Open query windows dont seem to have a valid connection and the server explorer has a green status, but expanding the table view for example shows an error.

Closing older issues.  We've made several improvements for this scenario in the ADS shell.  Please let us know if you still experience this bug on current builds.  Thanks!


[FEATURE] extend KV support to generic_links table

Feature Description
generic_links table is being used for storing payout link resources at the moment. Redis is used as in-memory cache for faster r/w operations in a local store (database). This is implemented for a couple of tables at the moment. The feature requirement here is to extend KV support to the generic_links table.
Possible Implementation
There already exists an application interface for performing KV operations on Redis - KVRouterStore. The same needs to be implemented for generic_links table
Have you spent some time checking if this feature request has been raised before?

[X] I checked and didn't find a similar issue

Have you read the Contributing Guidelines?

[X] I have read the Contributing Guidelines

Are you willing to submit a PR?
None

Hey, can I pick this up?


Add unit and system level tests to validate the "Reload Bitfile" feature

[x] This contribution adheres to CONTRIBUTING.md.

What does this Pull Request accomplish?
Adds further unit tests which validates the functionality of the "Reload Bitfile" functionality.
Why should this Pull Request be merged?
Not all corner cases are covered with the current unit tests.
What testing has been done?
Run the tests locally and they all passed.

The idea of the the test is good, however the test itself does not show or prove that there was a change in the datatype. In both deployments, the channel values are read as doubles, therefore we have to "trust" the author and the description to be accurate and that a datatype change actually occurred - I believe that this is not necessarily defying the purpose of the test, but makes the purpose of the test less apparent and harder to debug (you need to check the bitfiles to see what exactly is happening.)
My opinion is that the Scripting API could and should be used to "prove" the data-types before the deployments, to validate the change for a control of the same name. Additionally, extra data-types should be used - like FXP for example to cover a wider range.

Please create non-generic icons for these VIs. Text-only will do just fine.

Do we have tests with Speciality IO? Would it make sense to add such tests? I'd like us to discuss it.


Updated the Auth overview docs to include to include note about updating existing projects

We've got this report https://github.com/dotnet/aspnetcore/issues/21825, where the customer missed to add the auth script reference in the index.html file.
Need to add a small section in the Overview article to include that.

Ok, but it's in the OIDC topic itself ...
https://docs.microsoft.com/en-us/aspnet/core/security/blazor/webassembly/standalone-with-authentication-library?view=aspnetcore-3.1#index-page
@yopez83, were you actually looking at https://docs.microsoft.com/en-us/aspnet/core/security/blazor/webassembly/standalone-with-authentication-library and missed a step for the script in the Index page?

@guardrex I was looking at https://docs.microsoft.com/en-us/aspnet/core/security/blazor/webassembly/?view=aspnetcore-3.1#feedback

The page I was looking at it's like an overview sort of speak, whereas the implementation details are covered in the page you mentioned @guardrex. I would say, a link will be great.

I would say, a link will be great.

Yes, that's best ............
I was just responding with some additional TOC background for this issue ...
The Overview doesn't contain any configuration/implementation guidance, nor is it supposed to contain such information. I've even been trying to factor it out where it occurs because readers don't like heavy-handed "overviews" ... they want just literally an overview. What happens is that they either totally skip the implementation guidance when reading docs down the TOC ... then miss guidance when they get into the more detailed topics below (or we end up duplicating content, which is bad), or they read it but it doesn't make a lot of sense because it's a lot of disconnected, seemingly random bits of coverage.
@mkArtakMSFT This is going to lead to 😵 Messy Docs:tm: 😵 if we go down this road of semi-randomly calling out implementation guidance in Overview topics. For this bit of TOC, readers should (in the base case) access the first topic (on the Auth Lib for standalone projects) for basic OIDC coverage, or ... OR we should kill the first topic and move all of it's coverage into the Overview as the base case. Actually tho, I'm kind'a happy right now with what we have.
The Additional resources of the Overview topic states ...

Articles under this Overview provide information on authenticating users in Blazor WebAssembly apps against specific providers.

Perhaps, we should make it more promoninent in the Overview ... I can make a section that calls out the cross-links. That way, the reader will get to the right topic for their provider and the Overview won't get bogged down in duplicate implementation guidance.

@guardrex I think your concern is valid and the link is a fine way to handle this. I looked through the PR and it turned out fine.


docs: Update cloud-init tutorial for 24.04.1

With that release comes changes to the distro launcher making it no longer necessary for users to run cloud-init commands. The distro launcher does it, so the tutorial can be made shorter and more direct.
Also there was a caveat about wsl --install that's no longer applicable, as that installation command also goes through the launcher, both workflows are now consistent.

CI Link checks are failing spuriously 🤷🏽♂

Question for a follow-up PR maybe: shouldn't we refer to this tutorial in the page screenshotted below?

Question for a follow-up PR maybe: shouldn't we refer to this tutorial in the page screenshotted below?

The auto-installation should probably just be removed (discussed previously with JB).
As this PR relates to updating the doc it should point to, I think we could remove it here.

Next cycle we need to consider if this tutorial and a few others might be a better fit in the how-to guides section. This, for example, includes concise, step-by-step instructions on how to achieve one specific task and could work better as a how-to.


Audio Forwarding stops after launching OBS

[x] I have read the FAQ.
[x] I have searched in existing issues.

Environment

OS: macOS
scrcpy version: 2.3.1
installation method: via homebrew
device model: OnePlus 11
Android version: 14

Describe the bug
scrcpy works, with audio forwarding with both wireless & usb cable, however, as soon as I launch OBS the audio is lost. I enabled 'Monitor and Output' in OBS advanced properties but that did not have any effect.
Would appreciate any help in troubleshooting.
I notice these debug logs when OBS is running which may hint at something odd. Without OBS, it's typically Buffer underflow.
DEBUG: [Audio] Buffering threshold exceeded, skipping 960 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 960 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 960 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 480 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 960 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 960 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 960 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 960 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 960 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 240 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 942 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 461 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 941 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 461 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 941 samples
DEBUG: [Audio] Buffering threshold exceeded, skipping 127 samples

I did another test on Linux and this does not seem to be a problem on linux so is macOS specific.

@rom1v Thanks for your response. Will give that a shot.


Make met_db_load more tolerant of one bad file amidst other good files

Replace italics below with details for this issue.
Describe the Enhancement
Because of all the trouble we've been having on Jet lately, some METplus workflows (especially MODE) have spawned bad stat files, where the file was created, but then Jet died in the middle of the job. This has been a problem loading the day's stat files into our database, because most of them are good and full of valid stats, but as soon as met_db_load encounters a bad file, it immediately throws an error and loads none of them into the database. Met_db_load instead needs to handle that error, skip the problematic file in question, and continue loading all of the other good files.
Time Estimate
1 hour

I have added the requisite try/except statements and will submit a pull request.


CommLink Unit Tests

[ ] All API calls work
[ ] All bound methods also work

Comlink abstraction is on branch feature_188


Add missing dataset property to example

The changes look fine. But why should we make the dataset configuration optional? Without providing information about the dataset the result files will be less self-contained.

I left out the dataset in one of the exmples. I'm reusing this branch to add that in.


Adding github workflow for contributing to helm

Describe detail step-by-step of using github to contribute to
helm project.
Co-Authored-By: Dao Cong Tien<EMAIL_ADDRESS>Signed-off-by: Nguyen Hai Truong<EMAIL_ADDRESS>

Thanks for your interest in our contributions process. This type of content should be in the CONTRIBUTING.md file. That is the standard file name GitHub expects. Some of this is already in that file. Would you mind trying to make the improvements there.

I've just resolved your comment, the content of CONTRIBUTING.md has been updated.

Why were the file permissions on CONTRIBUTING.md changed from 644 to 755?

Why were the file permissions on CONTRIBUTING.md changed from 100644 to 100755?

Thank you for your review. I've already resolved all of your changes request.

@mattfarina could you help me to review this change, please.

@truongnh1992 Thanks for the PR. We discussed it in the Helm Dev Call. It was a useful conversation.

@truongnh1992 Thanks for the PR. We discussed it in the Helm Dev Call. It was a useful conversation.

You're welcome. I'm willing to contribute to opensource project. By the way, how can I become a member of helm org on Github?

@truongnh1992 You can contribute without being a member of the Helm org on GitHub. The members of the org are those who have been or currently are maintaining a project that is part of Helm.


Create an FAQ on deployment anti-patterns

From @piyush-singh:

We have a surprising number of 2 node clusters in our telemetry as well, so wonder if it is worth putting a question about it in the FAQ or something.

We should create an FAQ entry on this topic. We can leverage/reference:

https://www.cockroachlabs.com/docs/stable/recommended-production-settings.html#topology
https://www.cockroachlabs.com/docs/stable/topology-patterns.html#anti-patterns

Closing in favor of https://github.com/cockroachdb/docs/issues/6651.


"Analyzing Dataset" modal appears after any change in the SQL query

I dismiss it, but it keeps showing up.  Suggest it stay away after being dismissed once for a given dataset.

duplicated. We will activate skip option tomorrow morning. It is like that for testing purposes :)


test: use newer ASSERT_MEM_EQ macro

Should hopefully make it easier to debug CI flakiness because currently the test sometimes fails without a clear indication why.

It did :)
2: not ok 53 - fs_event_watch_dir
2: # exit code -1073740791
2: # Output from process `fs_event_watch_dir`:
2: # Assertion failed in D:\a\libuv\libuv\test\test-fs-event.c on line 202: `filename == file_prefix` (subdir == fsevent-)

Now to make sense of that.

So, the file path suggests the fs_event_watch_dir is picking up an event from the fs_event_watch_dir_recursive test - but that would mean time travel, because fs_event_watch_dir runs before fs_event_watch_dir_recursive.


Docker Swarm and ETCD misconfigured issue

Hi ,
I am facing a lot of issues with bringing up the swarm image and using the etcd cluster. I was having similar issues with TLS , so i decided to start fresh. I have a standalone etcd instance with the latest version  of etcd ( No TLS)
[root@cdaetcd-ch2-2p varora200]# etcd --version
etcd Version: 2.2.4
Git SHA: bdee27b
Go Version: go1.5.3
Go OS/Arch: linux/amd64
[root@cdaetcd-ch2-2p varora200]#

I created a "swarm" key on the etcd server-
[root@cdaetcd-ch2-2p varora200]# curl -L http://<IP_ADDRESS>:4001/v2/keys/swarm/
{"action":"get","node":{"key":"/swarm","value":"","modifiedIndex":13,"createdIndex":13}}
[root@cdaetcd-ch2-2p varora200]#

I then proceeded to use the latest swarm image on a node , and following your documentation here (https://github.com/docker/swarm/blob/master/docs/discovery.md) , I am attempting step 1 ( Swarm join) and end up facing issues indicating that the etcd instance is misconfigured which i have NO clue why.
Here are my steps. I tried with 2376 since my docker is running on that port on the node
[root@docker-po-3p varora200]# docker run swarm -l "debug" join --advertise=<IP_ADDRESS>:2376 etcd://<IP_ADDRESS>:4001/v2/keys/swarm/
time="2016-01-29T21:31:18Z" level=debug msg="Initializing discovery service" name=etcd uri="<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-01-29T21:31:18Z" level=debug msg="Initializing discovery without TLS"
time="2016-01-29T21:31:18Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2376" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-01-29T21:32:18Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2376" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-01-29T21:32:18Z" level=error msg="client: etcd cluster is unavailable or misconfigured"
time="2016-01-29T21:33:18Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2376" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-01-29T21:33:18Z" level=error msg="client: etcd cluster is unavailable or misconfigured"
time="2016-01-29T21:34:18Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2376" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-01-29T21:34:18Z" level=error msg="client: etcd cluster is unavailable or misconfigured"

I also attempted on port 2375 ..just for kicks, same issue -
C[root@docker-po-3p varora200]# docker run swarm -l "debug" join --advertise=<IP_ADDRESS>:2375 etcd://<IP_ADDRESS>:4001/v2/keys/swarm/
time="2016-01-29T21:34:31Z" level=debug msg="Initializing discovery service" name=etcd uri="<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-01-29T21:34:31Z" level=debug msg="Initializing discovery without TLS"
time="2016-01-29T21:34:31Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2375" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-01-29T21:35:31Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2375" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-01-29T21:35:31Z" level=error msg="client: etcd cluster is unavailable or misconfigured"

I'm really not sure why that is the case , so i went through a few steps to validate that there is no firewall blocking -
I am able to ping from the node to the etcd server
[root@docker-po-3p varora200]# ping <IP_ADDRESS>
PING <IP_ADDRESS> (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=57 time=25.2 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=57 time=25.1 ms
64 bytes from <IP_ADDRESS>: icmp_seq=3 ttl=57 time=25.0 ms
64 bytes from <IP_ADDRESS>: icmp_seq=4 ttl=57 time=25.2 ms
^C
--- <IP_ADDRESS> ping statistics ---
5 packets transmitted, 4 received, 20% packet loss, time 4004ms
rtt min/avg/max/mdev = 25.068/25.180/25.263/0.132 ms

I am able to telnet
[root@docker-po-3p varora200]# telnet <IP_ADDRESS> 4001
Trying <IP_ADDRESS>...
Connected to <IP_ADDRESS>.
Escape character is '^]'.

So i know it can reach the etcd server from my node , no problem.
Here is also some validation i did on the etcd server

I validated i can curl to the key "swarm"

[root@cdaetcd-ch2-2p varora200]# curl -L http://<IP_ADDRESS>:4001/v2/keys/swarm/
{"action":"get","node":{"key":"/swarm","value":"","modifiedIndex":13,"createdIndex":13}}
[root@cdaetcd-ch2-2p varora200]#

Now you might ask me for etcd logs , so i went ahead and did a grep for that.

Here is my paste from running journalctl --since today | grep etcd
http://pastebin.com/xQ1c9mvr
I don't see anything in the etcd logs regarding the node attempting to connect etc. I've noticed a few people have pasted similar issues regarding "etcd being misconfigured" but given my situation above , I am at a loss and can't tell anything from the logs and would LOVE some insight as to where I am going wrong as I'd love to get this rolling as soon as possible. Thanks in advance !

Hi @vandita31, just tried with the same etcd version (2.2.4) starting etcd simply with:
etcd --data-dir=/tmp/default.etcd --listen-client-urls 'http://<IP_ADDRESS>:4001' --advertise-client-urls 'http://localhost:4001'

This works fine, but quick question: are you using CoreOS distribution with proxy mode for etcd? (which would in this case be related to: #972)

@abronan - I am using the latest release from-
https://github.com/coreos/etcd/releases/
(https://github.com/coreos/etcd/releases/download/v2.2.4/etcd-v2.2.4-linux-amd64.tar.gz)
Also , I am using systemctl to start etcd and keep it running so I'm not doing it manually and the configuration is as below
[root@cdaetcd-ch2-2p varora200]# cat /etc/etcd/etcd.conf
# [member]
ETCD_NAME=default
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_SNAPSHOT_COUNTER="10000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
#ETCD_LISTEN_PEER_URLS="http://localhost:2380,http://localhost:7001"
ETCD_LISTEN_CLIENT_URLS="http://<IP_ADDRESS>:4001"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#ETCD_CORS=""
#
#[cluster]
#ETCD_INITIAL_ADVERTISE_PEER_URLS="http://localhost:2380,http://localhost:7001"
# if you use different ETCD_NAME (e.g. test), set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
#ETCD_INITIAL_CLUSTER="default=http://localhost:2380,default=http://localhost:7001"
#ETCD_INITIAL_CLUSTER_STATE="new"
#ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_SRV=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#
#[proxy]
#ETCD_PROXY="off"
#
#[security]
#ETCD_CA_FILE=""
#ETCD_CERT_FILE=""
#ETCD_KEY_FILE=""
#ETCD_PEER_CA_FILE=""
#ETCD_PEER_CERT_FILE=""
#ETCD_PEER_KEY_FILE=""
[root@cdaetcd-ch2-2p varora200]#

Can you see anything in the above configuration that needs to be enabled? I seem to have the same environments enabled as mentioned in your example above - both the listen-client-urls and advertise-client-urls set.
Also , here is my systemd file just incase-
[root@cdaetcd-ch2-2p varora200]# cat /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
User=root
ExecStart=/usr/bin/etcd
NotifyAccess=all

[Install]
WantedBy=multi-user.target
[root@cdaetcd-ch2-2p varora200]#

Please let me know if anything seems off or what else i can do to troubleshoot this problem via etcd logs.

@abronan - To add to your suggestion above , I decided to test it out manually , so i stopped it using systemd -
[root@cdaetcd-ch2-2p varora200]# systemctl status etcd
etcd.service - Etcd Server
   Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled)
   Active: inactive (dead) since Mon 2016-02-01 18:37:11 UTC; 6min ago
  Process: 30707 ExecStart=/usr/bin/etcd (code=killed, signal=TERM)
 Main PID: 30707 (code=killed, signal=TERM)

Feb 01 17:09:20 cdaetcd-ch2-2p etcd[30707]: saved snapshot at index 790079
Feb 01 17:09:20 cdaetcd-ch2-2p etcd[30707]: compacted raft log at 785079
Feb 01 17:09:29 cdaetcd-ch2-2p etcd[30707]: purged file /var/lib/etcd/default.etcd/member/snap/000000000000000c-00000000000b4aea.snap successfully
Feb 01 18:32:40 cdaetcd-ch2-2p etcd[30707]: start to snapshot (applied: 800080, lastsnap: 790079)
Feb 01 18:32:40 cdaetcd-ch2-2p etcd[30707]: saved snapshot at index 800080
Feb 01 18:32:40 cdaetcd-ch2-2p etcd[30707]: compacted raft log at 795080
Feb 01 18:32:59 cdaetcd-ch2-2p etcd[30707]: purged file /var/lib/etcd/default.etcd/member/snap/000000000000000c-00000000000b71fb.snap successfully
Feb 01 18:37:11 cdaetcd-ch2-2p systemd[1]: Stopping Etcd Server...
Feb 01 18:37:11 cdaetcd-ch2-2p etcd[30707]: received terminated signal, shutting down...
Feb 01 18:37:11 cdaetcd-ch2-2p systemd[1]: Stopped Etcd Server.

I then started it manually
[root@cdaetcd-ch2-2p wal]# etcd --data-dir=/var/lib/etcd/default.etcd --listen-client-urls 'http://<IP_ADDRESS>:4001' --advertise-client-urls 'http://localhost:4001'
2016-02-01 18:37:46.273336 I | etcdmain: etcd Version: 2.2.4
2016-02-01 18:37:46.273441 I | etcdmain: Git SHA: bdee27b
2016-02-01 18:37:46.273451 I | etcdmain: Go Version: go1.5.3
2016-02-01 18:37:46.273460 I | etcdmain: Go OS/Arch: linux/amd64
2016-02-01 18:37:46.273472 I | etcdmain: setting maximum number of CPUs to 2, total number of available CPUs is 2
2016-02-01 18:37:46.273553 N | etcdmain: the server is already initialized as member before, starting as etcd member...
2016-02-01 18:37:46.274052 I | etcdmain: listening for peers on http://localhost:2380
2016-02-01 18:37:46.274188 I | etcdmain: listening for peers on http://localhost:7001
2016-02-01 18:37:46.274235 I | etcdmain: listening for client requests on http://<IP_ADDRESS>:4001
2016-02-01 18:37:46.276166 I | etcdserver: recovered store from snapshot at index 800080
2016-02-01 18:37:46.276185 I | etcdserver: name = default
2016-02-01 18:37:46.276194 I | etcdserver: data dir = /var/lib/etcd/default.etcd
2016-02-01 18:37:46.276218 I | etcdserver: member dir = /var/lib/etcd/default.etcd/member
2016-02-01 18:37:46.276228 I | etcdserver: heartbeat = 100ms
2016-02-01 18:37:46.276237 I | etcdserver: election = 1000ms
2016-02-01 18:37:46.276245 I | etcdserver: snapshot count = 10000
2016-02-01 18:37:46.276265 I | etcdserver: advertise client URLs = http://localhost:4001
2016-02-01 18:37:46.276312 I | etcdserver: loaded cluster information from store: <nil>
2016-02-01 18:37:47.033677 I | etcdserver: restarting member ce2a822cea30bfca in cluster 7e27652122e8b2ae at commit index 800622
2016-02-01 18:37:47.033932 I | raft: ce2a822cea30bfca became follower at term 12
2016-02-01 18:37:47.033987 I | raft: newRaft ce2a822cea30bfca [peers: [ce2a822cea30bfca], term: 12, commit: 800622, applied: 800080, lastindex: 800622, lastterm: 12]
2016-02-01 18:37:47.034909 I | etcdserver: starting server... [version: 2.2.4, cluster version: 2.2]
2016-02-01 18:37:47.035577 E | etcdmain: failed to notify systemd for readiness: No socket
2016-02-01 18:37:47.035604 E | etcdmain: forgot to set Type=notify in systemd service file?
2016-02-01 18:37:47.435175 I | raft: ce2a822cea30bfca is starting a new election at term 12
2016-02-01 18:37:47.435269 I | raft: ce2a822cea30bfca became candidate at term 13
2016-02-01 18:37:47.435286 I | raft: ce2a822cea30bfca received vote from ce2a822cea30bfca at term 13
2016-02-01 18:37:47.435318 I | raft: ce2a822cea30bfca became leader at term 13
2016-02-01 18:37:47.435343 I | raft: raft.node: ce2a822cea30bfca elected leader ce2a822cea30bfca at term 13
2016-02-01 18:37:47.437048 I | etcdserver: published {Name:default ClientURLs:[http://localhost:4001]} to cluster 7e27652122e8b2ae

I validated the process is running -
[root@cdaetcd-ch2-2p varora200]# ps -ef | grep etcd
avahi      714     1  0  2015 ?        00:00:21 avahi-daemon: running [cdaetcd-ch2-2p.local]
root     11000 10649  0 18:37 pts/0    00:00:02 etcd --data-dir=/var/lib/etcd/default.etcd --listen-client-urls http://<IP_ADDRESS>:4001 --advertise-client-urls http://localhost:4001
root     11119 11080  0 18:43 pts/1    00:00:00 grep --color=auto etcd
[root@cdaetcd-ch2-2p varora200]#

Does the way it start above look correct to you? Bceause my swarm issue still remains the same stating it is misconfigured.
[root@docker-po-3p varora200]# docker run atlas-registry.sys.comcast.net/automation/swarm -l "debug" join --advertise=<IP_ADDRESS>:2376 etcd://<IP_ADDRESS>:4001/v2/keys/swarm/
time="2016-02-01T18:42:02Z" level=debug msg="Initializing discovery service" name=etcd uri="<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-02-01T18:42:02Z" level=debug msg="Initializing discovery without TLS"
time="2016-02-01T18:42:02Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2376" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-02-01T18:43:03Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2376" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-02-01T18:43:03Z" level=error msg="client: etcd cluster is unavailable or misconfigured"
time="2016-02-01T18:44:03Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2376" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"

@abronan
Your comments on IRC -
 <abronan> varora200: taking a second look thanks for the added informations
[18:21] == rodlogic<EMAIL_ADDRESS>has quit [Remote host closed the connection]
[18:23] <abronan> varora200: I see nothing wrong and proxy mode is disabled, I also see that you tried curl'ing `<IP_ADDRESS>:4001` for keys, but can you try the same using the IP you give to the swarm command and see what's the result.
[18:24] <abronan> varora200: imho this is a connectivity issue and the client can't connect to that IP, if you can curl on that IP that should work though.

Here is my curl from the docker node
cker-po-3p varora200]# curl -L <IP_ADDRESS>:4001/v2/keys/swarm/
{"action":"get","node":{"key":"/swarm","value":"","modifiedIndex":13,"createdIndex":13}}
[root@docker-po-3p varora200]#

As you can see , i can access the etcd keys without any issues from the node , so it can connect to it without an issue

@abronan -
Updating ticket with testing using --net=host
[root@docker-po-3p varora200]# docker run --net=host atlas-registry.sys.comcast.net/automation/swarm -l "debug" join --advertise=<IP_ADDRESS>:2376 etcd://<IP_ADDRESS>:4001/v2/keys/swarm/
time="2016-02-02T00:41:24Z" level=debug msg="Initializing discovery service" name=etcd uri="<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-02-02T00:41:24Z" level=debug msg="Initializing discovery without TLS"
time="2016-02-02T00:41:24Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2376" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-02-02T00:42:24Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2376" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-02-02T00:42:24Z" level=error msg="client: etcd cluster is unavailable or misconfigured"

@abronan:
Attempting starting etcd using
[root@cdaetcd-ch2-2p varora200]# etcd --data-dir=/tmp/default.etcd --listen-client-urls http://<IP_ADDRESS>:4001 --advertise-client-urls http://<IP_ADDRESS>:4001
2016-02-02 00:57:10.915703 I | etcdmain: etcd Version: 2.2.4
2016-02-02 00:57:10.915815 I | etcdmain: Git SHA: bdee27b
2016-02-02 00:57:10.915849 I | etcdmain: Go Version: go1.5.3
2016-02-02 00:57:10.915861 I | etcdmain: Go OS/Arch: linux/amd64
2016-02-02 00:57:10.915874 I | etcdmain: setting maximum number of CPUs to 2, total number of available CPUs is 2

Docker swarm image -
[root@docker-po-3p varora200]# docker run --net=host atlas-registry.sys.comcast.net/automation/swarm -l "debug" join --advertise=<IP_ADDRESS>:2376 etcd://<IP_ADDRESS>:4001/v2/keys/swarm/
time="2016-02-02T00:58:37Z" level=debug msg="Initializing discovery service" name=etcd uri="<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-02-02T00:58:37Z" level=debug msg="Initializing discovery without TLS"
time="2016-02-02T00:58:37Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2376" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-02-02T00:59:38Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2376" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-02-02T01:00:38Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2376" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"
time="2016-02-02T01:01:38Z" level=info msg="Registering on the discovery service every 1m0s..." addr="<IP_ADDRESS>:2376" discovery="etcd://<IP_ADDRESS>:4001/v2/keys/swarm/"

@abronan -
The swarm join worked. However i get this error with swarm manage -
[root@docker-po-3p varora200]# docker run --net=host atlas-registry.sys.comcast.net/automation/swarm -l "debug" manage tcp://<IP_ADDRESS>:2376 etcd://<IP_ADDRESS>:4001/v2/keys/swarm/
time="2016-02-02T01:12:51Z" level=fatal msg="discovery required to manage a cluster. See 'swarm manage --help'."
 Are you trying to connect to a TLS-enabled daemon without TLS?"
time="2016-02-02T01:27:24Z" level=debug msg="Watch triggered with 1 nodes" discovery=etcd

@abronan -
Since my docker Daemon is using TLS to run , i assumed i will have to use swarm manage using the same certs as the daemon to connect , however  it refuses to recognize the arguments
I attempted the following -
[root@docker-po-3p ssl]#  docker run --net=host atlas-registry.sys.comcast.net/automation/swarm -l "debug" manage -H <IP_ADDRESS>:2380  --tlsverify --tlscacert=/app/support/docker/ssl/ca.pem --tlscert=/app/support/docker/ssl/dockerhost-cert.pem --tlskey=/app/support/docker/ssl/dockerhost-key.pem etcd://<IP_ADDRESS>:4001/v2/keys/swarm/
Got the following error
time="2016-02-02T02:54:06Z" level=fatal msg="Couldn't load X509 key pair (/app/support/docker/ssl/dockerhost-cert.pem, /app/support/docker/ssl/dockerhost-key.pem): open /app/support/docker/ssl/dockerhost-cert.pem: no such file or directory. Key encrypted?"
[root@docker-po-3p ssl]#

I moved the flags around , but got a different error -
[root@docker-po-3p ssl]# docker run --net=host --tlsverify --tlscacert=/app/support/docker/ssl/ca.pem --tlscert=/app/support/docker/ssl/dockerhost-cert.pem --tlskey=/app/support/docker/ssl/dockerhost-key.pem atlas-registry.sys.comcast.net/automation/swarm -l "debug" manage -H <IP_ADDRESS>:2380 etcd://<IP_ADDRESS>:4001/v2/keys/swarm/
flag provided but not defined: --tlsverify
See 'docker run --help'.
[root@docker-po-3p ssl]#

I also tried Without --net=host flag , no luck
[root@docker-po-3p ssl]# docker run --tlsverify --tlscacert=/app/support/docker/ssl/ca.pem --tlscert=/app/support/docker/ssl/dockerhost-cert.pem --tlskey=/app/support/docker/ssl/dockerhost-key.pem --net=host atlas-registry.sys.comcast.net/automation/swarm -l "debug" manage -H <IP_ADDRESS>:2380   etcd://<IP_ADDRESS>:4001/v2/keys/swarm/
flag provided but not defined: --tlsverify
See 'docker run --help'.

What is the right way to run this? I could not find any documentation and not sure why it complains about tlsverify? Could you update this ticket if I do not respond online please? Thanks :)

@vandita31 You need to mount the certs directory as a volume because you are using the docker Image to do that so using -v /mycerts /path/to/mycerts where /path/to/mycerts is your certs directory on the host. Also you should now use --tlscacert=/mycerts/ca.pem instead, etc. etc.
You can refer to the volumes documentation for that.

@vandita31 did this resolve your question?

@vandita31 I'm closing this one since the original question was solved (misconfigured --advertise-client-urls for etcd). But feel free to open a new one if you still encounter any issue. Also please take a look at the new TLS documentation if you want clear steps on how to setup TLS. Thanks!


bundlerEnv doesn't work with local "unbuilt" gems ("path" source)

Describe the bug
Running nix-shell on a derivation that references a gemset.nix created from a Gemfile with a "path" gem fails with an error message
/nix/store/npp6x63wjgnh2yarmva01i678x7l7nzz-bundler-2.3.9/lib/ruby/gems/3.0.0/gems/bundler-2.3.9/lib/bundler/source/path.rb:209:in `load_spec_files': The path `/nix/store/mn46x6k95dwib0w1nkvf4al1lkwr21qn-gemfile-and-lockfile/gems/foo` does not exist. (Bundler::PathError)

Steps To Reproduce
Given a Gemfile containing the line
gem 'foo', path: 'gems/foo'

when I run bundix this generates something like the following stanza in the gemset.nix
  foo = {
    dependencies = [];
    groups = ["default"];
    platforms = [];
    source = {
      path = gems/foo;
      type = "path";
    };
  };

and when I run nix-shell on a derivation that references this gemset.nix:

it doesn't install this gem anywhere - which is fine, because it's right here in my working directory

but: all of the created "wrapped" commands in the gemset - "rake" or "rails" or "bundle" etc - fail with

/nix/store/npp6x63wjgnh2yarmva01i678x7l7nzz-bundler-2.3.9/lib/ruby/gems/3.0.0/gems/bundler-2.3.9/lib/bundler/source/path.rb:209:in `load_spec_files': The path `/nix/store/mn46x6k95dwib0w1nkvf4al1lkwr21qn-gemfile-and-lockfile/gems/foo` does not exist. (Bundler::PathError)

As far as I can tell, this is because the wrapping copied my Gemfile{.lock} to a location in the store and set $BUNDLE_GEMFILE to that location. Ruby then decides to resolve the pathname in the Gemfile  'gems/foo' relative to that store location instead of the current directory.
Expected behavior
For interactive nix-shell use I think the behaviour I would like is to resolve the pathname relative to the working directory (or more precisely, to the directory that the Gemfile was copied from)

if there were some way to generate wrappers that don't set $BUNDLE_GEMFILE, that would actually work fine for this use case.  I don't know what the purpose of copying Gemfile* to the store is, but I assume it's needed for actual packaging when running nix-build instead of nix-shell.

alternatively maybe the copied Gemfile could be rewritten to do something like

gem 'foo', path: "#{ENV['PWD']}/gems/foo"

but TBH that seems like it would make things more brittle.
As a general observation, the Bundler docs say

gemfile (BUNDLE_GEMFILE): The name of the file that bundler should use as the Gemfile. This location of this file also sets the root of the project, which is used to resolve relative paths in the Gemfile, among other things. By default, bundler will search up from the current working directory until it finds a Gemfile.

(emphasis mine) which makes me think that setting it to someplace else is probably going "against the grain" of how the bundler authors expect things to work. But :shrug: it's ruby
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.60, NixOS, 22.05 (Quokka), 22.05.2590.9a91318fffe`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.8.1`
 - channels(dan): `""`
 - channels(root): `"nixos-22.05, telent-nixos-config"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Guessing tags based on history, apologies if I've summoned anyone who doesn't do this any more
@alyssais @nyarly @manveru @lheckemann

It looks like it would be not impossible to remove BUNDLE_GEMFILE from the wrapper, but at the expense of making installPhase in derivations a bit more drawn out. For example
diff --git a/pkgs/tools/networking/maphosts/default.nix b/pkgs/tools/networking/maphosts/default.nix
index 6632485b709..acf9cf779d8 100644
--- a/pkgs/tools/networking/maphosts/default.nix
+++ b/pkgs/tools/networking/maphosts/default.nix
@@ -1,4 +1,4 @@
-{ stdenv, lib, bundlerEnv, ruby, bundlerUpdateScript }:
+{ stdenv, lib, bundlerEnv, makeWrapper, ruby, bundlerUpdateScript }:

 let
   env = bundlerEnv {
@@ -10,11 +10,14 @@ in stdenv.mkDerivation {
   pname = "maphosts";
   version = env.gems.maphosts.version;

+  nativeBuildInputs = [ makeWrapper ];
+
   dontUnpack = true;

   installPhase = ''
     mkdir -p "$out/bin"
-    ln -s "${env}/bin/maphosts" "$out/bin/maphosts"
+    makeWrapper "${env}/bin/maphosts" "$out/bin/maphosts" \
+      --set BUNDLE_GEMFILE ${env.confFiles}/Gemfile
   '';

   passthru.updateScript = bundlerUpdateScript "maphosts";

Has to be said that wrapping a wrapper doesn't feel good, but in the grand scheme of things it's probably not a really big deal
Another approach might be to change the wrapper to be something like
ENV["BUNDLE_GEMFILE"] = #{gemfile.dump} unless ENV.key?('IN_NIX_SHELL')

though again I'm dubious about this, because it introduces differences between nix-shell and regular derivations

I have made a flake that is based on builderEnv here, mostly for my own learning

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/using-bundix-with-local-gems-or-rails-engines/14487/2

Related: https://github.com/nix-community/bundix/issues/84

@sagittaros I will look into it, is ruby-nix specific to ruby development or could it also work for packaging? (assuming that the path of the gem is inside the application source)

ruby-nix is simply a modified bundlerEnv, exposed as flake. It is designed for ruby development.
@zarelit  can you clarify what you meant by packaging? Are you writing a public gem?