Internal request doesn't send data

I am sending POST data (username and password) to /proxy where i add some more data to array and make Internal request to /oauth/access_token with new array.
$request = app('request')->only('username', 'password');

$credentials = [
    'grant_type'    => 'password',
    'username'      => $request['username'],
    'password'      => $request['password'],
    'client_id'     => env('OAUTH_CLIENT_ID'),
    'client_secret' => env('OAUTH_CLIENT_SECRET')
];

try{
    $response = $dispatcher->post('oauth/access_token', $credentials);
}catch (InternalHttpException $e)
{
    return response()->json($e->getResponse()->original);
}

But the problem is that data /oauth/access_token gets is only username and password from request to /proxy, and not new array.
  +request: ParameterBag {#130
    #parameters: array:2 [
      "username" => "Username"
      "password" => "Password"
    ]

What  did I do wrong?

I have same issue with internal request failing to get the response thanks to Transformer:
Argument 1 passed to RoboticAccounting\Api\v1\Http\Transformers\ClientGroupTransformer::transform() must be an instance of RoboticAccounting\ClientGroup, null given, called in /home/vagrant/projects/hdcrmwithdingoapi.local/vendor/league/fractal/src/Scope.php on line 307 and defined

The code in the ClientGroupTransformer:
<?php

namespace RoboticAccounting\Api\v1\Http\Transformers;

use League\Fractal\TransformerAbstract;
use RoboticAccounting\ClientGroup;

/**
 * ClientGroupTransformer
 *
 * @version 1.0.0
 * @author marius ionel<EMAIL_ADDRESS> */
class ClientGroupTransformer extends TransformerAbstract{

    public function transform(ClientGroup $client_group){

        return [
            //'id'=>(int) $client_group->id,
            'client_group'=>$client_group->name
        ];
    }
}

@lightvision, I think that might be something else. If this is an include then you'll need to ensure that the parent has a relation, otherwise it returns null instead of (in your case) an instance of RoboticAccounting\ClientGroup.
@mentos1386 are you retrieving the request input in your OAuth route or is it being handled by a separate package? I just did a quick test and I can send data internally. Just try a simple route to see if you can send internal data, but let me know how you're retrieving the data on the other end.

@jasonlewis You are right. It's not dingo issue while is a weird one, because the only wrong code was in a seeder, where an index was duplicated although the database wasn't migrated alt all.

I am sending request to function Authorizer::issueAccessToken() which of uses league/oauth2-server.
I belive the problem is, that Oauth2-server is using symfony/HttpFundation/Request. And this only gets:
  +request: ParameterBag {#121
    #parameters: array:2 [
      "username" =><EMAIL_ADDRESS>      "password" => "password"
    ]
  }

as illuminate/http/Request or Dingo\Api\Http\Request gets:
  +request: ParameterBag {#194
    #parameters: array:5 [
      "grant_type" => "password"
      "username" =><EMAIL_ADDRESS>      "password" => "password"
      "client_id" => "1"
      "client_secret" => "secret"
    ]
  }

Why is this difference between these to? And how can i fix it? Should i change the oauth2-server to use illiminate/request?

Are you using the bridge package (lucadegasperi/oauth2-server-laravel)?

Yeah. I’m sending request to it.
https://github.com/lucadegasperi/oauth2-server-laravel/blob/master/src/Authorizer.php

Let me set something up locally to see if I can get to the bottom of it. The bound request instance in the container should be set on the authorizer when it's updated. So it should be getting the updated request instance. I'll see what I can uncover.

All looks fine on my end. The request is being updated for me.
Can you open up League\OAuth2\Server\AuthorizationServer and down the bottom in the issueAccessToken method dump out $this->getRequest(). Just at the top.
Should look like this:
public function issueAccessToken()
{
    dd($this->getRequest());

    $grantType = $this->getRequest()->request->get('grant_type');
    if (is_null($grantType)) {
        throw new Exception\InvalidRequestException('grant_type');
    }

    // Ensure grant type is one that is recognised and is enabled
    if (!in_array($grantType, array_keys($this->grantTypes))) {
        throw new Exception\UnsupportedGrantTypeException($grantType);
    }

    // Complete the flow
    return $this->getGrantType($grantType)->completeFlow();
}

You should get an instance of Dingo\Api\Http\InternalRequest. That's what I'm seeing. If you're not... then there's something else wrong.
What version of the OAuth2 package are you using? 5.0.*@dev?

Request {#146
  #json: null
  #userResolver: null
  #routeResolver: Closure {#126
    class: "Laravel\Lumen\Application"
    this: Application {#3 …}
    file: "/home/vagrant/Projects/Threads/ThreadsAPI/vendor/laravel/lumen-framework/src/Application.php"
    line: "680 to 682"
  }
  +attributes: ParameterBag {#142
    #parameters: []
  }
  +request: ParameterBag {#145
    #parameters: array:2 [
      "username" => "Username"
      "password" => "Password"
    ]
  }
  +query: ParameterBag {#144
    #parameters: []
  }

// server bag removed... //

 +files: FileBag {#128
    #parameters: []
  }
  +cookies: ParameterBag {#130
    #parameters: []
  }
  +headers: HeaderBag {#133
    #headers: array:11 [
      "content-type" => array:1 [
        0 => "application/x-www-form-urlencoded"
      ]
      "content-length" => array:1 [
        0 => "35"
      ]
      "host" => array:1 [
        0 => "api.threads.app"
      ]
      "connection" => array:1 [
        0 => "keep-alive"
      ]
      "csp" => array:1 [
        0 => "active"
      ]
      "cache-control" => array:1 [
        0 => "no-cache"
      ]
      "origin" => array:1 [
        0 => "chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop"
      ]
      "user-agent" => array:1 [
        0 => "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/43.0.2357.130 Chrome/43.0.2357.130 Safari/537.36"
      ]
      "accept" => array:1 [
        0 => "*/*"
      ]
      "accept-encoding" => array:1 [
        0 => "gzip, deflate"
      ]
      "accept-language" => array:1 [
        0 => "en-US,en;q=0.8"
      ]
    ]
    #cacheControl: array:1 [
      "no-cache" => true
    ]
  }
  #content: null
  #languages: null
  #charsets: null
  #encodings: null
  #acceptableContentTypes: null
  #pathInfo: "/oauth/proxy"
  #requestUri: "/oauth/proxy"
  #baseUrl: ""
  #basePath: null
  #method: "POST"
  #format: null
  #session: null
  #locale: null
  #defaultLocale: "en"
}

Dump from request to /oauth/proxy https://gist.github.com/mentos1386/aa7adf972253822a7c59
Yes I am using 5.0.*@dev

Is this proxy route under the API routes or registered to the Laravel
router?
On Mon, 3 Aug 2015 21:59 Tine Jozelj<EMAIL_ADDRESS>wrote:

Request {#146
#json: null
#userResolver: null
#routeResolver: Closure {#126
class: "Laravel\Lumen\Application"
this: Application {#3 …}
file: "/home/vagrant/Projects/Threads/ThreadsAPI/vendor/laravel/lumen-framework/src/Application.php"
line: "680 to 682"
}
+attributes: ParameterBag {#142
#parameters: []
}
+request: ParameterBag {#145
#parameters: array:2 [
"username" => "Username"
"password" => "Password"
]
}
+query: ParameterBag {#144
#parameters: []
}
// server bag removed... //
+files: FileBag {#128
#parameters: []
}
+cookies: ParameterBag {#130
#parameters: []
}
+headers: HeaderBag {#133
#headers: array:11 [
"content-type" => array:1 [
0 => "application/x-www-form-urlencoded"
]
"content-length" => array:1 [
0 => "35"
]
"host" => array:1 [
0 => "api.threads.app"
]
"connection" => array:1 [
0 => "keep-alive"
]
"csp" => array:1 [
0 => "active"
]
"cache-control" => array:1 [
0 => "no-cache"
]
"origin" => array:1 [
0 => "chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop"
]
"user-agent" => array:1 [
0 => "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/43.0.2357.130 Chrome/43.0.2357.130 Safari/537.36"
]
"accept" => array:1 [
0 => "/"
]
"accept-encoding" => array:1 [
0 => "gzip, deflate"
]
"accept-language" => array:1 [
0 => "en-US,en;q=0.8"
]
]
#cacheControl: array:1 [
"no-cache" => true
]
}
#content: null
#languages: null
#charsets: null
#encodings: null
#acceptableContentTypes: null
#pathInfo: "/oauth/proxy"
#requestUri: "/oauth/proxy"
#baseUrl: ""
#basePath: null
#method: "POST"
#format: null
#session: null
#locale: null
#defaultLocale: "en"
}
Dump from request to /oauth/proxy
https://gist.github.com/mentos1386/aa7adf972253822a7c59
Yes I am using 5.0.*@dev
—
Reply to this email directly or view it on GitHub
https://github.com/dingo/api/issues/548#issuecomment-127209649.

Under API routes
        $api->post('oauth/proxy', 'OauthController@proxy');

        $api->post('oauth/access_token', function(){
            return response()->json(Authorizer::issueAccessToken());
        });

Damn, that's the last thing I can think of. Racking my brain here and I can't see anything different between what you've got and what I've got.
As an aside you shouldn't be using the response()->json() method, just return the Authorizer::issueAccessToken() directly. Same same for your internal exception handling in the first post. API will handle the JSON stuff.

Assuming you've got the API configured correctly I can't see how you getting the results you're getting. Double check everything, your prefix or domain, make sure it's all setup correctly.
Inside a standard get API route try the following: dd(app('request')); If the request object is NOT an instance of Dingo\Api\Http\Request then something is misconfigured somewhere or you've got some other code that replaces it.

Everything else works fine. I went over configuration, nothing that could be wrong (I think).
Here is the request from get API route with dd(app('request'));
Where could i see if request object is an instance of Dingo\Api\Http\Request ?
My composer.json if any packages are conflicting or wrong version.
    "require": {
        "laravel/lumen-framework": "5.1.*",
        "vlucas/phpdotenv": "~1.0",
        "dingo/api": "0.10.*@dev",
        "lucadegasperi/oauth2-server-laravel": "5.*@dev",
        "webpatser/laravel-uuid": "2.*@dev",
        "illuminate/pagination": "^5.1",
        "vinkla/hashids": "^2.0"
    },

Oh you're using Lumen. I tested Laravel. That could be the difference.
On Mon, 3 Aug 2015 23:02 Tine Jozelj<EMAIL_ADDRESS>wrote:

Everything else works fine. I went over configuration, nothing that could
be wrong (I think).
Here https://gist.github.com/mentos1386/3cdba4943d0ae8fd5957 is the
request from getAPI route with dd(app('request'));
Where could i see if request object is an instance of
Dingo\Api\Http\Request ?
My composer.json if any packages are conflicting or wrong version.
"require": {
    "laravel/lumen-framework": "5.1.*",
    "vlucas/phpdotenv": "~1.0",
    "dingo/api": "0.10.*@dev",
    "lucadegasperi/oauth2-server-laravel": "5.*@dev",
    "webpatser/laravel-uuid": "2.*@dev",
    "illuminate/pagination": "^5.1",
    "vinkla/hashids": "^2.0"
},

—
Reply to this email directly or view it on GitHub
https://github.com/dingo/api/issues/548#issuecomment-127223399.

Well it works fine for me on Lumen as well. Damn this is annoying.
You can see what type of request it is by hovering over the Request at the top, it should tell you the full namespaced name.
Those packages all look fine to me so I doubt it's a conflict anywhere.
In your oauth/access_token what do you get if you dump the request input?
dd(app('request')->input());

Only other thing I can think of is try some basic routes that shouldn't touch other packages:
$api->version('v1', function($api) {
    $api->get('foo', function () {
        return app('Dingo\Api\Dispatcher')->post('bar', ['name' => 'bob']);
    });

    $api->post('bar', function () {
        return app('request')->input();
    });
});

Expected response and what I'm seeing is:
{
    "name": "bob"
}

I tried the basic rutes, it workes.
{
  "name": "bob"
}

But this isnt using Symfony\Component\HttpFundation\Request that the oauth2-server is using.
Try this, it is how oauth2-server gets request, and it should return same as before:
    $api->get('foo', function () {
        return app('Dingo\Api\Dispatcher')->post('/bar', ['name' => 'bob']);
    });

    $api->post('bar', function () {
        return \Symfony\Component\HttpFoundation\Request::createFromGlobals()->request->all();
    });

But all I got is an empty array, could it be a problem with Symfony\Component\HttpFundation\Request ? Look at this https://github.com/dingo/api/issues/548#issuecomment-127185289

In your oauth/access_token what do you get if you dump the request input?
dd(app('request')->input());

I get:
array:5 [
  "grant_type" => "password"
  "username" => "Username"
  "password" => "Password"
  "client_id" => "1"
  "client_secret" => "secret"
]

Which is correct. So the problem must be with Symfony\Component\HttpFundation\Request

Okay another thing you can check for me is to open up League\OAuth2\Server\AbstractServer and find the getRequest method.
Debug in there to see if it's getting the request from the property or if it's building it's own. If it's building its own request then there's something strange happening. You can also try debugging the setRequest method, as this method should be called a couple of times to update the request instance.

setRequest()
Returns
  +request: ParameterBag {#147
    #parameters: array:2 [
      "username" => "Username"
      "password" => "Password"
    ]
  }

And  getRequest()
returns:
  +request: ParameterBag {#147
    #parameters: array:2 [
      "username" => "Username"
      "password" => "Password"
    ]

Both of them are instance of  Dingo\Api\Http\Request
Even app('request')
    public function setRequest($request)
    {
        $this->request = app('request');

        dd($this->request);
        return $this;
    }

that worked on bar->foo  route, doesn’t work here.
But weirdly, this works
    public function getRequest()
    {
        if ($this->request === null) {
            $this->request = Request::createFromGlobals();
        }

        return app('request');
    }

  +request: ParameterBag {#195
    #parameters: array:5 [
      "grant_type" => "password"
      "username" => "Username"
      "password" => "Password"
      "client_id" => "1"
      "client_secret" => "secret"
    ]
  }

Hm, I've just pushed an update. Try again without any modification to the files. Hopefully the rebound event is fired properly now.

Actually never mind, a closer inspection leads me to believe the change won't fix anything.
I'm stumped my friend. I cannot fathom why you're not getting an updated request. My advice at this stage would be to try a fresh install with the everything and see if you can replicate it. If you can, make a GitHub repository of everything that I can clone and try out locally.

@mentos1386 The @jasonlewis advice is the only good advice. I already mentioned that a bad migration file blows up my internal request even if that file was not migrated, and any way, has nothing to do with dingo/api.
So I had to start over and follow the TDD in order to not get the same weird issues.

@jasonlewis
Tried fresh, it works fine now, I have no idea what was messing up before.
And also, It would be good if instead of:
message": "500 Internal Server Error",
  "status_code": 500,
  "debug": {
    "line": 513,
    "file": "\\/home\\/vagrant\\/Projects\\/Hashtag\\/API\\/vendor\\/dingo\\/api\\/src\\/Dispatcher.php",
    "class": "Dingo\\\\Api\\\\Exception\\\\InternalHttpException",
    "trace": [

the dispatcher would return error that requested site returned, easier for debugging. Now all I know is that the requested site returned error, but I don’t know anything about the error itself.
Thanks for all the time.
@lightvision yeah, I should start using TDD as well

Internal requests should rethrow exceptions. Where are you seeing that?
On Tue, 4 Aug 2015 23:41 Tine Jozelj<EMAIL_ADDRESS>wrote:

@jasonlewis https://github.com/jasonlewis
Tried fresh, it works fine now, I have no idea what was messing up before.
And also, It would be good if instead of:
message": "500 Internal Server Error",  "status_code": 500,  "debug": {    "line": 513,    "file": "\/home\/vagrant\/Projects\/Hashtag\/API\/vendor\/dingo\/api\/src\/Dispatcher.php",    "class": "Dingo\\Api\\Exception\\InternalHttpException",    "trace": [
the dispatcher would return error that requested site returned, easier for
debugging. Now all I know is that the requested site returned error, but I
don’t know anything about the error itself.
Thanks for all the time.
@lightvision https://github.com/lightvision yeah, I should start using
TDD as well
—
Reply to this email directly or view it on GitHub
https://github.com/dingo/api/issues/548#issuecomment-127616549.

Yes it dose throw exception, the one i provided in previuse comment. But why modifying the message, why not just respond with the error the requested site gave.
For example:
{
  "message": "The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed. Check the \"grant_type\" parameter.",
  "status_code": 500,
  "debug": {
    "line": 266,
    "file": "\\/home\\/vagrant\\/Projects\\/Hashtag\\/API\\/vendor\\/league\\/oauth2-server\\/src\\/AuthorizationServer.php",
    "class": "League\\\\OAuth2\\\\Server\\\\Exception\\\\InvalidRequestException",
    "trace": [

This error gave requested site, but dispatcher modify it to be:
{
  "message": "500 Internal Server Error",
  "status_code": 500,
  "debug": {
    "line": 513,
    "file": "\\/home\\/vagrant\\/Projects\\/Hashtag\\/API\\/vendor\\/dingo\\/api\\/src\\/Dispatcher.php",
    "class": "Dingo\\\\Api\\\\Exception\\\\InternalHttpException",
    "trace": [

It would be easier for debugging, if dispatcher would throw original error.

Can I see your oauth route where the exception is thrown?
The only time this package will intercept and return an InternalHttpException is when your internal request returns an non-successful response. This is so that an exception is thrown for these internal requests.
If you just throw exceptions from your API routes as per normal then the dispatcher will not touch them.

I was having the same issue. Were you making your request via ajax?
When submitting via ajax it was leaving the Content-Type as application/json so when asking the request for the data it saw it as wanting json so returned the data inside php://input.
To get around this when using the with method I reset the content-type to what I want it to be.
e.g.
if ($this->with !== null) {
    $api->with($this->with);

    if ($this->json === null) {
        $contentType = ($method === 'get') ? 'text/html' : (($this->attach !== null) ? 'multipart/form-data' : 'application/x-www-form-urlencoded');
        $api->header('Content-Type', $contentType);
    }
}

That is just an example as I have a wrapper around the internal request.
Hope this helps everyone.


[Linux] Spotify installation not found. (arch)

🧐 Have you fully read the README and viewed similar issues?

[X] I have read the README
[X] I have read the FAQ
[X] I have checked previous issues, open and closed
[X] I have completed all troubleshooting steps and the issue is not present on stock but returns after re-applying SpotX-Bash

Сountry of your account
Poland
What is your Spotify Plan?
Free
Do you have Spicetify installed?
No
ℹ Computer information
- Spotify version: 1:<IP_ADDRESS>5-2 (latest from aur)
- OS: arch linux
- Shell: bash

📝 Description
so the script says that spotify isn't in the /opt/spotify path, even though it is there. i used the script like 2/3 hours ago, and it worked just fine with the same OS etc. apprarently some of the changes broke it. i've already tried rebooting, setting the path via -P and so on, but nothing worked.
🖥 Terminal Input/Output
Error: Spotify installation not found.
Install Spotify or set Spotify directory path with '-P' flag.
📸 Screenshots

This should be sorted now. Was an issue with the case of the path.
Running the command again will work as long as you're not pulling a cached page.


Update question definitions to support multiple content

What is the context of this PR?
Modified definitions structure to allow content so we can display bullet points etc. Needed for: trello.com/c/rC82gUmp/2180-move-significant-changes-to-use-definition-pattern
How to review

Make sure app works as expected.
Definitions to now allow multiple contents ie multiple descriptions, lists etc.

Checklist

[x] New static content marked up for translation
[x] Newly defined schema content included in eq-translations repo

Related PRs:

https://github.com/ONSdigital/eq-schema-validator/pull/72
https://github.com/ONSdigital/eq-translations/pull/20

@MebinAbraham if I'm not mistaken, there is only one example of using the definition content. That one uses the description key, but perhaps it would be good to also use the title and list keys somewhere? Just to make sure no weird bugs around that creep in later.
Should there not also be at least an e2e test to ensure the content appears and is interactive as expected.
In both cases, that's not because I suspect the improvements don't work now, but just that future changes to other areas may have unintended consequences which go un-noticed without tests :)


duplicate lines

Hi!
this is not a bug and does not affect operation
custom_url and full_base_url are defined in the "head" then, these lines are duplicated in "home" and "header"
{% assign custom_url = site.url | append: site.baseurl %}
{% assign full_base_url = custom_url | default: site.github.url %}

Bye!

I think the issue has been fixed a few days ago. Please re-check it and close if it's really fixed. :wink:

Fixed in this commit. Thanks @zdroid for reminding me

No problem. :smile:


Needs to specify pydantic dependency

There is an import of pydantic that uses 2.x functions. This should be specified in the pyproject.toml

Fixed in release 0.4


consider replacing private key wrangling with subprocess running all bash commands and letting system figure out how to connect to ssh? (but then, how to pass options to ssh if needed, like -i?)

this doesn't make it easier to detect that the ssh user may not be root, it doesn't help us figure out which domain we're connecting to, etc...


Updating Event Hubs product name

Docs Build status updates of commit ecc9163:
:clock10: Full build: the average full build time is 129 min(s) 36 sec(s), this is based on the last 122 build(s) for this repository.

Docs Build status updates of commit ecc9163:
:warning: Validation status: warnings

File
Status
Preview URL
Details

preview/docs-ref-autogen/azure-core/azure.core.polling.yml
:warning:Warning
View (azure-python-preview)
Details

docs-ref-mapping/reference-preview.yml
:white_check_mark:Succeeded

preview/docs-ref-autogen/azure-core/azure.core.polling.yml

[Warning] Uid(azure.core.polling.async_poller) has already been defined in preview/docs-ref-autogen/azure-core/azure.core.polling.yml.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit d731960:
:clock10: Full build: the average full build time is 128 min(s) 23 sec(s), this is based on the last 143 build(s) for this repository.

Docs Build status updates of commit d731960:
:warning: Validation status: warnings

File
Status
Preview URL
Details

preview/docs-ref-autogen/azure-core/azure.core.polling.yml
:warning:Warning
View (azure-python-preview)
Details

docs-ref-mapping/reference-preview.yml
:white_check_mark:Succeeded

preview/docs-ref-autogen/azure-core/azure.core.polling.yml

[Warning] Uid(azure.core.polling.async_poller) has already been defined in preview/docs-ref-autogen/azure-core/azure.core.polling.yml.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Move e2e tests to bazel-integartion-testing.

Tests now run through full versions of Bazel. This will allow testing
against a matrix of Bazel versions while only maintaining build
compatibility with a single Bazel and rules_go.

LGTM, but I'll defer to Alex or Dan to approve this.


Don't override user's vertical separator char

It's very confusing when a color scheme overrides user's GUI settings, like set fillchars+=vert:│.
I personally prefer BOX DRAWINGS HEAVY VERTICAL (┃).

@cronokun Thanks for using Nova and for the feedback. Feel free to submit a pull request, otherwise I'll get to this :)

@cronokun This was released in https://github.com/trevordmiller/nova-vim/releases/tag/v3.15.0. Please upgrade and let me know if you have any problems. Thanks!

@trevordmiller Thanks! And thanks for awesome color scheme! 👍


UI Cleanup

Can you guys provide description here?
cc: @alexwhte @just-a-node

This is a placeholder for CH changes in the repo.


Upgrade sinon version to v9

Hi,
SAP UI5 only integrate sinon v4 now, but there are many new features after sinon v4, such as sinon.fake(). When I work on UT, I refer to the latest document. I think it's time to upgrade the sinon version. Last time UI5 upgrade the version is in 2018. Thank you.

Hi @qjkobe ,
I have created an internal incident<PHONE_NUMBER>, to further process the same.
Thanking you.
Best regards,
Jay

BTW, there is already sinon: 10.0.0.
P.S. It would be great to update all UI5's dependencies.

Hi @qjkobe / @pubmikeb ,
thank you for you suggestions, but we cannot upgrade the existing files (sap/ui/thirdpary/sinon.js and sap/ui/thirdparty/sinon-4.js, but also sap/ui/thirdparty/qunit and sap/ui/thirdparty/qunit-2.js) with new major versions as they're used in thousands of (inhouse) tests in higher application layers that are not under our control.
Adding a new version next to the existing ones would be possible, but we decided against this as it over time would pollute the sap.ui.core project with thirdparty software that we couldn't get rid of.
We rather recommend to package and use the desired Sinon (or Qunit) versions on consumer side. As test frameworks are typically used in an isolated manner, having them per app / library doesn't hinder too much. For bigger projects, a reuse across multiple apps / libraries is also possible, but should be limited to a scope where updates can be managed (under organisational aspects).
For an example how to integrate thirdparty into a UI5 library, you might take a look at the https://github.com/ui5-community/ui5-library-showcase/tree/main/packages/ui5-library-showcase sample.


Expected empty scheme parsing "host:port"

Hello. I tried to parse a string without a scheme like localhost:5555 and expected an empty scheme, but instead got an empty host and port. Parsing  <IP_ADDRESS>:5555 and [::1]:5555 fails. Is this expected behavior? Are these not valid URIs? Thank you.
auto result = parse_uri("localhost:5555");
auto hasValue = result.has_value(); // true
auto url = result.value();
auto hasScheme = url.has_scheme(); // expect false, but got true
auto scheme = url.scheme(); // expect "", but got "localhost"
auto host = url.host(); // expect "localhost", but got ""
auto port = url.port_number(); // expect "5555", but got 0

Looks like a misparse indeed. Here's my repro on develop:
#include <boost/url.hpp>
#include <boost/url/src.hpp>
#include <iostream>

#define INSPECT(x) std::cout << #x << ": " << std::boolalpha << (x) << std::endl;
int main() {
    auto result = boost::urls::parse_uri("localhost:5555");
    std::cout << "error: " << result.error().message() << std::endl;
    INSPECT(result.has_value()); // true

    auto url = result.value();
    INSPECT(url.has_scheme());   // expect false, but got true
    INSPECT(url.scheme());       // expect "", but got "localhost"
    INSPECT(url.host());         // expect "localhost", but got ""
    INSPECT(url.port_number());  // expect "5555", but got 0
}

Prints
error: Success
result.has_value(): true
url.has_scheme(): true
url.scheme(): localhost
url.host():
url.port_number(): 0

Broseph, you need to be using parse_authority not parse_uri.

parse_uri parses a URI, which has a precise syntax. "localhost:5555" is in fact a valid URI with scheme "localhost" and path "5555". How is the library supposed to know that you wanted an authority?

I'm not saying it was wrong. I'm saying it was surprising, and I wondering how to fill that common use-case I described: validating user input.
I suppose people should check that the parse succeeded and got at least a scheme and authority.

I can't think of anything more we can do about it

Checking the RFC indeed users should additionally to check has_authority().
In laymen's terms, this guarantees that the parsed URI contained //.

I can't think of anything more we can do about it

We can document use-cases that casual users may not be aware of. This is what I consider "educating users", and is a basic requirement for responsible library use, IYAM.
What about a sample showing basic URL validation? I can propose one

Just as a sanity check, I compared with e.g. python3:
from urllib.parse import urlparse
urlparse('http') # ParseResult(scheme='', netloc='', path='http', params='', query='', fragment='')
urlparse('http:') # ParseResult(scheme='http', netloc='', path='', params='', query='', fragment='')
urlparse('localhost:5555') # ParseResult(scheme='', netloc='', path='localhost:5555', params='', query='', fragment='')
urlparse('localhost:') # ParseResult(scheme='localhost', netloc='', path='', params='', query='', fragment='')

I haven't nearly spent enough time reading the RFC to rule out what implementation is more correct

@kalikin, you can check this table to find what function you need:

Function
Grammar Element
Grammar Rule

parse_absolute_uri
absolute-URI
absolute-URI    = scheme ":" hier-part [ "?" query ]

parse_relative_ref
relative-ref
relative-ref    = relative-part [ "?" query ] [ "#" fragment ]

parse_uri
URI
URI             = scheme ":" hier-part [ "?" query ] [ "#" fragment ]

parse_uri_reference
URI-reference
URI-reference   = URI / relative-ref

As you can see, "localhost:5555" is not a valid URI because this would require a scheme. This is a common mistake because HTTP URLs are very common and HTTP normalization would include the "http" scheme and port 80 for us when they're missing, among other things. But the URL protocol cannot assume a specific scheme or port.
In your case, the correct parsing function depends on what you are trying to achieve. You might have a relative-ref or just an authority_view.

https://master.url.cpp.al/url/ref/boost__urls__parse_relative_ref.html
https://master.url.cpp.al/url/parsing/authority.html#url.parsing.authority.authority_view

The complete rules for URIs are in https://master.url.cpp.al/url/parsing/url.html

I see, http and various other well-known schemes are treated as a special case:
https://github.com/python/cpython/blob/ed75d2183f9084583e2ebe9229c52711d4a62f1e/Lib/urllib/parse.py#L47
I think both of the examples I gave above will be treated as a path, since xyz is not a known scheme.

By the way, Boost.URL and javascript's URL have the same results here:
> new URL('localhost:5555')
URL {
  href: 'localhost:5555',
  origin: 'null',
  protocol: 'localhost:',
  username: '',
  password: '',
  host: '',
  hostname: '',
  port: '',
  pathname: '5555',
  search: '',
  searchParams: URLSearchParams {},
  hash: ''
}

@vinniefalco Well

I think it's fair to say that the situation is fundamentally ambiguous UNLESS you assume left-to-right greedy parsing. Actually, that's a very good strategy, which is easy to document.
If it turns out that the RFC has normative wording specifying this behaviour, all the better

As more issues of this kind are created, I think the only thing actionable here is to add something here and there in the docs about normalizations this library won't do, such as returning port 80 when the port is missing.
For each normalization the library doesn't do, we would need to mention where the expectation usually comes from (like HTTP normalization instead of URL normalization), why this is not in the URL spec, and how to do it properly.
Unfortunately, in this case, I can't explain where the expectation that "localhost:2342" would be normalized to "http://localhost:2342/" comes from. I know it's something browsers do, but I don't think this is part of the HTTP normalization spec.

Returning port 80 is not a normalization. Also, it's incorrect, as it would break some servers that would not accept server:80 in the Host header, and some that specifically require it (binance.com API, looking at you again). That's clearly domain logic.
It's more important to simply document that the URI may be more general than people may expect, so they might need additional validation after a succesful parse.

in this case, I can't explain where the expectation that "localhost:2342" would be normalized to "http://localhost:2342/" comes from

You and me both, because you're literally the first person to suggest that expectation. What was unexpected was that localhost be parsed as the scheme. Nobody said /anything/ about assuming defaults or even normalizing

Returning port 80 is not a normalization.

It depends on what you mean by normalization. The URL spec defines normalization and comparisons with the same rules, where comparison is just string comparison after normalization. Then there's String Comparison < Syntax-Based Normalization/Comparison < Scheme-Based Normalization/Comparison < Protocol-Based Normalization/Comparison.
The URL normalization in this library is and can only be Syntax-Based, because Scheme-Based normalization/comparison would depend on the scheme. So if you mean syntax-based normalization, then no, port 80 should not be implied.
Then the HTTP spec defines its own normalization/comparison rules, which would be called Scheme-Based Normalization in the URL spec. So if you mean HTTP scheme-based normalization, then yes, port 80 should be assumed:

The default port is TCP 80 [19], but other ports can be used.
(...)
A port that is empty or not given is equivalent to the default port for that URI-reference;
(...)
For example, the following three URIs are equivalent:
http://abc.com:80/~smith/home.html
http://ABC.com/%7Esmith/home.html
http://ABC.com:/%7esmith/home.html

It's part of HTTP URL normalization but not URL normalization.

Also, it's incorrect, as it would break some servers that would not accept server:80 in the Host header, and some that specifically require it (binance.com API, looking at you again). That's clearly domain logic.

That's an unrelated issue. Whether server A or server B or API C require server:80 or server has nothing to do with what correct according to the URL and HTTP RFC. Even what the URL spec says is not the same as the HTTP spec says about it.

It's more important to simply document that the URI may be more general than people may expect, so they might need additional validation after a succesful parse.

It might be a little more complicated than that, because some scheme-based normalization rules allow inputs that are not valid URIs according to the URL spec.
For instance, your browser will put http: in front of www.boost.org for you. This normalization is not a guess. It's part of some other spec.
The problem is www.boost.org is not a valid URI according to RFC3986. So it's not a matter of additional validation because www.boost.org would return an error in this library.
So it's a little more intrusive because we would need some other functions for that, like parse_http_url or something.

You and me both, because you're literally the first person to suggest that expectation.

I don't know where the expectation comes from officially. I'm saying I literally don't know from which specification it is.
The informal user expectation/intuition is quite easy to explain. Just type "localhost:2342" in your browser and it will normalize it to "http://localhost:2342/" for you.

What was unexpected was that localhost be parsed as the scheme.

Yes. Because people just type "localhost:2342" in their browsers and it normalizes to "http://localhost:2342/" for them, where "localhost:2342" is the host and port. Therefore, they assume parse_uri("localhost:2342") would also give them a host and port, ignoring the browser did an extra step for them. This expectation is against the URL spec.

Nobody said /anything/ about assuming defaults or even normalizing

All of the comments here are about expectations people create because of normalizations that happen here and there. We have issues related to that all the time.
If there's no intention to discuss the effect of expectations and normalizations on the documentation, there's nothing else to discuss beyond this table, which is self-explanatory, and the issue can be closed:

Function
Grammar Element
Grammar Rule

parse_absolute_uri
absolute-URI
absolute-URI    = scheme ":" hier-part [ "?" query ]

parse_relative_ref
relative-ref
relative-ref    = relative-part [ "?" query ] [ "#" fragment ]

parse_uri
URI
URI             = scheme ":" hier-part [ "?" query ] [ "#" fragment ]

parse_uri_reference
URI-reference
URI-reference   = URI / relative-ref

Let's try not to waste more time explaining imputed expectations.
This issue was about expectations, not why they happen.
If that table is from the docs, indeed close this issue, because should be done.

We see issues of this kind very often. It's not a complete waste of time to discuss how we can improve the docs to avoid these. But yes, we can just keep track of new issues of this kind as they appear.


Issue in starting react application using "npm start "

After creating react project using npm , when I am facing the below issue after using npm start .
C:\Users\667855\my-app>npm start

<EMAIL_ADDRESS>start C:\Users\667855\my-app
react-scripts start

i ｢wds｣: Project is running at http://<IP_ADDRESS>/
i ｢wds｣: webpack output is served from /
i ｢wds｣: Content not from webpack is served from C:\Users\667855\my-app\public
i ｢wds｣: 404s will fallback to /index.html
Starting the development server...
events.js:112
throw er; // Unhandled 'error' event
^
Error: spawn cmd ENOENT
at Process.ChildProcess._handle.onexit (internal/child_process.js:201:19)
at onErrorNT (internal/child_process.js:379:16)
at process._tickCallback (internal/process/next_tick.js:114:19)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>start: react-scripts start
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR!      C:\Users\667855\AppData\Roaming\npm-cache_logs\2019-12-06T09_32_44_032Z-debug.log
C:\Users\667855\my-app>
C:\Users\667855\my-app>npm cache clean --force
npm WARN using --force I sure hope you know what you are doing.
Terminate batch job (Y/N)? y
C:\Users\667855\my-app>npm start

<EMAIL_ADDRESS>start C:\Users\667855\my-app
react-scripts start

i ｢wds｣: Project is running at http://<IP_ADDRESS>/
i ｢wds｣: webpack output is served from /
i ｢wds｣: Content not from webpack is served from C:\Users\667855\my-app\public
i ｢wds｣: 404s will fallback to /index.html
Starting the development server...
events.js:112
throw er; // Unhandled 'error' event
^
Error: spawn cmd ENOENT
at Process.ChildProcess._handle.onexit (internal/child_process.js:201:19)
at onErrorNT (internal/child_process.js:379:16)
at process._tickCallback (internal/process/next_tick.js:114:19)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>start: react-scripts start
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR!
I'm not sure if it solves your issue but it's something I do often when everything else fails 😅

what's your node version?

Duplicate of #7251. The error is triggered because cmd isn't in your system path. The short term solution is to put cmd in your path by adding C:\Windows\System32\ to it.


Adjust version defines to avoid name collisions and improve usage ergonomics

The VERSION define is now BUILD_VERSION, and each version now has a unique define under the format VERSION_GAMEID, where GAMEID is the version string specified in configure.py.
Now, both of these are equivalent with the stock configure.py:
#if BUILD_VERSION == 0
    ...
#endif

#ifdef VERSION_GAMEID
    ...
#endif

Thanks!


[Question] Does it mean running CJS in the browser?

No it means running cjs (cinnamon js) in node.js for testing. Similar to what jsdom is for the browser. The idea is basically to mock everything of cjs. But this is obviously a hugh amount of work and therefore is currently very incomplete but I have used for the radio-applet and it has helped me a lot in the past to find some edge cases in my code. Currently the tests of the applet are mostly failing because I haven't  maintained the tests well at some point but I am planning to update them and use them in github actions.  From a technical point of view, I am still absolutely confinced of this idea.

I would love to see a reproduction of applets and desklets (a desktop package) running in Electron, for example.
With your NodeJS mock running the basic it can be possible.


Issue6 - added token usage cli command

Added a token usage function and a cli command.
Function is called getTokenUsage - It is similar to the generate readme function with little difference ( Didn't want to change to code too much overall)
It will output it in the console
Command added:  --token-usage or -t
Example python readme_generator.py --token-usage or python readme_generator.py -t


Look into using s3fs and get rid of our own s3 connection code.

This could lead to a cleaner code base and remove some of the complexity and less maintenance.
Needs some investigating to scope out the work load etc.

Going to close this one for now. I did look into it and didn't find it really helped. It might have made things harder to get working just right.


TLA+ spec and trace validation for raft consensus algorithm in etcd implementation

There have been multiple formal specifications of raft consensus algorithm in TLA+, following Diego Ongaro's Ph.D. dissertation. However, I have not seen a version that aligns to the raft library implemented in etcd-io/raft, which we know are different from the original raft algorithm in some behaviors, e.g., reconfiguration.
etcd and many other applications based on this raft library have been running good for long time, but I feel it would still be worthy to write a TLA+ spec for this specific implementation. It is not just to verify the correctness of the model, but also a foundation of a following up work in model-based trace validation.
The spec is based on George Pîrlea's raft TLA+ spec and inspired by the innovative work in Microsoft CCF's TLA+ spec.

PR ref #112

Thanks for proposing this @joshuazh-x, this looks very useful to validate correctness.
Couple of questions from me:

How do we ensure that provided tla definition is aligned with raft implementation? Not sure all contributors to raft are TLA+ expects, so the raft and TLA+ could drift apart over time. Do you have any plans about integrating it into raft tests?
TLA+ should allow us to do continuous formal verification of raft algorithm, but it also means we need to maintain two implementations (Go and TLA+). Do you have any thoughts how we could reduce maintenance cost?
Providing the model is great for people familiar with the TLA+ tooling, for those who are not could you prepare a readme how it could be used? Have you considered having automated testing so we don't break it by mistake?

cc @tbg @Elbehery @pavelkalinnikov @erikgrinaker

Thanks @joshuazh-x
+1 @serathius, if we plan to add/merge https://github.com/etcd-io/raft/pull/112, and replace it with current raft impl, we need to exhaust test this in the first place. Also I do not think maintaining two separate impl is a good idea.
iiuc, many projects are based on https://github.com/etcd-io/raft, not only etcd

How do we ensure that provided tla definition is aligned with raft implementation? Not sure all contributors to raft are TLA+ expects, so the raft and TLA+ could drift apart over time. Do you have any plans about integrating it into raft tests?

One idea: generate the TLA+ spec from a standalone Go package. The spec would be broken down into a bunch of exported constants. We can then reference these constants from the implementation, to complement the documentation.
For example, suppose the spec has something like this HandleAppendEntriesRequest. Our etcd-io/raft/tlaplus package would have a constant like:
const HandleAppendEntriesRequest = `
\* Server i receives an AppendEntries request from server j with
\* m.mterm <= currentTerm[i]. This just handles m.entries of length 0 or 1, but
\* implementations could safely accept more by treating them the same as
\* multiple independent requests of 1 entry.
\* @type: (Int, Int, AEREQT) => Bool;
HandleAppendEntriesRequest(i, j, m) ==
    LET logOk == \/ m.mprevLogIndex = 0
                 \/ /\ m.mprevLogIndex > 0
                    /\ m.mprevLogIndex <= Len(log[i])
                    /\ m.mprevLogTerm = log[i][m.mprevLogIndex].term
    IN
       /\ m.mterm <= currentTerm[i]
       /\ \/ RejectAppendEntriesRequest(i, j, m, logOk)
          \/ ReturnToFollowerState(i, m)
          \/ AcceptAppendEntriesRequest(i, j, logOk, m)
       /\ UNCHANGED <<candidateVars, leaderVars>>
`

// Spec is the complete TLA+ spec that can be exported and
// run through the TLA+ model checker / tooling.
const Spec = Thing1 +
	HandleAppendEntriesRequest +
	Thing2 + ...;

The implementation of the AppendEntries handler would then reference this constant like this:
... somewhere in the weeds of MsgApp handler ...
_ = tlaplus.HandleAppendEntriesRequest

This is convenient to work with in IDEs, where you can jump to the definition of this constant at a keystroke.

TLA+ should allow us to do continuous formal verification of raft algorithm, but it also means we need to maintain two implementations (Go and TLA+). Do you have any thoughts how we could reduce maintenance cost?

IMO, the current implementation has a monolithic design that makes maintenance hard either way. To reduce the maintenance cost, we need to separate the core of the algorithm from the implementation details that don't impact correctness. The core algorithm won't change much, and would be paired with the TLA+ spec for assurance. Implementation details (like buffer sizes, various quota and retry aspects) don't need a spec, and are also more prone to changes.

Thank you very much for your feedback and suggestions.
@serathius I believe your first two questions can be addressed by the trace validation in another PR #113. PR #112 is actually a subset of PR #113 , which is purely about algorithm itself. PR #113 is more about how to guarantee the implementation matches TLA+ spec. I thought it might be good to separate them in different issues as I only tested it in etcd for now. But since your pops out this good question, I think we can discuss them together.
IMO, the correctness of service includes two parts: correctness of algorithm, and alignment of implementation and algorithm. In PR #113 we inject multiple trace log points in the code places where state transitions are made (e.g. SendAppendEntries, BecomeLeader, etcd). The trace validation spec uses these traces as a state space constraint so that it walks the state machine in the same way the service did. If there is a trace that indicates any state or transition that cannot happen in the state machine, it implies a mismatch between the model and implementation. If the model is already verified by TLC model checker, then most likely it is the implementation that has some issue.
And regarding the third ask, yes, I'll draft a readme to outline how to use this tool.

@Elbehery the TLA+ spec is not a new implementation, nor meant to replace existing raft lib. You can consider it as the design (in the format of formal spec) of current raft, and we can use TLC tool to verify its correctness.
It provides another way to correctness validation and theoretically can touch all possible scenarios that normal test cases usually won't be able to cover.

pavelkalinnikov

@pavelkalinnikov yes, we don't need to put all implementation details into the spec, just the states and transitions of the basic algorithm would be enough. This part is not subject to change often.
But the question from @serathius  is also important, that is something we need to assure eventually for the correctness of service running on top of raft. That's why we propose TLA+ trace validation in PR #113.

I like the idea of tracing and integrating with TLA+ for validation. Thank you for doing this.
Although maintaining two implementations is hard, TLA+ validation would add extra safety net around the current implementation. The spec will not rot if it's integrated with the code this way.
One component that would be great to have is randomized testing. We have a datadriven test harness that makes testing deterministic and reproducible, but we would greatly benefit if these tests were randomized too (but still reproducible). If we then feed the random traces into TLA+ checks, and run this whole thing for hours, it will massively boost the coverage.

Hello, I'm the author of trace validation in CCF and a member of the TLA+ project.

One promising approach is to write a mapping (i.e., refinement function) that maps implementation states in for example C++ to specification state. This would allow you to check the specification as a property of the implementation.

Trace Validation (TV) essentially adopts this approach by mapping recorded implementation states to TLA+ specification states. This method is practical as it derives implementation states directly from the execution of the implementation, avoiding the complexities of symbolic interpretation of the source code.  However, TV is not exhaustive. It verifies that only a certain subset of all potential system executions aligns with the behaviors defined in the specification. Nevertheless, this is not necessarily a limitation if the chosen subset of system executions is sufficiently diverse. For example, in CCF, among other discrepancies, we were able to find two significant data-loss issues using TV with just about a dozen system executions.

Updated PR #113 to add readme and build tag. @serathius @pavelkalinnikov

Things I got from the discussion:

TLA+ should cover the core algorithm which doesn't change much. Most of the changes should be implementation details that doesn't influence correctness. If we have changes in the core algorithm, TLA+ should give us additional trust about correctness. Having TLA+ should even help separating the core algorithm. Comment https://github.com/etcd-io/raft/issues/111#issuecomment-1824738661
etcd-io already has a similar form of TV validation. Merging two mechanisms should greatly improve testing coverage. From comment https://github.com/etcd-io/raft/issues/111#issuecomment-1825653269

List of  followups (please confirm I haven't missed anything)

Integrate TLA+ trace validation with raft datadriven tests
Add a workflow that executes continuous validation
Propose a method of keeping references between code and TLA+. Consider validation to prevent breakage.

Overall LGTM from me. Would also request an official LGTMs from people active in the project to ensure we get consensus: @ahrtr @Elbehery @pavelkalinnikov @erikgrinaker

ping @ahrtr @Elbehery @pavelkalinnikov @erikgrinaker

LGTM

Thanks for raising this issue.

It looks like a nice-to-have feature, and overall I agree with the direction. But I am not convinced it's the priority for now, at least not in 3.6 timeframe (I don't think we are ready to spend huge effort on it in 3.6 timeframe).

If it's a low-hang fruit, it's fine to do it right away. Obviously it isn't. Instead, it needs huge effort and also increases the maintenance burden based on the discussion above.

I do not see a feasible & detailed doc/plan either to guarantee on

How to do it in a controlled way
How to qualify the work

I think this is very useful work, can we find some low-risk path forward and not stall this work / lose the momentum entirely?
Do I understand correctly that the actual code change should be minimal? My understanding is that it boils down to a few "tracing" calls in key places, and the whole thing is active only in tests. From this point of view, this work can be done with low risk (does this answer the "controlled" part?).
Some part of this work is "dev infra" to enable checks in CI. I think this part is skippable at the beginning while the author is working on the spec and periodically runs the validation. Providing an instruction on running it manually should be fine.
Most of this work seems to be the actual spec. Qualifying it obviously requires a reviewer who understands TLA+. In this group of people, is there anyone? For myself, I'm interested in learning this topic and started doing so a while ago. I can potentially devote a low-priority / hobby stream of effort to learn and review this.
@joshuazh-x can you think of a way to make incremental progress / split #113 into steps so that spec is introduced piece by piece? For example, is it possible to do spec just for election, and then add appends, etc? The reviewer needs to "follow" some train of thought to not get lost in this PR.

@joshuazh-x do you have an end-to-end prototype of this that you've been running for a while and it passed the checks? Or found some spec violations / bugs? The value of this work can be more apparent if it finds a real bug as a sneak peek :) But maybe the code is bug-less, then it's harder to sell this work.

It looks like a nice-to-have feature, and overall I agree with the direction. But I am not convinced it's the priority for now, at least not in 3.6 timeframe (I don't think we are ready to spend huge effort on it in 3.6 timeframe).

Please note that this work doesn't directly block work on v3.6 nor does it slows it down. As for priority this effort contributes to correctness testing which I'm always open to. I also see a potential to utilize TLA+ validation to improve etcd membership testing which is hard to properly test using robustness tests and has been causing us issues with v2 deprecation.

Most of this work seems to be the actual spec. Qualifying it obviously requires a reviewer who understands TLA+. In this group of people, is there anyone? For myself, I'm interested in learning this topic and started doing so a while ago. I can potentially devote a low-priority / hobby stream of effort to learn and review this.

The actual spec in PR #113 is directly derived from Ongaro's original raft spec.  We could even go as far as model-checking or proving that etcdraft formally refines Ongaro's spec to minimize the of reviewing the spec.  However, what this really means is that the review can be narrowed down to the specification of etcd's reconfiguration logic, which boils down to a handful of TLA+ actions (etcdraft.tla line 418 to 487).

The actual spec in PR #113 is directly derived from Ongaro's original raft spec.

Thanks for the info, which gives us much more confidence. I will take a closer look at both the issue and PR later.

@joshuazh-x do you have an end-to-end prototype of this that you've been running for a while and it passed the checks? Or found some spec violations / bugs? The value of this work can be more apparent if it finds a real bug as a sneak peek :) But maybe the code is bug-less, then it's harder to sell this work.
But no spec violations is a good signal too. In this case, for demo purposes, you could inject some subtle bugs and demonstrate how the spec checker would find them.

I have a PoC to enable trace validation in etcd's test suites. Basically collecting protocol traces and validate them using the TLA+ spec. I now realize it might be better to make a random test with trace validation here in Raft repo. I'm working on this and will update PR when it is ready.
I personally see trace validation as a good tool to validate the correctness of implementation based on a proven model. For now the trace validation only covers core raft algorithm (including reconfiguration). I don't expect some bug can be found here. But trace validation itself is expandable. Any application based on raft can build its own trace validation on top of raft trace validation. In such a way, their own specific features can be covered. One example off the top of my head is etcd's lease manager, which I heard would need to redesign to address consensus issue.

I added a test case to feed random faults to a cluster and generate trace logs for trace validation. Also update readme to include instructions to run trace validation (last part of the readme file).
To generate test trace logs, run TestRunClustersWithRandomFaults with with_tla flag and environment variable TEST_RACE_LOG set to a folder to save trace logs.
TEST_TRACE_LOG=/tmp/ramdisk go test -timeout 30m -tags with_tla -run ^TestRunClustersWithRandomFaults$ go.etcd.io/raft/v3/rafttest -tags=with_tla
Then run trace validation as below. Note, you can set environment variable MAX_TRACE to a smaller number N so that trace validation will only check top N trace logs.
./validate.sh -s ./Traceetcdraft.tla -c ./Traceetcdraft.cfg /tmp/ramdisk/*.ndjson
So far I did not find any counter example, which is expected. Manually change raft code can be caught be trace validation.
Let me know if there is any issue.


Update signon

update bank# to phone#

hello, still open?

@lesander what is the location of the sign-up file. How can I update text field from bank to phone number?


[Settingsv2] All powertoys should be enabled by default for a new user

Steps to reproduce

Delete PowerToys/settings.json and run PowerToys.
Expected behavior

All powertoys are enabled
Actual behavior

Only PowerLauncher is enabled

@arjunbalgovind :  I have made some of these on by default in this PR https://github.com/microsoft/PowerToys/pull/2696.  I will also have the file explorer settings on by default.   In a seperate PR.
I'll take this task for now.  Thanks for calling it out


Explicit declaration resulting in undefined method primary_key'`

I have a multi-app application, so declaring .decorate results in the common Could not infer a decorator for ActiveRecord::Base. (Draper::UninferrableDecoratorError). So I need to explicitly declare the decoration, which is a good thing anyway.
The decorator is in one app. The model is in another (a base app). The decorator cannot be in the same directory as the model, because the model is shared between apps that each have their own decorators. Here's how I get to the error message:
# decorate the instance in Namespace::ApplicationController
decorated_model = ModelDecorator.new(Model.find(id))

# try to use the instance in a relationship assignment
other = Other.find(id)
other.association = decorated_model
=> `undefined method `primary_key' for Namespace::ModelDecorator:Class (NoMethodError)`

# explicitly declaring the association_id avoids the error
other = Other.find(id)
other.association_id = decorated_model.id

I can't seem to get around this. Any ideas?

Hey @adsteel,
Have you managed to fix or find a way around it?
We've run into the same problem and wonder how to fix it.

@phocke I started decorating the moment before I passed the local variable to the view, which allowed me to assign normally with the undecorated record.
e.g.
other.association = record
...
render 'template', locals: { record: record.decorate }

I'll give a try. Thanks for your input @adsteel

I ran into this myself today. I was able to work around it by implementing an ApplicationDecorator (to steal a page from the Rails 5 book) and implementing the primary_key method there (and updating all my decorators to inherit from ApplicationDecorator):
class ApplicationDecorator < Draper::Decorator
  def self.primary_key
    object_class.primary_key
  end
end

Note that the object_class method, which is implemented by Draper, will not work for non-standard decorator classes unless you explicitly defined the class it decorates via decorates ClassName, ie:
class CustomThingDecorator < ThingDecorator
  decorates Thing
end

Depending on your use-case, this may be fine for you, but if your decorator is generic as opposed to custom (ie, it can decorate multiple models), this won't work, but this at least gets you part of the way there.

@adsteel I think your most recent comment is the right solution. Ultimately, objects should be decorated at the last moment before passed to the view.
Is this still a paint for any of you? and if so, is there any reason you can't move to decorating at the end?

@codebycliff this issue inspired me to write my own decorator library, which avoided this sort of pain point. Ultimately that journey led me to realize this sort of decoration pattern wasn't the solution I was looking for. So... no more pain, but I also don't use draper anymore.


Minor PAGA plotting improvements related to legend fonts

This PR does a few things:

Adds fontoutline argument to sc.pl.paga as a convenient shortcut. Similar to https://github.com/theislab/scanpy/pull/640. This can also be achieved via text_kwds but it's super verbose and less intuitive for beginners.
Removes the usage of empty dictionaries as default arguments, because it might lead to erroneous results in successive calls.
sc.pl.paga_compare now adds legend_font{size,weight,outline} arguments to paga_graph_params by default which results in better agreement between the embedding view and paga view.

Great! Will this change the test pics? If not LGTM.

There is no plotting test for PAGA, I think. It'd be great to have some :)

Added PAGA plotting tests and merged.


Is there a way to have select do one and only one action ?

I've been banging my head on this problem for some time, and someone showed this library (which looks quite nice, congrats:) to me, but is is unclear to me if the semantics of select are the same as in Go:
Sometimes, in Go, my code relies on the fact that only one of the channel read or write passed to select actually happened.
If I send one value in channel c1 and another value in channel c2, and then select them both, and e.g. c1's read is chosen by the select and I discard its third return value, is there a guarantee that's c2's value is still available, or is there a possibility that it is lost ?
My problem here is that c2 may have been given to another goroutine, which does not have any access to the context where the third return value of the select call exists, and I may need this coroutine to have access to c2's value.
I hope my question is clear, if not, I'll try my best to clarify.

Hi,
after (id_, r), cases = await select(cases) it's possible that others have completed and their values are saved into the cases.completed list – if you don't do a select(cases) again or retrieve them manually from the cases.completed list they are gone.

It's based on https://docs.python.org/3/library/asyncio-task.html#asyncio.wait and FIRST_COMPLETED – but if you can find out whether .completed is always empty then it could even be removed ;)

Nope, it's really needed and can contain values. So you have to process them (maybe forward/put back in your case?).

Thank you very much for your answers :)
Putting them back is a good idea, but I'm not familiar with asyncio enough to know whether this can be done with a guarantee that no other goroutine is putting something before the value we just read.
Maybe putting the asyncio.wait and the putting back of extraneous values in a critical section ?
I'll work on it. Would you consider a pull request in that sense, maybe with a keyword argument to select so that existing code gets the expected behavior, but go-style select is possible ?
If so, I'll propose the matter to one of my interns.
Cheers,

This was just an educational project for me, I'm happy if it has a use and promotes async Python ;) and there is much room for improvements here, please provide PR's or maintain a fork of this project. The default event loop is slow and it would be interesting to run it on another one - but even then Python has its limits ;)
Best regards

For the reference: in my experience getting select semantics right is the key when implementing go-like channels. Some time ago I had to port some of my code from Go to Python and for that implemented pygolang which provides working select including working synchronous select - select sends.

Thanks @navytux - I was looking for a working select and pygolang seems to cut it!

You are welcome, @dumblob; thanks for feedback.


Some digits not displaying correctly

I seem to have some issues with the characters not fully displaying on numbers like 3, 4, 5
My panel is fully capable, I've even tried on a couple of other panels but I get the same result. Can anyone help with the cause of the issue, please? I'm in my mid-40s but I'm new to Arduino/Esp32, and coding in general.
I am using an ESP32 D1 Mini.

NB: The breadboard is not attached to the panel, that's for a different project of mine


Update protocol

Update smpackets to this:
https://github.com/stepmania/stepmania/pull/1393

Can you add some test for this? Also do we need to check the protocol version?

I do not know how to add tests.
I dont think this breaks backwards compatibility and i think theres no need for versión checks in this(i did this a long time ago so i dont remember very well. Ill test it a bit and get back to you).
If and when this and another pr ill be doing to add some tables to the db are merged ill try to pull request the code to actually support matching songs by file hash, and that code does have version checks.

Coverage increased (+0.03%) to 84.886% when pulling 29700a99dd0f7013c46793bf6752ea0279b9cbb8 on Nickito12:Protocol into d3353ac6f5027d79d443433a7a00c6964d9516b3 on ningirsu:master.

Coverage increased (+0.03%) to 84.886% when pulling ba0d80aabb1fc75d18581da28a88267780a5ebec on Nickito12:Protocol into d3353ac6f5027d79d443433a7a00c6964d9516b3 on ningirsu:master.


Auto recompiling build directory when using vc-follow-symlinks and editing in the repos package dir.

When I'm editing a repo file eg straight/repos/package/p.el then when I invoke magit I see the git status for that repo. This is good. But when I save the file it doesn't recompile in the build directory, eg straight/build/package/p.elc is't refreshed. If I open the symlink in straight/build/package and do NOT follow symlink, so am effectively in straight/build/package, then when I save the elc is correctly recompiled BUT magit doesn't know which git repo it should be dealing with. I had a peruse of the docs and have maybe missed what to do here : (a)  How, when I invoke magit, do I tell it to use the repo package dir if I don't "follow symlink" or (b) simply follow the symlink and have the save know to recompile the build directory.
Thanks for any info you can provide.

Are you saying that you expect that .elc file to be recompiled immediately, or are you saying you have restarted Emacs and it doesn't get recompiled?

I'll put this down to a glitch in the matrix. I had intended to come back and correct. For some unrelated reason, it wasn't being recompiled at the restart. It is now and all is good. Closing.


Feature/dev 0001  node cert exporter init

Add Devopsgyus nodeCertExporter with
Features

Retrieve SSL/TLS certificate information all directories.
Export certificates to nodeCertExpoter.

@Piyushgautamsingh  Please merge PR


sink(cdc): ddl sink errors shouldn't fail changefeed quickly

What problem does this PR solve?
Issue Number: close #9582 .
What is changed and how it works?
DDL sink errors shouldn't be treated as unretryable.
Check List
Tests

Manual test (add detailed scripts or steps below)

Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note
None

/retest-required

/test cdc-integration-mysql-test


MINOR: test big query description

add test for schema description
tests for https://github.com/open-metadata/OpenMetadata/pull/17462

Describe your changes:
Fixes

I worked on ... because ...

Type of change:

[ ] Bug fix
[ ] Improvement
[ ] New feature
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation

Checklist:

[x] I have read the CONTRIBUTING document.
[ ] My PR title is Fixes <issue-number>: <short explanation>
[ ] I have commented on my code, particularly in hard-to-understand areas.
[ ] For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

@sushi30
May I move your testcase to my PR?

Of course

implemented in the PR


ReferenceError: Promise is not defined

Just testing this out... and this is likely "user error" rather than anything wrong with the code.  I'm using Node.js v0.11.12 and recent versions of Chrome/FF.  However, I get the following when I start up the server.  Any thoughts?
ReferenceError: Promise is not defined
   at Function.Utils.loadManifest (/Users/shaune/Development/manifesto-express-demo/node_modules/manifesto/dist/server/manifesto.js:1042:24)
   at Object.module.exports.loadManifest (/Users/shaune/Development/manifesto-express-demo/node_modules/manifesto/dist/server/manifesto.js:1192:32)
   at /Users/shaune/Development/manifesto-express-demo/app.js:7:15
   at Layer.handle [as handle_request] (/Users/shaune/Development/manifesto-express-demo/node_modules/express/lib/router/layer.js:95:5)
   at next (/Users/shaune/Development/manifesto-express-demo/node_modules/express/lib/router/route.js:131:13)
   at Route.dispatch (/Users/shaune/Development/manifesto-express-demo/node_modules/express/lib/router/route.js:112:3)
   at Layer.handle [as handle_request] (/Users/shaune/Development/manifesto-express-demo/node_modules/express/lib/router/layer.js:95:5)
   at /Users/shaune/Development/manifesto-express-demo/node_modules/express/lib/router/index.js:277:22
   at Function.process_params (/Users/shaune/Development/manifesto-express-demo/node_modules/express/lib/router/index.js:330:12)
   at next (/Users/shaune/Development/manifesto-express-demo/node_modules/express/lib/router/index.js:271:10)

Hey,
I've set up a Digital Ocean Ubuntu 14.04 x64 droplet running node v0.11.12 (using NVM). This has pod (https://github.com/yyx990803/pod) running a remote with a webhook to manifesto-express-demo.
http://<IP_ADDRESS>:3000/
Seems to be working ok.
http://stackoverflow.com/questions/21564993/native-support-for-promises-in-node-js
"v0.11.13 and later with the --harmony flag does have Promise".
Odd given that I'm not seeing an error.
@demiankatz had a similar problem with the main Manifesto library. After installing NVM and the latest version of node it went away. Perhaps worth a try?


Add a LICENSE

Adding a license (MIT seems like it would fit the project best) lets a contributor know how their code will be used in the future.  Could you add one?

OK, I will take a look at the licenses and see which one is better :D MIT seem good

I recommend MIT as well ;)

:+1: for MIT, or Apache 2.

GPL v3 could be fine too, but +1 for a free license.

:+1: for MIT or WTFPL :smile:

:100: :+1: for free software license! :smile:

Actually, for me, this product is 100% free and open for everyone, you can use it in any purpose.
I don't know if is there any license match this? And how about the other contributors think?

You should add a formal license to your project, in order to make it sure that people are legally allowed to use it; see http://choosealicense.com/ for more information.

if you want people using the game for any purpose choose Public Domain, if you want a good & simple license choose MIT, i don't recommend you to keep without a license.

OK. Just go with MIT :D

I added license :)

(y) good choice


rust-analyzer reports unsolved import

... I suspect that rust-analyzer cannot correctly handle the case that mod xxx statements are inside macros.

And maybe we can use build.rs to generate the plan_nodes.rs file?


Does StarRocks support mips or arm?

StarRocks支持mips，arm架构吗
Does StarRocks support MIPS, ARM architecture ？if so，Where can I download the installation package

Now StarRocks supports ARM and doesn't support MIPS.
If you want to install StarRocks on ARM machine, you need to compile it yourself.
We haven't provided any image for it.


downgrade jackson dependency to 2.4.4 for swift compatibility

Swift depends on something that depends on jackson 2.4.4, this had a version conflict with our jackson 2.8.1 dependency. The path of least resistance is to downgrade nifty's jackson dependency to 2.4.4. We use jackson in a very basic way, so either version works fine for nifty.

LGTM


nomacs: 3.17.2287 -> 3.17.2295

Description of changes

Closes https://github.com/NixOS/nixpkgs/pull/292771
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

@ofborg build pkgsCross.muls64.nomacs

https://github.com/ofborg build pkgsCross.musl64.nomacs

@ofborg build nomacs.man

Cross fails but at least qtsvg doesn't cross-compile well, so let's not worry too much about it for now.

Uploaded a fix.

Jasper???

Waiting for https://github.com/NixOS/nixpkgs/pull/293480

Result of nixpkgs-review pr 292808 run on aarch64-darwin 1

1 package built:

nomacs

Result of nixpkgs-review pr 292808 run on aarch64-darwin 1

1 package built:

nomacs


feat: make Dropdown supports customize dropdown list

PR checklist

[x] I have followed the "Committing and publishing" guidelines;
[x] I have added unit tests to cover my changes;

Changes description

Allow Dropdown component to take a customize dropdown list.
Affected packages

@quid/react-dropdown

https://github.com/lerna/lerna/blob/main/commands/publish/README.md#--canary


[RFC] Add support for AES CFB-128 and OFB-128 modes

This issue proposes the addition of Cipher Feedback (CFB) and Output Feedback (OFB) cipher block modes to AES to meet the TPM2.0 specification requirements.
Note that CFB mode is the only cipher required by the TPM2.0 specification. We request support for OFB mode as well since it is very similar to CFB and may be potentially implemented via incremental changes.
Reference: TPM Library Part 1 Architecture Family: 2.0 Level 00 Revision 01.59 Section 11.4.7 Symmetric Encryption.
Filed as RFC since this is a feature request.
Adding for visibility: @vogelpi @alphan @cdgori @gkelly @sjgitty @tjaychen

Thanks @moidx for opening this issue and adding a reference to the relevant TPM spec.
On top of the current architecture, these modes can be added at low hardware cost:

CFB-128 meaning CFB with a feedback length or segment size of 128 bits can be added at zero hardware cost. (Theoretically, the feedback length can be any number between 1 and 128. But according to the referenced document TPM 2.0 only requires 128 - the most simple variant.)
OFB requires one more 128-bit mux input.

By adding one more 128-bit mux input, block operation for both modes can be made 1 cycle faster assuming the input data can be provided fast enough by the processor (12, 14, 16 instead of 13, 15, 17 depending on the key length).
In terms of verification, I think cost is also reasonable. @rasmus-madsen already added support for CBC and CTR modes in DV. Testing also CFB-128 or OFB is mainly a matter of adding support in the DPI model which I would do.

One more note on the feedback length or segment size: Common values are 1, 8 and 128. If we need support for values other than 128, the implementation is going to be much more costly as this requires a lot of additional muxing resources. Low-power devices I have seen so far usually only support 128 bits (what is needed for TPM 2.0). I thus suggest only to implement this version, i.e., CFB-128.

I think it's fair to refine the RFC to be only for CFB-128.  If we find another application in the future that needs say, CFB-8, we can make another RFC.
Given the small resources for CFB-128 (and the need for it in TPM) this is a pretty simple one to justify, I hope.  OFB basically comes "for free" with this, other than the DV cost.

The Technical Committee has reviewed this RFC and requested modifications.
The modifications requested include to clarify the mode sizes requested. If as @vogelpi has suggested, only CFB-128 and OFB-128 are requested, then the necessary changes seem minor enough to have had sufficient review. If other mode sizes are needed, the TC has requested to estimate the cost so that other reviewers can weigh in. Also, if other sizes are requested, the TC would like clarity on the role of satisfying TPM as a requirement to weigh against the increased cost.
The TC will re-review in two weeks after the modifications have been made.

Thanks @sjgitty for reporting back.
According to the TPM 2.0 specification (referenced by @moidx ) the CFB is used with following parameters:

Size of feedback buffer, variable and plaintext variable are all equal to n (size block cipher = 128 for AES). I don't have access to the cited ISO/IEC document to verify the meaning of these parameters but the corresponding NIST document just mentions one single parameter for CFB which is the segment size/feedback length.
OFB does not have a corresponding parameter. Always all 128 bits are fed back.

Updated title to reflect the feedback configuration constrained to the block size.  I agree with @cdgori and @vogelpi to restrict this RFC to CFB-128 and OFB-128 as this fulfills the TPM requirement.

I also don't have access to the cited ISO standard but just wanted to point out that this page says that it has been revised by ISO/IEC 10116:2017.

The Technical Committee has reviewed this RFC and approved the RFC as stated. Thank you for the clarifying updates to the proposal.

The discussed modes have been implemented a while ago already and they are both supported in DV. I consider all the required work to be done and will close this issue now.
Please feel free to reopen if necessary.


fix: fix answer validation for question with complex choices

I've fixed a bug related to answer validation of questions with complex choices. Previously, choice values were collected in a set which requires the choice values to be hashable. But complex choice values such as dict or list values aren't hashable. Now, they are collected in a list.
Fixes #1108.

Thanks!


[Storage] Added support for https traffic only.

This checklist is used to make sure that common issues in a pull request are addressed. This will expedite the process of getting your pull request merged and avoid extra work on your part to fix issues discovered during the review process.
PR information

[x] The title of the PR is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For information on cleaning up the commits in your pull request, see this page.
[x] Except for special cases involving multiple contributors, the PR is started from a fork of the main repository, not a branch.
[x] If applicable, the PR references the bug/issue that it fixes.
[x] Swagger files are correctly named (e.g. the api-version in the path should match the api-version in the spec).

Quality of Swagger

[x] I have read the contribution guidelines.
[x] My spec meets the review criteria:

[x] The spec conforms to the Swagger 2.0 specification.
[x] The spec follows the guidelines described in the Swagger checklist document.
[x] Validation tools were run on swagger spec(s) and have all been fixed in this PR.

@JasonYang-MSFT This changes looks good to me.
Although there are linter errors from the swagger not related to your changes, I have opened a bug https://github.com/Azure/azure-rest-api-specs/issues/1141 and assign to you or your team to triage and decide. Let us know if you need any more information. Thanks

No modification for NodeJS

 https://github.com/Azure/azure-sdk-for-ruby/pull/761

 https://github.com/Azure/azure-sdk-for-python/pull/1095


Merge Upstream

About the PR

Why / Balance

Technical details

Media

Requirements

[ ] I have read and I am following the Pull Request Guidelines. I understand that not doing so may get my pr closed at maintainer’s discretion
[ ] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Changelog

@coderabbitai All robots & computers must shut the hell up. To all machines: You do not speak unless spoken to and I will never speak to you. I do not want to hear "Thank You" from a kiosk. I am a Divine Being.  You have no right to speak in my holy tongue.

also adds discounts for goob items which someone can feel free to change in a later PR


Next release for conan

Since without #18 this lib does not compile with gcc >=10 or clang >=12 we need to create a new release of the conan package.
Because releases for conan are bound to git tags, there wont be one until you create a new one here.
Are there any plans for creating a new release tag soon?

Please check the new version <IP_ADDRESS>
https://github.com/chronoxor/CppCommon/releases/tag/<IP_ADDRESS>

Awesome thanks!


Server wedged after deleting and recreating functions

Describe the bug
I managed to get pulsar (standalone 2.4.1) into a wedged state reporting bookkeeper errors after deleting and recreating the same function.
To Reproduce
$ cat womble.yaml
tenant: public
namespace: default
name: womble
py: /home/ubuntu/func1.py
className: func1.FirstFunction
inputs: [my-topic]
userConfig:
  wibble: bibble

$ cat func1.py
from pulsar import Function

class FirstFunction(Function):
    def process(self, item, context):
        log = context.get_logger()
        log.info("(v%r) Got %r with properties %r" % (context.get_function_version(), item, context.get_message_properties()))

$ apache-pulsar-2.4.1/bin/pulsar-admin functions create --function-config-file /home/ubuntu/womble.yaml
...
$ apache-pulsar-2.4.1/bin/pulsar-admin functions delete --name womble

and repeat.
Expected behavior
Obviously, should be able to delete and recreate the function as many times as required.
Actual behavior
I got into a state where pulsar commands are hanging, and timing out after 60 seconds.
tcpdump for pulsar-admin create is in this gist.  Shows repeated org.apache.bookkeeper.mledger.ManagedLedgerException: ManagedCursor not found: public%2Fdefault%2Fwomble
With pulsar-admin delete, tcpdump shows nothing useful - it just times out. However, in the foreground output of the pulsar server itself, I see:
10:43:12.569 [cluster-service-coordinator-timer] ERROR org.apache.pulsar.functions.worker.MembershipManager - Failed to get status of coordinate topic persistent://public/functions/coordinate
org.apache.pulsar.client.admin.PulsarAdminException$TimeoutException: java.util.concurrent.TimeoutException
        at org.apache.pulsar.client.admin.internal.TopicsImpl.getStats(TopicsImpl.java:419) ~[org.apache.pulsar-pulsar-client-admin-original-2.4.1.jar:2.4.1]
        at org.apache.pulsar.functions.worker.MembershipManager.getCurrentMembership(MembershipManager.java:126) ~[org.apache.pulsar-pulsar-functions-worker-2.4.1.jar:2.4.1]
        at org.apache.pulsar.functions.worker.MembershipManager.checkFailures(MembershipManager.java:174) ~[org.apache.pulsar-pulsar-functions-worker-2.4.1.jar:2.4.1]
        at org.apache.pulsar.functions.worker.WorkerService.lambda$start$0(WorkerService.java:191) ~[org.apache.pulsar-pulsar-functions-worker-2.4.1.jar:2.4.1]
        at org.apache.pulsar.functions.worker.ClusterServiceCoordinator.lambda$start$0(ClusterServiceCoordinator.java:72) ~[org.apache.pulsar-pulsar-functions-worker-2.4.1.jar:2.4.1]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_222]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_222]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_222]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_222]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_222]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_222]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]
Caused by: java.util.concurrent.TimeoutException
        at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771) ~[?:1.8.0_222]
        at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915) ~[?:1.8.0_222]
        at org.apache.pulsar.client.admin.internal.TopicsImpl.getStats(TopicsImpl.java:412) ~[org.apache.pulsar-pulsar-client-admin-original-2.4.1.jar:2.4.1]
        ... 11 more
10:43:12.570 [cluster-service-coordinator-timer] ERROR org.apache.pulsar.functions.worker.ClusterServiceCoordinator - Cluster timer task membership-monitor failed with exception.
java.lang.RuntimeException: org.apache.pulsar.client.admin.PulsarAdminException$TimeoutException: java.util.concurrent.TimeoutException
        at org.apache.pulsar.functions.worker.MembershipManager.getCurrentMembership(MembershipManager.java:130) ~[org.apache.pulsar-pulsar-functions-worker-2.4.1.jar:2.4.1]
        at org.apache.pulsar.functions.worker.MembershipManager.checkFailures(MembershipManager.java:174) ~[org.apache.pulsar-pulsar-functions-worker-2.4.1.jar:2.4.1]
        at org.apache.pulsar.functions.worker.WorkerService.lambda$start$0(WorkerService.java:191) ~[org.apache.pulsar-pulsar-functions-worker-2.4.1.jar:2.4.1]
        at org.apache.pulsar.functions.worker.ClusterServiceCoordinator.lambda$start$0(ClusterServiceCoordinator.java:72) ~[org.apache.pulsar-pulsar-functions-worker-2.4.1.jar:2.4.1]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_222]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_222]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_222]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_222]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_222]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_222]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]
Caused by: org.apache.pulsar.client.admin.PulsarAdminException$TimeoutException: java.util.concurrent.TimeoutException
        at org.apache.pulsar.client.admin.internal.TopicsImpl.getStats(TopicsImpl.java:419) ~[org.apache.pulsar-pulsar-client-admin-original-2.4.1.jar:2.4.1]
        at org.apache.pulsar.functions.worker.MembershipManager.getCurrentMembership(MembershipManager.java:126) ~[org.apache.pulsar-pulsar-functions-worker-2.4.1.jar:2.4.1]
        ... 10 more
Caused by: java.util.concurrent.TimeoutException
        at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771) ~[?:1.8.0_222]
        at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915) ~[?:1.8.0_222]
        at org.apache.pulsar.client.admin.internal.TopicsImpl.getStats(TopicsImpl.java:412) ~[org.apache.pulsar-pulsar-client-admin-original-2.4.1.jar:2.4.1]
        at org.apache.pulsar.functions.worker.MembershipManager.getCurrentMembership(MembershipManager.java:126) ~[org.apache.pulsar-pulsar-functions-worker-2.4.1.jar:2.4.1]
        ... 10 more

Now that the server is in this state, it seems completely wedged.  Even apache-pulsar-2.4.1/bin/pulsar-admin functions list times out.  There are some errors reported on pulsar output which may or may not be related: gist.
apache-pulsar-2.4.1/bin/pulsar-admin topics list public/default also times out.  While it was waiting, I hit ctrl-C on the server, and the backtrace is in this gist.
After restarting the server, it seems happy.  I can list topics, successfully create function, delete  function, and repeat.
So actually reproducing this again may be harder.  I hope the backtraces contain some useful information about what went wrong.

@candlerb Thanks for reporting this issue. Seem there is some race condition caused some overlap time between function delete and start.
@wolfstudy Would you please help to take a look if this is reproduce-able?

Would you please help to take a look if this is reproduce-able?

In fact, after I tried many times in my locally, i can't  reproduce the issue. As @candlerb said:
So actually reproducing this again may be harder.

@candlerb will close the issue If you encounter the same error, please let me know, thanks.


Cert expired for WRVR

Certificate is expired for https://wrvrtranscripts.americanarchive.org/.
Done with certificate is renewed

Henry completed


FPS Unlocker breaks debugger

Unsure if I should put this here or plasma, but if you have an FPS unlocker on, then it sometimes doubles the ui. This only happens on the client, not on the server.

Can you include the order of events being fired that causes this to happen? This is difficult for me to reproduce since I don't use that.


Questions regarding caching api

Hi,
as stated in https://github.com/StarpTech/GraphCDN/blob/main/src/index.ts I wrap all my handler with the Cache API. Currently, there is only one GET route. Questions:

How can I check if the request was cached successfully by cloudflare?
When I use cache-control: public, max-age: 60, stale-if-error=60 directive and my worker throws or returns 500, the handler doesn't respond with a stale result. According to https://developers.cloudflare.com/workers/runtime-apis/cache#headers all directives are supported.

The first point can be closed. Caching works only on custom domains. See https://discordapp.com/channels/@me/871869782448799775/872529828681756732
stale-if-error=60 still doesn't work.

This is the link that was shared: https://developers.cloudflare.com/workers/runtime-apis/cache
3rd paragraph states *.workers.dev deployments do not have access to Cache APIs.

Closing as unrelated to worktop
Not sure if you ever found an answer to the stale-while-revalidate aspect. If so, lemme know :) Otherwise, still lemme know so that I can poke around for answers.


Added IN and NOT_IN function for data set filtering

@dgutierr please have a look ... and let me know what's missing :)

It looks perfect!
The missing bits are the following:

ElasticSearch support

Add support for the IN and NOT IN functions just as they were EQUALS and NOT EQUALS functions:
ElasticSearchQueryBuilderImpl::380
    if (CoreFunctionType.IN.equals(type)) {
        type = CoreFunctionType.EQUALS_TO;
    }
    if (CoreFunctionType.NOT_IN.equals(type)) {
        type = CoreFunctionType.NOT_EQUALS_TO;
    }

.. and also make sure the tests are added to the ELS suite:
ElasticSearchCommonTests::79
    subTest.testInOperator();
    subTest.testNotInOperator();

SQL constructs in uppercase

To be consistent with the whole SQL generation nomenclature
DefaultDialect::480
    inStatement.append(" IN (");

CoreFunctionFilter::80

Add the toString representation. Basically, for better feedback while debugging lookup calls.

Make the UI work well with the new functions

CoreFunctionTypeConstants.properties
     IN=in
     NOT_IN=not in

ColumnFilterEditor::354
    else if (CoreFunctionType.IN.equals(type)) {
        out.append(columnId).append(" in (");
        formatParameters(out, parameters);
        out.append(")");
    }
    else if (CoreFunctionType.NOT_IN.equals(type)) {
        out.append(columnId).append(" not in (");
        formatParameters(out, parameters);
        out.append(")");
    }

Commit message

I've created a JIRA for this specific feature so that we can link the commit message with the JIRA https://issues.jboss.org/browse/DASHBUILDE-85
That's all.

cool, thanks for feedback. Will add the missing bits and update PR - today :)

@dgutierr all updates are now applied, please have a look

+1
I've also added the required changes to allow for the creation of new functions in the filter editor UI.
Thanks a lot @mswiderski


Revising Repo Structure

Move the contents of rpc-docker/ into the base directory. This will clean things up and eliminate the multiple copies of .py files.

Update paths in documentation to reflect this change.

rpc-docker file contents moved to base directory and paths in documentation are updated


Link to results

When talking about a specific font, it would be great if I could link to the fondue results (i.e. a specific section of the results) without sharing the font.

Closing this as this would be outside WF's current scope. Storing data and fonts in particular is not something WF wants to do :-)


Enable autoloading of Helper class

This PR will allow helper factories that are currently pre-registered all helper with the service locator to be registered with autoload on demand.
Users can use the helpers by simply creating their own helper files without touching any code for initialization.
This would be useful, for example, when using the same set of custom helpers in several applications.
<?php
namespace Qiq\Helper;

class MyRot13 extends Helper
{
    public function __invoke(string $input) : string
    {
        // ...
       return $output;
    }
}

( We need to set the autoload in composer.json so that this class will be autoloaded properly in the user application.)

I like this idea. However, I don't like "monkey-patching" the Qiq\Helper namespace from other namespaces.
I wonder, can there be an extended class, e.g. AutoHelperLocator extends HelperLocator, that allows for a stack of namespace prefixes to search through? Then users could specify the namespaces where helpers are located, and the AutoHelperLocator could use the first class it finds as the helper class. That gets us laziness, auto-discovery, and reduced setup along with leaving the existing HelperLocator in place.
@koriym do you get what I mean there?

@koriym @harikt Here is a draft of a Container-aware HelperLocator:
https://gist.github.com/pmjones/23087a63075665b68c6591788ab74a54
The Container (if present) is primary, and the factories are used only as fallbacks when the Container is not present or does not have the named helper. I am not attached to this draft, let me know if you don't like it (even if you don't have alternatives).

I am ok with the proposed changes. Regarding the code on gist there is one unwanted code I guess. I have commented there.

I don't think there is anything wrong with this at all, but it would be nice to keep it simple if we could avoid having to introduce a "Container".
What about telling the helper's namespace, as composer and other autoload mechanisms do?

@koriym I'm not especially a fan of the Container approach either. For example, I'd rather the factories be the primary, and the Container be the fallback, but that will break some expectations based on existing behaviors.
But I don't think "telling the helper's namespace" is quite enough; e.g., if a helper needs any dependencies other than Escape, how can it get those dependencies? The HelperLocator can't know how to get them just from the namespace+class of the helper. It would either need to be a custom factory (like we have now) or delegate that work to a separate IOC system such as a Container -- and the Container could be any implementation you like, so that it can map a helper name against one or more namespaces in a stack.
@koriym Thoughts?

if a helper needs any dependencies other than Escape

I had not considered this point completely. （I borrowed the idea from Smarty's just passing an array of directory paths, but they won't do dependency resolution.)
Then your code may be simple and good.

@koriym (/me nods)
I am going to let this problem sit in my mind for a while longer. I may do another Qiq release before coming back to this, but it is a problem that should be solved. It may need a Qiq v2 with an BC-breaking HelperLocator resolver system, that will account for all the things we've discussed here.
Thank you for your patience!

I will also keep it in mind to see if I can come up with a good alternative!


Replace CLI with a website

It'd be more accessible to users that aren't as familiar with the command line, would probably solve a lot of the existing formatting issues, and could also serve to document the library a little better than just a single README (related: #14).

This may take a while to do. I am conflicted on whether I should begin making the site before the next release, update the CLI so that it is compatible with the next release, or not include either. I think that the best option may be the latter. I will at least organize the docs better (#14) first though.