Add parameter-wise RGE

The difficulty I felt is the pattern used with all together perturb and parameter-wise perturb is quite different. If we want to maximize the speed, the all together perturb one cannot be compatible with parameter-wise perturb.
Another problem is for restoring the model. We should not use rng as argument passing around but explicitly given seed then construct the rng.
The third modification is we now enforce each perturbation is generated vis one particular seed instead of using one rng to generate the all perturbations. If not this approach, we cannot guarantee the parameter-wise approach is the same the full parameters one.
I moved the some functionscezo_fl/shared into RGE class. (Maybe we should create another model update class?)
 After this, @ZidongLiu you can consider re-direct the function used in the server/client code.

Training seems off.
Use this command python cezo_fl_main.py --dataset=sst2 --eval-iterations=25 --large-model=opt-125m --model-dtype=float32 --seed=365 --iterations=2000 --train-batch-size=32 --test-batch-size=64 --no-iid --dirichlet-alpha=1 --num-clients=3 --num-sample-clients=2 --local-update-steps=1 --num-pert=5 --lr=0.000005 --momentum=0 --grad-estimate-method=rge-forward --mu=0.001
param-wise training with PR:
full parameter training with PR :

For the records, the reason that eval accuracy is not increased because the server model update is mismatched with the client one. After switching to self.random_gradient_estimator.update_model_given_seed_and_grad the problem is resolved.

full parameter memory

parameter-wise-memory

N = sizeof(params)
The implementation does not solve peak memory issue which is still 4 * N.
Best memory should be 2 * N. Which is model itself and last pulled state.
Current extra memory usage comes from trying to utilize optimizer.
1 * N from grad
1 * N from foreach operation of SGD. See below.

Will think about remove optimizer, and generate&optimize model on per parameter basis


Data management layer for MSPileup data-service

Fixes #11424
Status
In development
Description
New MSPileupData.py module provides data management layer for MSPileup data service. In particular, it implements the following set of APIs to deal with MSPileup data:

def create(self, cdict) creates MSPileup data from given campaign.json dictionary, and stores it to persistent storage
def update(self, data) updates MSPileup data in persistent storage with given data dictionary
def store(self, data) stores MSPileup data to persistent storage, e.g. someone can provide pileup data to store
def get(self, pileupId) fetches MSPileup data from persistent storage for a given pileupID
def delete(self, pileupId) deletes MSPileup data in persistent storage for given pileupID
def validate(self, pdict) to validate input data dictionary

Is it backward compatible (if not, which system it affects?)
YES
Related PRs
External dependencies / deployment changes

@amaltaro , @todor-ivanov I put in this PR initial implementation for data management layer. This is work in progress as I need your initial feedback on the following questions:

which database the data should be stored to, so far I put MongoDB as backend database
how data should be created, in #11424 there is an initial data-structure but it remains unclear how data should be created. I already looked at campaign.json and it remains unclear how it should be used. Please provide concrete example. For instance, we may create a dictionary out of proposed data structure, but I need input who will fill out the necessary values. Or, the data will be given from upstream caller.

Once, we'll clarify these I can proceed to adjusting the codebase. Meanwhile, I provided an initial unit test for the data management class.

@vkuznet I provided further context and information in the GH issue itself. Please have a look.

Valentin, I fully second what Todor said here. Ideally, this data structure object (module) should not depend on MongoDB itself. Please follow the same structure as done for other microservices (WMCore/MicroService/MSPileup/DataStructs/yourmodule).
Actions like get/store/update/etc do belong to the main logic of MSPileup though.
Last but not least, please have a look at the original description of this issue, it has been updated after further discussion with the P&R. Nothing has been changed or removed, only new stuff added.

Valentin, I fully second what Todor said here. Ideally, this data structure object (module) should not depend on MongoDB itself. Please follow the same structure as done for other microservices (WMCore/MicroService/MSPileup/DataStructs/yourmodule).
Actions like get/store/update/etc do belong to the main logic of MSPileup though. Last but not least, please have a look at the original description of this issue, it has been updated after further discussion with the P&R. Nothing has been changed or removed, only new stuff added.

Alan, it is totally fine to put Data structure into MSPileup/DataStrucs as you proposed, and I agree to do that. And, I think we (you and I) are also agree that apis like get/store/update belong to data layer which draft is here. But I disagree with Todor's suggestion to put data layer logic (like MongoDB) into service since in my experience it is not right design architecture. The service should be responsible for HTTP logic and nothing else, it should call data layer to handle database actions via its APIs. And, database layer should handle database connection and interaction. If we need to scale then service will either setup data layer pool or ask data layer to use the pool of connection. Again, pool belongs to data lay

I'm fine to put data structure into MSPileup/DataStructs, and I agree with Alan that get/update/create/delete APIs belong here.

thanks @vkuznet and @amaltaro. Let me use the current issue to make a more general comment.
Well, I do not want to say it now, since we are fully booked with other issues to work on. But we do tend to go in a well defined path with all services using MongoDB as a background database for keeping service states between runs.
We basically define a strict document schema for every service and we can do bidirectional validation for inbound and outbound  database communication.
This does lead us in a situation with few pros and cons:

We do have a simple and reliable model for using the Database
We do have a very good database integrity
We do provide a base for code and methods unification
We unfortunately, somewhat, restrain ourselves from using complex return result aggregations and utilize in full steam all the benefits provided by the database of choice.

While for the last of those bullets, we can think of optimizations in near or long future (even though for the purposes we use the database it is not mandatory), for the third one we  can take actions immediately. We may think of generalizing some (if not all) of those document validation methods based on the document schema we  provide (as defined in the micro service data structure modules - e.g MSRuleCleanerRSE, MSOutpuTemplate, etc...) and put those methods directly in the WMCore.Database.MongoDBwrapper module. This way, we can achieve even more flexibility for future changes of data structure schema for any micro service, which would be transformed into changes only in the relevantDocSchema`, while any database peculiarities and service specific methods should be left, as @vkuznet suggests, in a separate database management layer for the particular micro service (as here).

test this please

@amaltaro , @todor-ivanov please review a new edition of this PR. It includes full implementation of APIs based on our previous discussion. In particular:

def create(self, pdict)
def update(self, doc)
def get(self, spec)
def delete(self, spec)

Please note, that I opted out to use spec (JSON query) in get and delete APIs. Said that, initially I'll only implement spec={'pileupName': <value>} queries in HTTP layer which will pass this spec to data layer API. This will allow to maintain flexible queries via spec and at the same time constrain in HTTP layer the queries from end-users to one we discussed initially. When, we'll be ready to support more queries this layer will already be ready for that.
I also added time  serialization functions to support RFC339 for presentation layer. Again, this serialization will be used in HTTP layer of APIs to present time in human readable form, while at data layer everything is processed in seconds since epoch in GMT time zone.
I provided unit tests and ensure that code scores 10.00 in pylint.

test this please

@amaltaro , I addressed all issues you requested and this PR is ready for review. Please note, I see that unit tests are failing but those seems to be unstable tests and not realted to this PR. The score of the code is decreased due to some of the features you requested, e.g. put schema/validation back to class instead of making them stand-along functions, etc. But I tried to best of my ability to fix all outstanding pylint/pep issues.

Alan, thanks for review, I implemented some of the changes, but will proceed with further development next week when time permits.

test this please

@amaltaro , as far as I can tell all changes are in place and even pyling-py3k are not perfect I doubt nothing I can further improve here. The unit test are unstable and I always get different set of failures not related to this PR. Please review and let me know if any other changes will be required.

Alan, thanks for feedback, we're getting closer to a final version. To simplify your review task here are the changes I addressed based on your recent feedback:

timers code changes can be found in this https://github.com/dmwm/WMCore/pull/11438/commits/edf90c0d94523c87cb1ae1655e9e91c5762e2f6f
unit tests changes for timers code are here https://github.com/dmwm/WMCore/pull/11438/commits/03c69b5d041ea2e5870749f6ae28d5aeab1c37aa
and, changes to MSPileupObj are here https://github.com/dmwm/WMCore/pull/11438/commits/3f6d9dd1d8cf2bba6f405cfe0eb718ceb431de65

Now, I'll squash them for the final set.

Alan, I checked pylint/pep/unit tests (which are unstable) and up to my knowledge everything is ready. Please review again, and I posted individual commits for your convenience which addressed your recent review.

Alan, regarding your comment https://github.com/dmwm/WMCore/pull/11438#discussion_r1086757097, the unit test I provided clearly shows that both actual error is reported in a log and appropriate exception, e.g.
2023-01-25 11:04:02,808:ERROR:MSPileupObj: pileupName value pileupName does not match dataset pattern
2023-01-25 11:04:02,808:WARNING:MSPileupObj_t: expected exception MSPileup input is invalid
2023-01-25 11:04:02,808:ERROR:MSPileupObj: replicationGrouping value bla is neither of ['DATASET', 'ALL']
2023-01-25 11:04:02,808:WARNING:MSPileupObj_t: expected exception MSPileup input is invalid
.2023-01-25 11:04:02,809:ERROR:MSPileupObj: pileupName value pileupName does not match dataset pattern
2023-01-25 11:04:02,809:WARNING:MSPileupObj_t: expected exception MSPileup input is invalid

As you can see the ERROR message comes from validate function, while WARNING comes from unit test which catch the exception and show it. If you want actual exception to carry the error then the validate method should not return True or False but rather through exception. Please tell me which mode of operation do you want to have:

validate will through the exception (new behavior)
validate will return true or false but put actual error in logger error (current behavior)
Once I'll hear what do you prefer I can adjust code accordingly.

Please tell me which mode of operation do you want to have:

validate will through the exception (new behavior)
validate will return true or false but put actual error in logger error (current behavior)
Once I'll hear what do you prefer I can adjust code accordingly.

Hmm, I think we will need the exception to be raised (with the relevant error message), otherwise we cannot propagate what the actual error was back to the user (when interacting through a REST endpoint). In terms of the daemon service, boolean would be good enough because we already log it. Does it make sense to you?

ok, I modified return of validate method to be boolean, string tuple where first value is a status and second is associated message. And, now, the unit test report the following:
2023-01-25 13:02:23,120:ERROR:MSPileupObj: pileupName value pileupName does not match dataset pattern
2023-01-25 13:02:23,120:WARNING:MSPileupObj_t: expected exception MSPileup input is invalid, pileupName value pileupName does not match dataset pattern

Please let me know if it accepted from your side, see https://github.com/dmwm/WMCore/pull/11438/commits/90d922ae118a8063bf062c7acb644d79ff0eef08

@vkuznet your changes look good to me. I would just make a minor change, initialize msg (msg = "") at the very beginning of that method and remove the "" from the very last return statement

ok, I made this change and squashed commits. Feel free to have a final look.


--autobreak flag corrupts the shellcode execution

--autobreak flag provide the injection of "int 3" instruction into the shellcode, what could corrupt the shellcode execution.
Possible solution:
set the breakpoint before the jump to shellcode execution.

Thanks for the report @ALXIMIKS . Yes, this happens when the shellcode is using relative offsets as the util inserts a new byte. I have a fix in mind or feel free to submit a pull request.

I have fixed the bugs.
Also code refactoring has been done.
Not too familiar with git, need a little time for doing the pull requests.

Fixed 0.0.4


Preapproval various bugs

Description
It is impossible to create a Preapproval, there are at least two bugs:
1- The back_url parameter does not recognize the .website TLD
2- Even if i am including all the requested data, there is a "Bad request" error
Example code (PHP)
MercadoPago\SDK::setClientId(...);
MercadoPago\SDK::setClientSecret(...);

$preapproval_data = new MercadoPago\Preapproval();
$preapproval_data->payer_email = $this->email;
//Setting for example "https://mysite.website" does not work (1)
$preapproval_data->back_url = "http://google.com";
$preapproval_data->reason = substr( $this->subscription_name, 0, 127 );
$preapproval_data->external_reference =  "VIP-" . $this->subscription_id;
//Setting the required data (2)
$preapproval_data->auto_recurring = array(
          "frecuency" => 1,
          "frequency_type" => "months",
          "transaction_amount" => 60,
          "currency_id" => "ARS",
          "start_date" => "2018-07-29T14:58:11.778-03:00",
          "end_date" => "2018-08-29T14:58:11.778-03:00"
);
$preapproval_data->save();

Expected behavior
Get the init_point URL to let the user follow the standard flow of the API
Current behavior
Case 1, using any .website TLD URL, the API responds with:
[29-Jul-2018 16:22:34 UTC] {"message":"Invalid value for back_url, must be a valid URL","error":"bad_request","status":400,"cause":[]}
Case 2, i am correctly setting the required data, the API responds with:
[29-Jul-2018 16:35:55 UTC] {"message":"The parameters frequency, frequency_type, transaction_amount and currency_id are required in the recurring payments scheme","error":"bad_request","status":400,"cause":[]}
Additional information
PHP 7.2.0

@AldrinSalazar back url param doesn't support TLD by now, we have a regex matching validation which prevent this.
About he point 2, You have a typo error, you should use "frequency" instead of "frecuency"


feat: add lxc/incus

lxc/incus: Powerful system container and virtual machine manager
$ aqua g -i lxc/incus

How to confirm if this package works well
Reviewers aren't necessarily familiar with this package, so please describe how to confirm if this package works well.
Please confirm if this package works well yourself as much as possible.
Command and output
$ incus --help
Description:
  Command line client for Incus

  All of Incus's features can be driven through the various commands below.
  For help with any of those, simply call them with --help.

Usage:
  incus [command]

Available Commands:
  cluster     Manage cluster members
  config      Manage instance and server configuration options
  console     Attach to instance consoles
  copy        Copy instances within or in between servers
  create      Create instances from images
  delete      Delete instances
  exec        Execute commands in instances
  export      Export instance backups
  file        Manage files in instances
  help        Help about any command
  image       Manage images
  import      Import instance backups
  info        Show instance or server information
  launch      Create and start instances from images
  list        List instances
  move        Move instances within or in between servers
  network     Manage and attach instances to networks
  pause       Pause instances
  profile     Manage profiles
  project     Manage projects
  publish     Publish instances as images
  rebuild     Rebuild instances
  remote      Manage the list of remote servers
  rename      Rename instances
  restart     Restart instances
  snapshot    Manage instance snapshots
  start       Start instances
  stop        Stop instances
  storage     Manage storage pools and volumes
  version     Show local and remote versions

Flags:
      --all            Show less common commands
      --debug          Show all debug messages
      --force-local    Force using the local unix socket
  -h, --help           Print help
      --project        Override the source project
  -q, --quiet          Don't show progress information
      --sub-commands   Use with help or --help to view sub-commands
  -v, --verbose        Show all information messages
      --version        Print version number

Thank you always!

v4.99.0 is out 🎉
https://github.com/aquaproj/aqua-registry/releases/tag/v4.99.0


4x speedup with this simple trick!

If the episode is never restarted, it is not necessary to manually unroll the LSTM to reset the LSTM states. However, one call to the underlying implementation is sufficient. In a small example I was able to achieve a speedup of 3-4 (RTX 3080 Ti).

Hello,
thanks for the PR =)
will try to review tomorrow, should speedup things for sure, I will just check that the results are the same.

Looking good to me but I let araffin do the proper check. Meanwhile please check the steps in PR template (at least the changelog and contributors one :) )

Looking good to me but I let araffin do the proper check. Meanwhile please check the steps in PR template (at least the changelog and contributors one :) )

no need here to fill full PR template, as it is a special case: PR for an opened PR ;)

I tested it using
            lstm_output1, lstm_states1 = lstm(features_sequence, lstm_states)
            lstm_output1 = th.flatten(lstm_output1.transpose(0, 1), start_dim=0, end_dim=1)

            lstm_output2 = []
            # Iterate over the sequence
            for features, episode_start in zip_strict(features_sequence, episode_starts):
                hidden, lstm_states = lstm(
                    features.unsqueeze(dim=0),
                    (
                        (1.0 - episode_start).view(1, n_envs, 1) * lstm_states[0],
                        (1.0 - episode_start).view(1, n_envs, 1) * lstm_states[1],
                    ),
                )
                lstm_output2 += [hidden]
            # Sequence to batch
            lstm_output2 = th.flatten(th.cat(lstm_output2).transpose(0, 1), start_dim=0, end_dim=1)

            assert th.allclose(lstm_output1, lstm_output2)

and the test passes, but the code somehow changes the random generator state... (I get slightly different results with your PR, but it runs much faster).

I tested it using
lstm_output1, lstm_states1 = lstm(features_sequence, lstm_states)
lstm_output1 = th.flatten(lstm_output1.transpose(0, 1), start_dim=0, end_dim=1)

lstm_output2 = []
# Iterate over the sequence
for features, episode_start in zip_strict(features_sequence, episode_starts):
    hidden, lstm_states = lstm(
        features.unsqueeze(dim=0),
        (
            (1.0 - episode_start).view(1, n_envs, 1) * lstm_states[0],
            (1.0 - episode_start).view(1, n_envs, 1) * lstm_states[1],
        ),
    )
    lstm_output2 += [hidden]
# Sequence to batch
lstm_output2 = th.flatten(th.cat(lstm_output2).transpose(0, 1), start_dim=0, end_dim=1)

assert th.allclose(lstm_output1, lstm_output2)

and the test passes, but the code somehow changes the random generator state... (I get slightly different results with your PR, but it runs much faster).
EDIT: the results seem to change also with the number of threads used... not an used with that PR then

Also see https://pytorch.org/docs/stable/generated/torch.nn.LSTM.html:

thanks but my tests were cpu only... and things are reproducible, just not the same results depending on code used and number of threads

This makes intuitive sense to me. I'd be also interested in seeing the performance comparison.

After further investigation, the difference in results is probably due to numerical imprecision (the test actually fails after a while but still passes with assert th.allclose(lstm_output1, lstm_output2, rtol=1e-05, atol=1e-07) instead of assert th.allclose(lstm_output1, lstm_output2, rtol=1e-05, atol=1e-08))


yaml Marshal/Unmarshal adds regexp anchors

If you use the go library github.com/prometheus/alertmanager/config and then read in the configuration using LoadFile and then write it out again, the regular expressions are anchored. This means you can not faithfully write back the original file contents.
Example:
route:
    receiver: 'sre:bla:foo'
    routes:
        - receiver: 'sre:bla:foo-mail'
          match_re:
            job: 'sre:bla:bar-.*'

becomes via a LoadFile and Marshal:
route:
  receiver: sre:bla:foo
  routes:
  - receiver: sre:bla:foo-mail
    match_re:
      job: ^(?:sre:bla:bar-.*)$

The version used is: b63b5600741b07795e50cedf8818a853bacc5f76.
Now when alertmanager reads this ouput again; it will again add these anchors.
Is this WAI? Or can I prepare a PR to not add these anchors when reading/writing out these configs.

yes, config.go has
// UnmarshalYAML implements the yaml.Unmarshaler interface.
func (re *Regexp) UnmarshalYAML(unmarshal func(interface{}) error) error {
        var s string
        if err := unmarshal(&s); err != nil {
                return err
        }
        regex, err := regexp.Compile("^(?:" + s + ")$")
        if err != nil {
                return err
        }
        re.Regexp = regex
        return nil
}

// MarshalYAML implements the yaml.Marshaler interface.
func (re Regexp) MarshalYAML() (interface{}, error) {
        if re.Regexp != nil {
                return re.String(), nil
        }
        return nil, nil
}

Where return re.String() needs to be make slighty smarter I think. Maybe as lame as:
return re.String()[4:len(re.String())-2]

If you use the go library github.com/prometheus/alertmanager/config

In general this is not intended to be used as a library. It should have probably better been placed into an internal folder.
This does not mean we do not accept any contributions, just that we can not give any stability guarantees.
Before talking about technical solutions, would you mind elaborating on your use case a bit more?

Before talking about technical solutions, would you mind elaborating on your use case a bit more?

We're running a system that takes other people's alertmanager.yaml's, syntax checks them (hence the use of this library) and output a larger, merged alertmanager.yaml.
I can treat the yaml as a blackbox or implement a minimal structure to parse it, but it seems better to use the official code, so any format upgrades are (somewhat) automatic.
This all works pretty well, except for the match_re output.

Side-note: some fields are declared as Secret so the unmarshaled -> marshaled data isn't exactly the same as the raw YAML. But I guess you're mostly concerned with the routes configuration that don't contain secret data (yet?).

Indeed. This is all considered public and limited in what can be configured.
On Tue, 12 Feb 2019, 17:22 Simon Pasquier<EMAIL_ADDRESS>wrote:

Side-note: some fields are declared as Secret so the unmarshaled ->
marshaled data isn't exactly the same as the raw YAML. But I guess you're
mostly concerned with the routes configuration that don't contain secret
data (yet?).
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/prometheus/alertmanager/issues/1753#issuecomment-462851575,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAVkW7ukOQ2hcSTDrGr_lsfil0oM8clfks5vMvhvgaJpZM4a18dK
.

@mxinden and @stuartnelson3 have the final word but the request looks reasonable to me.
Regarding the implementation, we can use the same technique as in github.com/prometheus/prometheus/pkg/relabel

Ok, that makes sense. I'll prepare a PR; we can discuss merging that (nor not).
As a fallback I can fork the code and just change it for my use-case.

 service_key: <secret> is biting here as well, but that obviously begs the question if you should put those things in version control at all


Option to use Postmark's templates API

Instead of only using Laravel's mail builder schema, this package should have an option to use Postmark's templates API, which is one of the main features of Postmark.
https://postmarkapp.com/developer/api/templates-api

Hey @MatanYadaev, I think that sounds like a good idea. How would you imagine that works within a Laravel application?

Hi Craig, thanks! 😃
/**
 * Get the mail representation of the notification.
 *
 * @param  mixed  $notifiable
 * @return \Illuminate\Notifications\Messages\MailMessage
 */
public function toMail($notifiable)
{
    $templateId = '123456789';

    return (new MailMessage)->template($templateId, [
        'invoice' => $this->invoice
    ]);
}

Hey @MatanYadaev, I think it could be a good feature. I've personally never used Postmark's templates, but I'm totally cool with this package supporting them. I think it would have to be an extension of Laravel's MailMessage class, but it already has a template method, so we should come up with a different name I'm thinking. According to the documentation, the items that this would need to support (that aren't already supported) are TemplateId (or TemplateAlias) and TemplateModel.
I'd gladly accept a PR for this and your implementation looks good to me, we'd just need a different naming structure as that method name already exists and then any amended handling of the sending through the transport. If you're unable to work on this, I'll give it a try, but I can't promise any timelines at this moment. Let me know what you think.

@craigpaul I actually don't really know how Laravel's Mail module works, but I really want this feature to be included in this package, so I will give it a try, I will begin to read and research about the Mail module.
If you have any tips, short intro, or how do you think this feature should be implemented - it would be nice and helpful.

@MatanYadaev That's fair, I have some time this weekend to take a look into how I would like to see this available. If its not too complicated I will throw something together and put up a PR 👍

I actually began to write some code, I want to discuss with you about how you would implement it:
(let's ignore the naming convention for now, we'll cover it later)
I created new class called MailTemplateMessage that inherits Laravel's MailMessage. My class has some methods to define TemplateId, TemplateAlias, and TemplateModel.
How should I identify the incoming message in the PostmarkTransport@send method as a MailTemplateMessage message? I see the message that passes in the first argument is an instance of Swift_Mime_SimpleMessage and not MailMessage.
Any suggestions?

Hey @MatanYadaev, sorry for the delay. I just tried getting a very rough implementation of this going. You can check it out here. This approach utilizes a new MailMessage class that ends up rendering a view that returns a stringified JSON object. That string gets set as the body property on the Swift_Mime_SimpleMessage class that is passed to the send method. Hopefully this gives you some direction on how to get that information to the transport.
Note: I'm not saying it has to be done this way, this is just the way I found to get it in there 👍
Let me know if that helps

@craigpaul Looks like a good implementation. It took me some time to understand this Swift stuff and how it works behind the scenes.
Well, let's just finish this feature, do you want me to help you with this?

This is now available in v2.5.0, thanks for the feature request 😄


Fix for openCV in 4.0+

Changes:

CMakeList.txt:

SET(CMAKE_CXX_STANDARD 11)

main.cpp

some api changes in openCV 4.0+
change face_cascade_name.

Thanks for doing this, I'm sure it'll help people!
Do you happen to know whether these changes are compatible with OpenCV 3.0? If so I'm comfortable merging this.
Otherwise I just checked and 4.0 is pretty new, Debian stable doesn't have it yet for example. I can leave this as an open PR though for people who want to compile on 4.0 to find though. Alternatively if you could make it compile on both that would be nice but not not necessary.

Thanks for doing this, I'm sure it'll help people!
Do you happen to know whether these changes are compatible with OpenCV 3.0? If so I'm comfortable merging this.
Otherwise I just checked and 4.0 is pretty new, Debian stable doesn't have it yet for example. I can leave this as an open PR though for people who want to compile on 4.0 to find though. Alternatively if you could make it compile on both that would be nice but not not necessary.

Thanks for your reply and I attempted again by recovering old codes and adding some macros for the 4.x version api 😄
And also removed the "aa" and "aaa" windows as matrix dichotomy is no longer applied.
I think this would work with work with 3 and 4 versions, as I did not have 3.x version on my mac lol.


Resolving 'Invalid Credential key correctness proof' Error in Distributed Credential Service Setup

I have set up a credential issuing service on two separate machines with the intention to use the same cryptographic seed/DID/verkey on both. Initially, both machines could issue credentials without any issues.
However, after setting up and leaving the second machine in a different location, I encountered an 'Invalid Credential key correctness proof' error on the first machine. This error does not occur if I use a new seed/DID/verkey.
Could there be a file system element or a component I've overlooked that's necessary for operating the same seed/DID/verkey across multiple machines? Any guidance on making this work for more than one machine would be appreciated.

I wiped the key from where it was being saved, re-ran the service and it's working.

It depends a bit on the database you're using. But if you're using Postgres, it should be possible to use multiple machines. However, we're in the progress of removing Indy SDK and moving towards Aries Askar, which has some new feature which we're not fully leveraging yet (such as locking of files to prevent multiple processes overwriting each others' process) and transactions (to bundle writes to the database).

How would you use it on postgres on multiple machines?
Export and share the my issuer, i.e. "my-test-issuer" database?
Is there an easier way?


cargo risczero install fails

I first run cargo install cargo-risczero, then I tried cargo install cargo-risczero --locked --force but the error is the same.
Logs:
Finding latest release... (https://api.github.com/repos/risc0/rust/releases/latest)...
Could not download pre-built toolchain: Could not download release info

Caused by:
    HTTP status client error (403 Forbidden) for url (https://api.github.com/repos/risc0/rust/releases/latest)
Error: Download of pre-built toolchain failed

Caused by:
    0: Could not download release info
    1: HTTP status client error (403 Forbidden) for url (https://api.github.com/repos/risc0/rust/releases/latest)

OS: MacOS Venture 13.5.1
Rustc version: stable 1.71.1

Found a workaround
https://github.com/risc0/risc0/blob/1d6656d55601abdaf2f95d18ddce48dbe224a789/risc0/cargo-risczero/src/commands/install.rs#L92
It would be nice if the suggestion message is printed to the user when the given error occurs


Add schema to doc url.

Description
Old url was a relative path and would therefore route locally (e.g. old url would route to "http://localhost:3000/nuxtseo.com/sitemap/guides/customising-ui"). I've simply added an "https://" schema so you're properly routed to the docs.
I've also removed a spare space before the link.
Linked Issues
Quick fix, no issue created (sorry!)

Good catch, thanks!


Cant Connect to postgres?

I'm trying to connect to my database hosted by https://aiven.io/ but getting some configuration error do you know what it could be
web_1       | > Using "webpack" config function defined in next.config.js.
web_1       | error: no pg_hba.conf entry for host "<IP_ADDRESS>", user "avnadmin", database "defaultdb", SSL off
web_1       |     at Connection.parseE (/usr/src/app/node_modules/pg/lib/connection.js:554:1

Its connecting fine using psql cli. So why isn't postgraphql connecting? Don't you set ssl to true? That seems to be the problem.

@pyros2097 The error states "SSL off" but it looks like your connection URL wants SSL; it seems that the postgres query string library doesn't support sslmode=require instead requiring ssl=1; my connection string looks like:
postgresql://username:password@example.com/dbname?sslmode=require&ssl=1

Yes it worked thanks @benjie :) :+1:

Reopening as a task.

Starting to getting this error, with the latest version of postgraphql weird. I'm using heroku and I can't change DATABASE_URL as it is audo-generated and it is locked. I wonder if there is an easier way to do this via more command line approach. Does postgraphql provide any flag like ssl=true?

Failing that a bash script that does the same.

I ended up doing similar things, by doing something like "${DATABASE_URL-default}?ssl=1" in the parameter


Not working for certain NTP server

I have tried setting up NTP server on a Windows PC (WinPC) and a Raspberry Pi (RPi) and noticed that the framework works well syncing time with WinPC but fail to do so when setting RPi as the server.
I simply run "Clock.sync(from: (IP of WinPC/RPi))" during initialization of the app and use "Clock.now" to get server time. When setting the IP of WinPC, it works, while setting IP of RPi doesn't.
The way I set up NTP server on RPi is by adding following lines to /etc/ntp.conf
server <IP_ADDRESS>
fudge <IP_ADDRESS> stratum 10
restrict <IP_ADDRESS> mask <IP_ADDRESS> nomodify
I have tested on a Macbook Pro with command "ntpdate -u (IP of RPi)" to sync time from RPi and it works well, which proves that the NTP server on RPi is running.
I would appreciate if you could provide any help/comment on this issue. Thank you.

Can you try debugging a bit more where the problem is? Is your server using ntp v3?

Is this still an issue? Please reopen if so


Update Composer dependencies (2023-12-19-00-16)

Visual regression test passed!
View the visual regression test report


Update title of nav item for EE webcasts

https://www.pivotaltracker.com/story/show/173015264

@Alixers505 is this ready for review?

@zarathustra323 - I'm still waiting on feedback from the story submitter. I noticed there is another menu item in the dropdown which is labeled "EE Resources", but it goes to a different page.


Large memory consumption in constant memory mode

Current behavior
Hello @jmcnamara
First of all thanks to take time to maintain and improve this project !
I just want to let you known than i spotted an issues with current constant memory mode implementation. The memory consumption is not constant when writing worksheet to zip archive; this is mainly due to the fact than worksheet is fully loaded in memory just before writing it to archive.
We could simply read worksheet file in smaller slice to bypass this issues.
I'll be glad to open a pull request to implement a fix !
Expected behavior
When using constant memory mode feature flag, memory consumption should be linear and not vary based on worksheet sizes.
Sample code to reproduce
#!/usr/bin/env rust-script

//! cargo
//! [dependencies]
//! rust_xlsxwriter = { version = "0.79.4", features = ["constant_memory"] }
//!
fn main() -> Result<(), rust_xlsxwriter::XlsxError> {
    let mut workbook = rust_xlsxwriter::Workbook::new();
    let worksheet = workbook.add_worksheet_with_constant_memory();
    for row in 0..600_000 {
        for col in 0..40 {
            // memory will remain constant when adding data to worksheet
            worksheet.write(row, col, "content")?;
        }
    }

    // memory footprint will increase when calling save()
    workbook.save("test.xlsx")?;

    Ok(())
}

### Environment

```text
- `rust_xlsxwriter` version: 0.79.4
- Cargo.toml dependency line for `rust_xlsxwriter`: { version = "0.79.4", default-features = false, features = ["constant_memory", "zlib"]}
- rustc version: 1.80.0

Fix proposal
I fork your repos and come with a small fix, a simple update of packager.rs write_worksheet_file() function
#[cfg(feature = "constant_memory")]
{
    // Section 2. On disk cell data.
    // We also need to flush the last remaining row.
    worksheet.flush_last_row();

    worksheet.file_writer.rewind().unwrap();
    let mut reader = BufReader::new(worksheet.file_writer.get_ref());
    loop {
        let buffer = reader.fill_buf().unwrap();
        let length = buffer.len();
        if length == 0 {
            break;
        }
        self.zip.write_all(&buffer).unwrap();
        reader.consume(length);
    }
}

I've been using valgrind to monitor heap consumption (with upper sample code, 600 000 rows and 40 columns) :
With version 0.79.4, maximum heap footprint: 1.268 GB
With patch version 0.79.4, maximum heap footprint is 339.4 KB

Thanks. That is a good catch.
Please go ahead and submit a PR.

I've been using valgrind to monitor heap consumption (with upper sample code, 600 000 rows and 40 columns) :

For reference could you show a sample command line (or otherwise) that you used to do this?

To generate heap profile: valgrind --massif-out-file ./valgrind --tool=massif ${binaryPath}

Thank you. That is very useful.


Business key null

This reworks https://github.com/camunda/camunda-modeler/pull/3631 + adds test coverage.

Closes https://github.com/camunda/camunda-modeler/issues/3623

This Pull Request targets develop branch, but contains fix commits.
Consider targeting master instead.

I think we shouldn't care about this right now as we will release it in a feature release.


Remove whatwg-fetch dependency that is incompatible with jscore

Fixes #9599 #12128
The polyfill from whatwg-fetch assumes that either self or this are defined, which is not true with the combination of JavaScriptCore (no self) and strict mode (this is undefined outside of classes)
This reverts https://github.com/facebook/react-native/commit/eb7442ba09dc13cc32c6eab8f42eb6ad6fad1651 and updates to the latest version of the polyfill from https://github.com/github/fetch/

Hi there! This issue is being closed because it has been inactive for a while.
But don't worry, it will live on with Canny! Check out its new home: https://react-native.canny.io/feature-requests/p/remove-whatwg-fetch-dependency-that-is-incompatible-with-jscore


Does GetChildKeys from ConfigurationProvider work correctly?

Hi,
I think there is a bug in method GetChildKeys of https://github.com/aspnet/Configuration/blob/dev/src/Config/ConfigurationProvider.cs.
According to the documentation, it supposes to return the list of keys for this provider.
Let's imagine that I have a provider that contains keys with names which contain semicolon in their name, e.g. part1:part2:part3
then if I do something like
var result = new List<string>();
foreach (var provider in Configuration.Providers)
{
	foreach (var key in provider.GetChildKeys(Enumerable.Empty<string>(), null))
	{
		result.Add(key);
	}
}
return result;

It will return me just part1, cutting of the rest of the key name because of the delimiter.
So it doesn't return a full list of keys for this provider, but just names of parent "nodes" of the data used to populate a provider.
If this is expected, how do I iterate through a full key names then?

Hm, I've achieved what is needed recursively iterating through keys and their parts. That's pretty ugly, but looks like it works though. Is it the way to go, or I have missed something?
private HashSet<string> _keys;
private HashSet<string> GetKeys(IConfigurationProvider provider, string rootKey)
{
	foreach (var key in provider.GetChildKeys(Enumerable.Empty<string>(), rootKey))
	{
		string surrogateKey = key;
		if (rootKey != null)
		{
			surrogateKey = rootKey + ":" + key;
			GetKeys(provider, surrogateKey);
		}
		else
		{
			GetKeys(provider, key);
		}

		if (_keys.RemoveWhere(k => surrogateKey.StartsWith(k)) > 0)
		{
			_keys.Add(surrogateKey);
		}
		else
		{
			if (!_keys.Any(k => k.StartsWith(surrogateKey)))
			{
				_keys.Add(surrogateKey);
			}
		}
	}

	return _keys;
}

You shouldn't use the KeyDelimiter in keys


ReflectionProbe captures incomplete image of scene surroundings when async shader compilation is enabled

Godot version
v3.5.beta1.official
System information
Windows 10, GLES3, Hybrid Laptop - AMD iGPU(Ryzen 7 4800HS) + GTX 1660 Ti Max-Q
Issue description
Async Shader Compilation disabled:

Async Shader Compilation Enabled:

I suspect the ReflectionProbe node captures an image of the scene before asynchronous shader compilation finishes, causing the resulting capture to be incomplete and look broken.
If you set the Update Mode for the ReflectionProbe to always, the image looks correct, but this is obviously not a proper solution.
UPDATE: I can only reproduce this when Godot is running on my integrated AMD GPU, when using the dedicated NVIDIA GPU the reflectionprobe displays correctly, so it could potentially be an AMD or iGPU specific issue.
Steps to reproduce
Try using ReflectionProbe in any scene in a project with Asynchronous shader compilation enabled, preferably in a scene with at least a couple different Shaders or SpatialMaterial configurations.
Use a Mesh with a material that has roughness 0 and metallic 1 for easier testing.
Make sure to run Godot using an Integrated GPU if you have access to one.
Minimal reproduction project
MRP.zip

cc @RandomShaper

MRP added.
MRP.zip

I can't reproduce the issue with the MRP. Neither out of the box nor by tweaking some materials to fall back to hidden during compilation.
Furthermote, currently reflection probes are already protected against this issue, which is only relevant to materials configured like that. For materials that use the ubershader during compilation (the default), reflections can be rendered just fine from the very beginning.
Since the MRP doesn't correspond to the screenhots, @EzraT, could you tell, or show, exactly how the issue shows itself for you in it?

@RandomShaper
Synchronous:

Asynchronous:

Are you testing on a dedicated GPU?
I found out just now, that this only seems to happen when Godot is running on my integrated GPU.

Might be that the iGPU doesn't support GL_KHR_parallel_shader_compile?
I can't find it on https://opengl.gpuinfo.org (at least with 4800 or Ryzen, maybe it the GPU component has another identifier?), might be worth using glCapsViewer to upload a report if there's none for this iGPU: https://opengl.gpuinfo.org/download.php

That was just a hunch, but good to see that it's supported :+1:
I tested the MRP and I can also reproduce the issue on Linux with both:

AMD VEGAM (DRM 3.42.0, 5.15.14-desktop-1.mga9, LLVM 13.0.0)

and:

Mesa Intel(R) HD Graphics 630 (KBL GT2)

With both Asynchronous and Asynchronous + Cache (not sure the latter actually works properly for me, it still takes a while to process the materials which appear one at a time).

Interesting, I actually wasn't expecting it to also happen on an Intel iGPU, since AMD's OpenGL drivers have never been that great, so it is indeed an iGPU related problem.

Yes, I'm testing with a discrete GPU. Now, I think I now what's going on. I have been trying to fix a bug on Android that maybe is precisely this one: the ubershader won't be rendered at all even if it's ready because it may be too complex or, more likely, reaching invalid states on the non taken branches. Therefore, only when the conditioned shaders are ready you start to get stuff rendered.
I'll try to reproduce on the ntegrated Intel GPU because that would help a lot.
P. S.:

With both Asynchronous and Asynchronous + Cache (not sure the latter actually works properly for me, it still takes a while to process the materials which appear one at a time).
Currently only the ubershaders are cached. That was not always the case, but it turned out that caching the conditioned shaders was causing a huge cache while not providing performance benefits. Therefore, the conditioned shaders will take the same time to get ready regardless caching is enabled or not. So to speak, caching can only save "loading" time.

(Probably) fixed by #57674.
Please test in the upcoming 3.5 beta 2 to confirm.

Tested in 3.5 beta 2, and no longer able to reproduce, thanks!


command vscord.enable/disable not found

Hello 👋🏻
First of all, thank you for this extension 🙏🏻
I have a problem though. When I start VSCode, the extension does not start automatically, I have to start it manually, and when I do, I get an error :
vscord.enable not found
When I uninstall and reinstall the extension, it works again until I restart VSCode..
Do you have any idea how to fix this problem?
Do you need more information?
Thanks for any help you can give me.
edit: I'm running VSCode on Windows 11

This is related to .git not loading correctly most likely
https://github.com/leonardssh/vscord/commit/98db028af5da06ef29f0926883bfae1d4e1b5643
I suspect you can confirm by going help -> toggle dev tools and looking for the vscord error.

Yes, that looks like it is the same error:
Activating extension 'LeonardSSH.vscord' failed: Extension 'vscode.git' is not known or not activated. $onExtensionActivationError @ workbench.desktop.main.js:2153 workbench.desktop.main.js:616  WARN UNRESPONSIVE extension
I am currently using what appears to be the latest version of the extension from the VSCode store.

any update on this?

No progress on this, it happened randomly and is hardly re-producable


[backport-13.1] UG-543 Verifies Orchestration

Adds verification Orchestration service post upgrade in existing
test format. Also includes Orchestration in pre-upgrade check.
(cherry picked from commit 69269b40cd47e48d89d5938f3be5ab9989aaaf47)

recheck_cit_upgrade
recheck_cit_majorupgrade

mirror fails

recheck_cit_upgrade
recheck_cit_majorupgrade

This looks like a legitimate failure:
TASK: [rpc_pre_upgrade | Gather orchestration service list] *******************
failed: [localhost -> rammp-13-b7b7_utility_container-63658709] => {"changed": true, "cmd": "source ~/openrc; openstack orchestration service list --insecure", "delta": "0:00:00.827288", "end": "2017-06-02 15:29:16.532870", "rc": 2, "start": "2017-06-02 15:29:15.705582", "warnings": []}
stdout: openstack: 'orchestration' is not an openstack command. See 'openstack --help'.
Did you mean one of these?
  configuration show

FATAL: all hosts have already failed -- aborting

hm.. in testing I found it true for liberty, but worked after upgrade to mitaka.

recheck_cit_ceph - TEMPEST FAIL scenario defcore (304 tests)


[FEATURE] Seed the randomness for deterministic model based test generation

Is your feature request related to a problem? Please describe.
Currently every time we regenerate tests we get new ones.
Describe the solution you'd like
A way to regenerate the same set of tests, typically in a case where we've changed some data structure on the Rust side but otherwise want to be testing the same scenario.

To do the testing in the proper way, we need to have it both ways:

allow Z3 to choose it's random seed, and then provide it to the user in the output (for exploratory testing)
accept the seed in the user input, and pass it to Z3 (for test reproducibility)

I am reading the z3 docs, and the developers say that z3 should be deterministic. The default seed is 0. Are we sure that we get non-determinism from z3? @andrey-kuprianov,  you can produce the smt log with --debug, run it with z3 several times and see, whether the results are different.
I will add an option to set the seed. But it may happen that non-determinism is also on our side.

well, the outcome is that smt logging seems to be broken. first, it's log0.smt that's being produced. Second, after some initial valid SMT commands, it contains some printout, which is not an SMT-LIB:
(error "line 44536 column 1: invalid command, '(' expected")
(error "line 44538 column 7: unexpected character")
(error "line 44538 column 19: unexpected character")
(error "line 44550 column 8: unexpected character")

(assert (= $C$1994 (>= $C$1776 $C$1993)))
; } IntCmpRule returns $C$1994 [1995 arena cells])
;; assert $C$1994
(assert $C$1994)
(check-sat)
;; sat = true
$C$0 = FALSE
$C$1 = TRUE
$C$2 = {FALSE, TRUE}
$C$3 = Nat

That's weird. It used to work pretty well. Are you feeding this back to z3?

log0.smt is fine. There are multiple solver instances in the parallel version.

Oh wait. The last four lines are the output of the sat model. Just delete all these things.

I have added #320 on that issue. But if you remove the model output at the end of the log, it should work.

That command seems to produce stable results:

The code in this branch sets all random seeds to 0 by default, or to the value passed as smt.randomSeed in the tuning properties, with --tuning=tuning.properties.
When I run Apalache in Intellij Idea on test/tla/Rand.tla, the produced counterexample is always the same. When I run Apalache in the command line, two different results are produced. The file log0.smt only differs in the values of the produced model. It might be something about pointers having different values, not really sure about what is going on.
@andrey-kuprianov can you try this branch and see, whether you also get different results on test/tla/Rand.tla?

Related discussion on stackoverflow: https://stackoverflow.com/questions/47839976/z3-producing-different-models-when-run-multiple-times

The observation that it worked with Intellij Idea led me to try different versions on java.

Java(TM) SE Runtime Environment (build 1.8.0_172-b11)

z3 behaves non-deterministcally

Java(TM) SE Runtime Environment (build 9.0.4+11)

z3 behaves deterministically (on Rand.tla)
We need JDK8 to compile scala, but the compiled code should work in later JREs too.  So I guess the current solution will be to run MBT in JRE9 or later.

It should work unless you are using Java 8


Error in todos_with_provider (in chapter 3)

Currently in todo_view.dart file, "context.select((TodosController m) => m.todos);" is used for listening to changes for todos list.
But this list is declared as final, and createTodo function adds todo to this list, effectively the Todo List object doesn't change and hence the component is not rebuilt.
Simple fix:
Make todo list non-final, and replace createTodo function with following:
Future<void> createTodo(Todo todo) async {
    _todos = [..._todos, todo];
    notifyListeners();
  }

Reference: https://github.com/rrousselGit/provider/issues/560
cc: @albertodev01 @chimon2000

The actual issue was using read instead of watch.  The former does not update the UI when changes occur @Amitbhave


openfl shader “input null”  error on html5 target

First， I found the function "set_glVertexSource"  set value to  _glFragmentSource   but no set value to _glVertexSource. I think it's a bug otherwise howto keep GLSL "varying" is same between glFragment and glVertex。 Second，I write a custom shader using new glVertexSource and glFragmentSource，browser arise a “set property input of null” error,but I set bitmapdata to a "uniform stamper" variable. the browser arise  the same error even if I use the below glVertexSource and glFragmentSource.
var customShader = new Shader();
shader.glVertexSource =
“void main(void)
{
}";
shader.glFragmentSource =
“void main(void)
{
}";
mybitmap.filters=[ new shaderFilter(customShader) ];
Although has a flaw, but It's great that openfl 4.x support GLSL shader more widely!

The base openfl.display.Shader class should add a call that sets the glVertexSource if you do not define it. It comes from a macro, so it's a little bit hidden from plain sight.
The shaders (right now) are reliant upon matching the same core naming scheme for core uniforms and attributes. Most shaders will have one sampler, called "uImage0", though I posted a sampler that illustrated how you could add another sampler and set its input.
Without seeing your code, is it possible you define more than one sampler? Try removing the second sampler (if that's the case) and use only uImage0. The source for openfl.display.Shader should help illustrate what the standard shader source looks like

Dear jgranick，
I rewrite Fragment Source according your advice and it's OK!  Thank your very much！

Awesome! Glad to hear 😁


Add completion modal component

currently not hooked up to anything, but you can do a dispatchShowCompletionModal to trigger the completion text for whichever activity!

oops forgot to add you as a reviewer


Be able to display a custom cursor appearance using a mesh, work in stereo

Have that mesh act as the cursor and hide the pointer in VR mode.

Ahhhh sorry forgot doing this.
Will do this shortly.

@ngokevin fixed!


release: v0.8.0-SNAPSHOT

This pull request releases Chameleon v0.8.0-SNAPSHOT with the following changes:
Refactoring:

improve README.md file
platforms: rename spigot platform to bukkit
*: improve overall code quality and fix styling

Additions:

*: add javadoc
events: add UserChatEvent#setMessage

Fixes:

platform: better gamemode conversion
platform: fix listeners not working (bug introduced and fixed)
configuration: fix map ordering
annotations: fix velocity generation

Build:

add checkstyle plugin
use version catalogs
speed up builds

Dependencies:

update platforms to 1.19
update adventure-platform-* to 4.1.1 (#95, #94, #93)

Contains breaking changes

I think it would be a good idea to start using the main branch for development as it prevents us from having to make release pull requests like this.


Fetching Ads

Please implement a way to fetch ads from a username with twscrape.
Here is an example of ad from FortniteGame on twitter: https://twitter.com/FortniteGame/status/1696926119700709551
Also note that this ad does not come up when using api.user_tweets()
Thanks in advance @vladkens !

Related to this: https://github.com/vladkens/twscrape/issues/72


feat(lnd): lnd crash recovery (show dialog)

Description:
If lnd crashes show a dialog advising what happened. Require user interaction to acknowledge the crash event. Clean up and return the user to the launchpad after closing the dialog.
Motivation and Context:
Fix #1824
How Has This Been Tested?
Start local wallet and try killing lnd at various different points in the cycle, such as:

on the wallet unlocker screen
on the sync screen
when the wallet is fully open

Screenshots:

Types of changes:
Enhancement
Checklist:

[x] My code follows the code style of this project.
[x] I have reviewed and updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes where needed.
[ ] All new and existing tests passed.
[x] My commits have been squashed into a concise set of changes.

Coverage decreased (-0.07%) to 21.017% when pulling 84fb171cea08325de62278f444390926f4236f3b on mrfelton:feat/lnd-crash-dialog into e91a6ee2d92a7122b57acf9804e6e812092328dd on LN-Zap:next.


result<V, E> type

This PR adds Rust-like result<V, E> type, which can be used instead of optional<V>.
This type is either V (value) or E (error).
Also this PR adds GSL library (not used in the PR, but for future use).
Pattern matching and monads can be used with this type:
using R = iroha::result<int, std::string>;
R increment_but_less_5(int a) {
  if (a < 5) {
    return Ok(a + 1);
  } else {
    return Error(":("s);
  }
}

TEST(Result, monadic) {
  R a = 1;
  ASSERT_TRUE(a);

  R b = a | increment_but_less_5 | increment_but_less_5;

  ASSERT_TRUE(a);
  ASSERT_TRUE(b);
  ASSERT_EQ(*b, *a + 2);

  R c = b | increment_but_less_5 | increment_but_less_5 | increment_but_less_5;
  ASSERT_FALSE(c);
  ASSERT_EQ(c.error(), ":("s);
}

TEST(Result, pattern_matching) {
  R a = 1;
  ASSERT_TRUE(a);

  // in one order
  a.match([](Ok_t<int>& v)            { SUCCEED(); },
          [](Error_t<std::string>& e) { FAIL();    });

  // in reversed order
  a.match([](Error_t<std::string>& e) { FAIL();    },
          [](Ok_t<int>& v)            { SUCCEED(); });
}

result<V, E> is useful instead of optional in sequential tasks, such as deserialization of fields in JSON/protobuf.

Codecov Report

Merging #577 into develop will increase coverage by 0.12%.
The diff coverage is 98.83%.

@@            Coverage Diff             @@
##           develop    #577      +/-   ##
==========================================
+ Coverage    80.17%   80.3%   +0.12%
==========================================
  Files          313     314       +1
  Lines        12302   12342      +40
==========================================
+ Hits          9863    9911      +48
+ Misses        2439    2431       -8

Impacted Files
Coverage Δ

test/module/libs/result_test.cpp
100% <100%> (ø)

libs/common/operators.hpp
100% <100%> (ø)

libs/common/result.hpp
100% <100%> (ø)

libs/common/visitor.hpp
100% <100%> (ø)

libs/common/types.hpp
86.58% <85.71%> (-1.06%)
:arrow_down:

irohad/ordering/impl/ordering_service_impl.cpp
98.21% <0%> (-0.04%)
:arrow_down:

...ule/irohad/ordering/ordering_gate_service_test.cpp
100% <0%> (ø)
:arrow_up:

irohad/main/application.cpp
0% <0%> (ø)
:arrow_up:

irohad/main/impl/ordering_init.cpp
0% <0%> (ø)
:arrow_up:

irohad/ordering/impl/ordering_gate_impl.cpp
100% <0%> (ø)
:arrow_up:

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db579d3...2738c69. Read the comment docs.

Codecov Report

Merging #577 into develop will increase coverage by 0.14%.
The diff coverage is 98.92%.

@@             Coverage Diff             @@
##           develop     #577      +/-   ##
===========================================
+ Coverage    80.17%   80.31%   +0.14%
===========================================
  Files          313      314       +1
  Lines        12302    12349      +47
===========================================
+ Hits          9863     9918      +55
+ Misses        2439     2431       -8

Impacted Files
Coverage Δ

libs/common/result.hpp
100% <100%> (ø)

libs/common/operators.hpp
100% <100%> (ø)

libs/common/visitor.hpp
100% <100%> (ø)

test/module/libs/result_test.cpp
100% <100%> (ø)

libs/common/types.hpp
86.58% <85.71%> (-1.06%)
:arrow_down:

irohad/ordering/impl/ordering_service_impl.cpp
98.21% <0%> (-0.04%)
:arrow_down:

irohad/main/impl/ordering_init.hpp
0% <0%> (ø)
:arrow_up:

test/module/irohad/ordering/ordering_mocks.hpp
100% <0%> (ø)
:arrow_up:

irohad/ordering/impl/ordering_gate_impl.cpp
100% <0%> (ø)
:arrow_up:

irohad/main/impl/ordering_init.cpp
0% <0%> (ø)
:arrow_up:

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db579d3...4508b1f. Read the comment docs.


Tiled/repeating textures

It would be nice to set a texture to tile or repeat on a mesh instead of stretch.

Sounds like a great idea

I think that can already be done using SamplerDescriptors? https://github.com/bevyengine/bevy/blob/8106f770de2ae3c841651499e0637123d7ae6688/crates/bevy_render/src/texture/sampler_descriptor.rs

I'm not sure how to specify a SamplerDescriptor for a texture. if I am reading this code correctly, it seems like the sampler is being generated automatically with a From impl. it does seem like a SamplerDescriptor would be the right API for this

How about adding SamplerDescriptor to Texture as a member and grabbing it from there? The code that constructs a SamplerDescriptor from a Texture doesn't even use that texture.
https://docs.rs/bevy_render/0.2.1/src/bevy_render/texture/sampler_descriptor.rs.html#37-52

Yup adding a SamplerDescriptor to Texture has been on my "mental to-do list" for awhile. I think that's the right approach.

In #747 I've moved the SamplerDescriptor and adjusted the GLTF loader.
Still, when loading a texture manually (not from a GLTF file), like it's done in the texture example, we'd still need to specify the sampler somehow. Of course, we could just keep it this way, initialized with default, and the user could still mutate it after loading.

@W4RH4WK sounds reasonable.
Atelier Assets (which we will eventually migrate to) allows you to specify per-asset configuration in a ".meta" file. I think we will eventually put the SamplerDescriptor config there for formats like png.

#747 largely resolves this problem, but I think we should probably hold off on closing this until we can do it for arbitrary texture files (aka using the ".meta" config in atelier assets)

I've tried using this feature, and it doesn't seem to be working the way I expected as of 0.4.0.
It does get as far as recreating the Sampler: I can modify a Texture's SamplerDescriptor, and the generated AssetEvent::Modified event is being picked up by texture_resource_system, which re-creates the sampler from the descriptor.
However, that new sampler has a new unique SamplerId, which is not passed on to any already existing BindGroups or RenderResourceBindings, and no new bindings are created, so rendering will still use the old sampler.
As an example of this, my project (at commit 85229caa). It's supposed to render a 2x2 grid with a texture as the tiles, and the texture is just a plain color with a border. (If it is small, you can scroll up to zoom in and see it better.) I set the sampler's address mode in the configure_textures system, once it is created. However, sampling is still effectively clamped.
A workaround that works for now is to defer spawning the sprite until the texture is loaded and the address modes are set (e.g. moving lines 39-52 to immediately after line 75, and also adding the resources it requires to the system).

I've tried using this feature, and it doesn't seem to be working the way I expected as of 0.4.0.
It does get as far as recreating the Sampler: I can modify a Texture's SamplerDescriptor, and the generated AssetEvent::Modified event is being picked up by texture_resource_system, which re-creates the sampler from the descriptor.
However, that new sampler has a new unique SamplerId, which is not passed on to any already existing BindGroups or RenderResourceBindings, and no new bindings are created, so rendering will still use the old sampler.
As an example of this, my project (at commit 85229caa). It's supposed to render a 2x2 grid with a texture as the tiles, and the texture is just a plain color with a border. (If it is small, you can scroll up to zoom in and see it better.) I set the sampler's address mode in the configure_textures system, once it is created. However, sampling is still effectively clamped.
A workaround that works for now is to defer spawning the sprite until the texture is loaded and the address modes are set (e.g. moving lines 39-52 to immediately after line 75, and also adding the resources it requires to the system).

#747 largely resolves this problem, but I think we should probably hold off on closing this until we can do it for arbitrary texture files (aka using the ".meta" config in atelier assets)

Seems to me like #747 only works for gltf::image::Source::View textures, which is the GLB ones, and not for externally loaded ones.
And we have another issue, with GltfLoader loading all its textures as SRGB. This can be seen here crates\bevy_render\src\texture\image_texture_conversion.rs. It could be fixed in the same way, but again, this only works for gltf::image::Source::View textures.
Regular textures get passed to the AssetServer as an AssetPath dependency, with zero configation, which will then happily load them using its defaults. I haven't dug any deeper.

It would be nice to be able to offset a repeating texture dynamically for parallax backgrounds as well

Hey, is there a way to tile textures now?

Hey, is there a way to tile textures now?

I think we're still waiting on the distill integration for asset parameters.

#[derive(Default)]
struct BackgroundImage(Handle<Image>);

fn configure_background_image(background: Res<BackgroundImage>, mut images: ResMut<Assets<Image>>) {
    // Doing this in response to AssetEvents seem to be broken, instead we try to set the sampler every frame
    let mut image = images.get_mut(background.0.clone());
    if let Some(image) = image {
        image.sampler_descriptor.address_mode_u = AddressMode::Repeat;
        image.sampler_descriptor.address_mode_v = AddressMode::Repeat;
    }
}

const MAP_SIZE: f32 = 100.;

fn setup(
    mut commands: Commands,
    asset_server: Res<AssetServer>,
    mut materials: ResMut<Assets<ColorMaterial>>,
    mut meshes: ResMut<Assets<Mesh>>,
    mut background: ResMut<BackgroundImage>,
) {
    background.0 = asset_server.load("background.png");
    let mut mesh = Mesh::from(shape::Quad::default());
    if let Some(VertexAttributeValues::Float32x2(uvs)) = mesh.attribute_mut(Mesh::ATTRIBUTE_UV_0) {
        for uv in uvs {
            uv[0] *= MAP_SIZE;
            uv[1] *= MAP_SIZE;
        }
    }

    commands.spawn_bundle(ColorMesh2dBundle {
        transform: Transform::from_scale(Vec3::splat(MAP_SIZE)),
        material: materials.add(background.0.clone().into()),
        mesh: meshes.add(mesh.into()).into(),
        ..Default::default()
    });

I ended up doing this in a project. Relevant discord thread: https://discord.com/channels/691052431525675048/742884593551802431/932954492394164255
Would be nice to know if there is a less hacky solution that works with 0.6

This doesn't work. I've spent 1 hour trying this and other solutions on Discord, none of them work on 0.6.

run into this one today, here's what worked for me:
pub fn set_texture_tiled(
  mut texture_events: EventReader<AssetEvent<Image>>,
  mut textures: ResMut<Assets<Image>>,
) {
  for event in texture_events.iter() {
      match event {
          AssetEvent::Created { handle } => {
              if let Some(mut texture) = textures.get_mut(handle) {
                texture.sampler_descriptor.address_mode_u = bevy::render::render_resource::AddressMode::Repeat;
                texture.sampler_descriptor.address_mode_v = bevy::render::render_resource::AddressMode::Repeat;
                texture.sampler_descriptor.address_mode_w = bevy::render::render_resource::AddressMode::Repeat;
              }
          }
          _ => (),
      }
  }
}

One caveat though is this system has to run before you use the Handle<Image> in a material (not sure why). For example, the  system below doesn't work if I use terrain_mat.material but it works if I construct a new material (in a different
pub fn startup_system(
  asset_server: Res<AssetServer>,
  mut terrain_mat: ResMut<TempTerrainMaterial>,
  mut materials: ResMut<Assets<StandardMaterial>>,
) {
  terrain_mat.tex = asset_server.load("textures/test.png");
  terrain_mat.normal = asset_server.load("textures/test_n.png");

  terrain_mat.material =  materials.add(StandardMaterial {
    base_color_texture: Some(terrain_mat.tex.clone()),
    normal_map_texture: Some(terrain_mat.normal.clone()),
    perceptual_roughness: 0.89,
    ..default()
  });
}

pub fn spawn_mesh(
  mut commands: Commands,
  mut meshes: ResMut<Assets<Mesh>>,
  mut materials: ResMut<Assets<StandardMaterial>>,
  terrain_mat: Res<TempTerrainMaterial>,
) {
      commands.spawn_bundle(PbrBundle {
        mesh: meshes.add(some_mesh),
        //material: terrain_mat.material.clone(), // doesn't work
        material: materials.add(StandardMaterial { // works
          base_color_texture: Some(terrain_mat.tex.clone()),
          normal_map_texture: Some(terrain_mat.normal.clone()),
          perceptual_roughness: 0.89,
          ..default()
        }),
        ..default()
      });
}

@sanisoclem I have also ran into the issue where "pre-loaded"materials doesnt pick up the address modes if its texture! So strange. I also had to resort to re-creating the material at entity spawning time.

@ManevilleF, do any of your PRs completely resolve this? If so, please link this issue there :)

@ManevilleF, do any of your PRs completely resolve this? If so, please link this issue there :)

My MR on slicing and tiling only works on 2d texture rects, so sprites or UI

For anyone comfortable with all of their textures using the same wgpu::SamplerDescriptor. I found that adding the following ImageSettings as a resource works rather well at the moment :)
.insert_resource(ImageSettings {
    default_sampler: SamplerDescriptor {
        address_mode_u: bevy::render::render_resource::AddressMode::Repeat,
        address_mode_v: bevy::render::render_resource::AddressMode::Repeat,
        address_mode_w: bevy::render::render_resource::AddressMode::Repeat,
        ..Default::default()
    },
})

Unable to reproduce any of the fixes suggested in this issue, I ended up writing a little shader to fix the problem in my project: https://github.com/janhohenheim/foxtrot/blob/main/assets/shaders/repeated.wgsl
Most of the code is just boilerplate in order to support the shadows and light features from the default PBR shader.

Sadly, this doesn't help when you need textures with different parameters. And current way of listening for events when image is loaded is way to cumbersome while also not working when image is attached to the material beforehand :(

For those looking for the v0.10 compatible workaround:
.add_plugins(
  DefaultPlugins
    .set(ImagePlugin {
      default_sampler: SamplerDescriptor {
        address_mode_u: AddressMode::Repeat,
        address_mode_v: AddressMode::Repeat,
        address_mode_w: AddressMode::Repeat,
        ..Default::default()
      },
    })
)

Specifying sampler works only if you have a single texture that you want to repeat.
My problem is that I have a large texture (texture atlas) and I want to repeat a region of it, which is not currently possible without writing custom shader.
So having support for it in StandardMaterial would be nice

I ended up just bypassing AssetsServer and loading the images directly. It's not ideal, but it doesn't require listening for events and it allows for both tiled and clamped textures. The loading is done synchronously this way and you'll need to give it a path relative to your working directory (probably just adding assets/ to the start of the string for most people).
pub fn load_tiled_texture(
    images: &mut Assets<Image>,
    texture_path: &str
) -> Handle<Image> {
    let ext = std::path::Path::new(texture_path).extension().unwrap().to_str().unwrap();
    let img_bytes = std::fs::read(texture_path).unwrap();
    let mut image = Image::from_buffer(&img_bytes, ImageType::Extension(ext), CompressedImageFormats::all(), true).unwrap();
    image.sampler_descriptor = ImageSampler::Descriptor(SamplerDescriptor {
        address_mode_u: AddressMode::Repeat,
        address_mode_v: AddressMode::Repeat,
        ..Default::default()
    });
    images.add(image)
}

With the new asset server in 0.12, this worked for me

    let sampler_desc = ImageSamplerDescriptor {
        address_mode_u: ImageAddressMode::Repeat,
        address_mode_v: ImageAddressMode::Repeat,
        ..Default::default()
    };

    let settings = move |s: &mut ImageLoaderSettings| {
        s.sampler = ImageSampler::Descriptor(sampler_desc.clone());
    };

    let texture_handle = assets.load_with_settings("texture.png", settings);

For tiling backgrounds specifically, see https://lib.rs/crates/bevy_tiling_background for prior art.

This is now just a documentation issue: we should demonstrate the code showed by @blunted2night in a simple 2D example.

With the new asset server in 0.12, this worked for me
    let sampler_desc = ImageSamplerDescriptor {
        address_mode_u: ImageAddressMode::Repeat,
        address_mode_v: ImageAddressMode::Repeat,
        ..Default::default()
    };

    let settings = move |s: &mut ImageLoaderSettings| {
        s.sampler = ImageSampler::Descriptor(sampler_desc.clone());
    };

    let texture_handle = assets.load_with_settings("texture.png", settings);

@blunted2night how did you spawn the sprite? this is my attempt and it doesn't seem to work properly:
fn load_tiled_texture(assets: Res<AssetServer>, mut commands: Commands) {
    let sampler_desc = ImageSamplerDescriptor {
        address_mode_u: ImageAddressMode::Repeat,
        address_mode_v: ImageAddressMode::Repeat,
        ..default()
    };

    let settings = move |s: &mut ImageLoaderSettings| {
        s.sampler = ImageSampler::Descriptor(sampler_desc.clone());
    };

    commands.spawn(SpriteBundle {
        texture: assets.load_with_settings("tile.png", settings),
        ..default()
    });
}

I'm a beginner to both rust and bevy, sorry if it's a rookie mistake.

@evrsen in my case, I was using the texture for a terrain component I was playing with. I'm not sure but I think you would need to override the set of texture coordinates used for the sprite. I assume by default it would select 0 -> 1 for both axis.
commands.spawn(SpriteBundle {
    sprite: Sprite {
      rect: Some(Rect::new(0,0,10,1)),
      ..default()
    },
    texture: assets.load_with_settings("tile.png", settings),
    ..default()
});

Is it possible that changing the sampler after the image has already been loaded still has no effect? If so, can I somehow force the change? The reason I ask is because I want to apply the repeating effect to a texture that I'm loading from a .glb scene, so I don't really have the opportunity to call load_with_settings for just that one texture (AFAIK).

Since version 0.12 there is Bevy Asset v2. So you can add ImageSamplerDesctription settings via asset meta files near you png, instead of changing it from a code. For example there is my config:
(
    meta_format_version: "1.0",
    asset: Process(
        processor: "bevy_asset::processor::process::LoadAndSave<bevy_render::texture::image_loader::ImageLoader, bevy_render::texture::compressed_image_saver::CompressedImageSaver>",
        settings: (
            loader_settings: (
                format: FromExtension,
                is_srgb: true,
                sampler: Descriptor(
                    ImageSamplerDescriptor(
                        address_mode_u: Repeat,
                        address_mode_v: Repeat,
                        address_mode_w: Repeat,
                        mag_filter: Nearest,
                        min_filter: Linear,
                        mipmap_filter: Linear,
                        lod_min_clamp: 0.0,
                        lod_max_clamp: 32.0,
                        anisotropy_clamp: 1,
                    ),
                ),
                asset_usage: ("RENDER_WORLD"),
            ),
            saver_settings: (),
        ),
    ),
)

But it adds just abbility to Repeat texture but not abbility to set proportions. For proportions you can use StandardMaterial::uv_transform. Which is pretty fresh, because it has been added for KHR_texture_transform support to current main and will be available from next 0.14 release

@bugsweeper would that meta file entry also work for setting only a specific texture of a gltf to repeat?

would that meta file entry also work for setting only a specific texture of a gltf to repeat?

No, bevy_gltf has code for sampler definition and it defines ImageAddressMode according to info loaded from gltf, but you define sampler inside your gltf file, which is more appropriate way

@bugsweeper thanks for the info, I didn't know that! Do you know whether Bevy respects these sampler settings in the gltf? If yes, I'd say we can close this issue.

Do you know whether Bevy respects these sampler settings in the gltf? If so, I'd say we can close this issue.

Yes, it uses code I mentioned above to generate sampler for images loaded from gltf

Nice! @alice-i-cecile, since I last blocked the closing of this issue, I think all of my concerns have been addressed, or at least they will be once 0.14 lands with my KHR_texture_transform changes. Voting to add this to the 0.14 milestone and closing it :)

But I think you should try to modify your gltf, to ensure it works as you expected, and then close this issue.

But I think you should try to modify your gltf, to ensure it works as you expected, and then close this issue.

Sounds like a good idea, but I won't have time for at least a couple of weeks. I'd be glad if someone else confirmed this :)


[Template] dokuwiki syntax

Can you create a new template for an export in dokuwiki langage ? (or explain how to create a simple writer for this)
Thanks you

Hello @sulian,
At this moment we have several other priorities to deal with and I would not consider it fair to keep this issue open while I think we can't get to it in time. I like the idea and can see how it would be useful to people.
I can point you to the documentation page where we give an overview on how to write your own template and use twig, for example, to render the necessary items in the right locations: https://phpdoc.org/docs/latest/getting-started/changing-the-look-and-feel.html

Hello @mvriel, thanks for your response. I'm looking the documentation.


FEAT: Categorise products and add a comprehensive list of products.

Bug Description: The search funciton at the top of the page does not work.

Expected Behaviour: The search function should filter out the products as per the keyword entered in the searchbar.

Current Behaviour: Nothing happens, the products are shown exactly as before, no filter is applied.

Screenshot:

Contributing under: GSSOC

Kindly assign me this issue, I would like to work on it.

closing this and raising a new issue.


feat(save): remove dead code to prevent nil value on save

get_config() is removed and replaced by
get_term_config() and get_mark_config()

and rubber stamp

And actually rubber stamp


Add glfwCreateWindowWGPUSurface for native WebGPU

As noticed in #2243 it seems logical that GLFW provides a glfwCreateWindowWGPUSurface for WebGPU similar to the glfwCreateWindowSurface it has for Vulkan, as this is the only part of the process of creating a "hello triangle" in native WebGPU that requires OS-specific code (more details).
I am not sure about all the details of this PR, I know the original glfw3webgpu repo works well but maybe I did not integrate it as well as I could in this project's CMakeLists!

It does. It shouldn't change meaningfully, the API already done completely through extension structs.

Hey @elmindreda I see you've added this to your "Needs Investigation" list, is there any way I can help?
FWIW I've been actively using this for the past year, and it is a part of the LearnWebGPU guide without trouble, so it's not just me!

WIP support for the new runtime platform switch (X11/Wayland): https://github.com/eliemichel/glfw3webgpu/pull/8 Will be ported to this PR once tested enough.

What's the update?

FTR it seems that Dawn renamed the WGPUSType_* enum names lately, will do an update soon (and investigate whether this is meant to be standard).

FTR it seems that Dawn renamed the WGPUSType_* enum names lately, will do an update soon (and investigate whether this is meant to be standard).

Done!


OSM Community Index

Community Resource Name

What is it?

URL link to the resource

Location

Description (optional)

Languages spoken (optional)

Points of contact (optional)

can you please fill in the details?


\dl

I build this application as EXE and run the setup file.Setup file installed successfully but while run application got exception "Could not load file or assembly 'ImageQuality.OpenCV.dll' or one of its dependencies. The specified module could not be found."
Is need required software ?


Package does not build

error: failed to initialize alpm library

See also:

https://gitlab.archlinux.org/archlinux/archlinux-docker/-/issues/56
actions/virtual-environments#2658

Fixed by 769919dd7c243e6d4aed813875b1998cbd670ecd.


out of memory using V100

RuntimeError: CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 15.78 GiB total capacity; 1.22 GiB already allocated; 8.04 GiB free; 1.32 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
Process Process-3:

Please double check your environment, as far as I know, my classmates at ZJU and ETH are able to run the code on devices like 2080ti, TITAN X, 3090.


Support Arrays Up to U1277

This is a bit annoying I can imagine but NTRU-prime performs ring operation on polynomials of lengths that can go up to 1277
so naturally, it will be nice if hybrid arrays can support that. There is another problem which is which is U1277 is not even part of typenum so we can open PR there to extend the numbers in there, or we can add the numbers locally to hybrid-array so we can implement the appropriate trait.
If you can accept numbers up to U1277 to be defined in hybrid array along side the respective derives. I can roll up a quick PR tonight actually.
To observe the usage of such number check possible values of p here then you can check the functions that operate on arrays of size p such as R3_mult here.
If I would add the arrays here, I will add them under custom use flags so they don't have to slow the build time of everyone else.

There is another problem which is which is U1277 is not even part of typenum

That's not a problem. We already support sizes not defined in typenum.
It would be good to discuss this under #66, the existing issue for tracking adding additional array sizes.
Closing as dup.


Need a clear doc page that shows which OS versions Terminal will run on

from https://github.com/microsoft/terminal/issues/15864

Coming from this issue in the PowerShell Repo (that I created) I found PowerShell has this nice easy to understand document section for supported versions of windows
This Doc is the closest similar one & I think could be updated to show where you expect Terminal to actually run, like as show above would provide much needed clarity going forward.

I mean, it's pretty front and center at the top of the README

Windows Terminal requires Windows 10 2004 (build 19041) or later

But we can just add a note at the top of that docs page too. No need to have a whole matrix of options - it's just >10.0.19041 for all terminal versions.

Sorry but this is what I mean by it's not clear & why just using build numbers makes it unclear & would with a matrix with clearer notes as to why it's unsupportable on those os'es would likely have stopped you having ever had the support 2016/2019 issues being raised. It also does not help that Windows 11 still shows up as Windows 10 in many places being crystal clear helps stop confusion, and your docs in this case aren't crystal clear, hence the ask for a matrix. I know to you & others within & outside Microsoft that build numbers make a lot of sense, but the way they are right now is super confusing to many of us, especially if we are managing mixed estates.
Just trying to give some honest and useful feedback here that would definitely help many going forward for what isn't really a big ask in much needed clarity.

We want to make something similar to this in the distributions type.


a_idx2str up and down inverted in GridWorld ?

Hi, I found something weird in the controls in GridWorld. It seems like up and down are inverted:
I used the first cells of the Google Colab tutorial in Google Colab:
from IPython import get_ipython
COLAB = False
if 'google.colab' in str(get_ipython()):
    COLAB = True

if COLAB:
    # install rlberry library
    !git clone https://github.com/rlberry-py/rlberry.git
    !cd rlberry && git pull && pip install -e . > /dev/null 2>&1

    # install ffmpeg-python for saving videos
    !pip install ffmpeg-python > /dev/null 2>&1

    # install optuna for hyperparameter optimization
    !pip install optuna > /dev/null 2>&1

    # packages required to show video
    !pip install pyvirtualdisplay > /dev/null 2>&1
    !apt-get install -y xvfb python-opengl ffmpeg > /dev/null 2>&1

    print("")
    print(" ~~~  Libraries installed, please restart the runtime! ~~~ ")
    print("")

Then,
# Create directory for saving videos
!mkdir videos > /dev/null 2>&1

# Initialize display and import function to show videos
import rlberry.colab_utils.display_setup
from rlberry.colab_utils.display_setup import show_video

And finally, I just move down: action 3
from rlberry.envs import GridWorld

# A grid world is a simple environment with finite states and actions, on which
# we can test simple algorithms.
# -> The reward function can be accessed by: env.R[state, action]
# -> And the transitions: env.P[state, action, next_state]
env_ctor = GridWorld
env_kwargs =dict(nrows=3, ncols=10,
                reward_at = {(1,1):0.1, (2, 9):1.0},
                walls=((1,4),(2,4), (1,5)),
                success_probability=0.9)
env = env_ctor(**env_kwargs)

## ---- MY CODE ---- ##
print("Information on action: ", env.a_idx2str)

env.enable_rendering()
print(env.state)
env.reset()
print(env.state)
env.step(3)
print(env.state)
env.step(0)
## ---- MY CODE ---- ##

# save video and clear buffer
env.save_video('./videos/gw.mp4', framerate=5)
env.clear_render_buffer()
# show video
show_video('./videos/gw.mp4')

And this is the output:

The agent went upward...

Thank you for pointing out, I think they are indeed inverted, will fix it.


Add Markov model in MTK

Basically a stochastic version of this:
https://github.com/epirecipes/sir-julia/blob/master/markdown/function_map_mtk/function_map_mtk.md
which is a port of the below to MTK:
https://github.com/epirecipes/sir-julia/blob/master/markdown/markov/markov.md
Any suggestions @ChrisRackauckas? I know I have to use substitute due to Sym/Int64 conflicts with Binomial

We don't have a symbolic setup right now, but https://github.com/SciML/DifferenceEquations.jl is getting written.


Support for aggregations in the TypeScript client

FrankCastle — Today at 2:53 PM
Btw, is there a way to perform aggregate operations like finding sums using the Tigris SDK? I know these are possible in mongo (https://www.mongodb.com/docs/manual/aggregation/) and I can use Tigris MongoDB compatibility. But I really want to stick to using Tigris typescript SDK.

I'm looking for just SUM and COUNT on simple fields after applying a filter. Aggregating over the fields in nested array-objects would be super-helpful


feat: update cv for 2021

:tada: This PR is included in version 1.15.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Feature/api table styles

Changes proposed in this Pull Request:

Make the tables look pretty

There are quite a few places where the markdown did not get interpreted correctly. I don't know if this is within the scope of this PR though

There are quite a few places where the markdown did not get interpreted correctly. I don't know if this is within the scope of this PR though

I'd say fixing anything table-related can be a part of this story, yes.  But 'primary' 'accent' and 'warn' are strings which is why they appear quoted instead of as a type boolean or undefined.
I see 'primary' is displayed as a string & a type in 2 separate places, I'll update it to be a string.

There are quite a few places where the markdown did not get interpreted correctly. I don't know if this is within the scope of this PR though

I'd say fixing anything table-related can be a part of this story, yes. But 'primary' 'accent' and 'warn' are strings which is why they appear quoted instead of as a type boolean or undefined.
I see 'primary' is displayed as a string & a type in 2 separate places, I'll update it to be a string.

@huayunh I just noticed what you pointed out, the table is all wonky - I'll patch that.

If the goal is to make the table look exactly what we specify in our design docs,

Other than that, i think your table looks good!
I'd say this thing does not belong in the "API  Docs" tab, as it should be shown under "Example". But that is out of the scope

Thanks @huayunh , PR has been updated.


Training for federated model

I have a question about Training for federated models.
I don't understand the difference between the Custom Dataset and the distributed Training for federated model.
Am I correct in assuming that the Training for federated model in the readme link is the KITTI format data that you created in Custom Dataset Generation?
Thank you very much for your help.

First, the command
python3 gen_data/Scenario.py record [x_1,...x_32] [y_1,...y_5]
only generates raw data (not KITTI format).
The raw data folder is something like raw_data/record2020_xxxx_xxxx
Second, to generate a customized dataset with KITTI format, further run the command
python3 gen_data/Process.py raw_data/record2020_xxxx_xxxx
This will create a folder $ROOT_PATH/dataset/record2020_xxxx_xxxx
It looks like:
dataset
+- record2020_xxxx_xxxx
+- global_label   #global labels
+- vhicle.xxx.xxx_xxx
+- calib00
+- 0000.txt
+- 0001.txt
+- image00
+- 0000.jpg
+- 0001.jpg
+- label00
+- 0000.txt
+- 0001.txt
+- velodyne
+- 0000.bin
+- 0001.bin
+- vhicle.xxx.xxx_xxx
+- img_list.txt   # frame IDs
Finally, our federated training is based on PCDet project. The data format is slightly different from the above (kindly see the PCDet readme). The dataset folder looks like:
data
+- record2020_xxxx_xxxx
+- global_label    # same as global labels in “dataset”
+- vhicle.xxx.xxx_xxx
+- Imagesets
+- train.txt   # same format as img_list.txt in “dataset”
+- test.txt
+- val.txt
+- training
+- calib  # same as calib00 in “dataset”
+- 0000.txt
+- 0001.txt
+- image_2   # same as image00 in “dataset”
+- 0000.jpg
+1 0001.jpg
+- label_2   # same as label00 in “dataset”
+- 0000.txt
+- 0001.txt
+- velodyne   # same as velodyne in “dataset”
+- 0000.bin
+- 0001.bin
With the above data structure, run the following:
cd ./PCDet/pcdet/datasets/kitti
python3 preprocess.py create_kitti_infos record2020_xxxx_xxxx vhicle_id

First, the command
python3 gen_data/Scenario.py record [x_1,...x_32] [y_1,...y_5]
only generates raw data (not KITTI format).
The raw data folder is something like raw_data/record2020_xxxx_xxxx
Second, to generate a customized dataset with KITTI format, further run the command
python3 gen_data/Process.py raw_data/record2020_xxxx_xxxx
This will create a folder $ROOT_PATH/dataset/record2020_xxxx_xxxx
It looks like:
dataset
+- record2020_xxxx_xxxx
+- global_label   #global labels
+- vhicle.xxx.xxx_xxx
+- calib00
+- 0000.txt
+- 0001.txt
+- image00
+- 0000.jpg
+- 0001.jpg
+- label00
+- 0000.txt
+- 0001.txt
+- velodyne
+- 0000.bin
+- 0001.bin
+- vhicle.xxx.xxx_xxx
+- img_list.txt   # frame IDs
Finally, our federated training is based on PCDet project. The data format is slightly different from the above (kindly see the PCDet readme). The dataset folder looks like:
data
+- record2020_xxxx_xxxx
+- global_label    # same as global labels in “dataset”
+- vhicle.xxx.xxx_xxx
+- Imagesets
+- train.txt   # same format as img_list.txt in “dataset”
+- test.txt
+- val.txt
+- training
+- calib  # same as calib00 in “dataset”
+- 0000.txt
+- 0001.txt
+- image_2   # same as image00 in “dataset”
+- 0000.jpg
+1 0001.jpg
+- label_2   # same as label00 in “dataset”
+- 0000.txt
+- 0001.txt
+- velodyne   # same as velodyne in “dataset”
+- 0000.bin
+- 0001.bin
With the above data structure, run the following:
cd ./PCDet/pcdet/datasets/kitti
python3 preprocess.py create_kitti_infos record2020_xxxx_xxxx vhicle_id

@bearswang Please help answer this question, thanks.

First, the command
python3 gen_data/Scenario.py record [x_1,...x_32] [y_1,...y_5]
only generates raw data (not KITTI format). The raw data folder is something like raw_data/record2020_xxxx_xxxx
Second, to generate a customized dataset with KITTI format, further run the command
python3 gen_data/Process.py raw_data/record2020_xxxx_xxxx
This will create a folder $ROOT_PATH/dataset/record2020_xxxx_xxxx
It looks like:
dataset
+- record2020_xxxx_xxxx
      +- global_label   #global labels
      +- vhicle.xxx.xxx_xxx
          +- calib00
              +- 0000.txt
              +- 0001.txt
          +- image00
              +- 0000.jpg
              +- 0001.jpg
          +- label00
              +- 0000.txt
              +- 0001.txt
          +- velodyne
              +- 0000.bin
              +- 0001.bin
      +- vhicle.xxx.xxx_xxx
      +- img_list.txt   # frame IDs

Finally, our federated training is based on PCDet project. The data format is slightly different from the above (kindly see the PCDet readme). The dataset folder looks like:
data
   +- record2020_xxxx_xxxx
      +- global_label    # same as global_label in “$ROOT_PATH/dataset/record2020_xxxx_xxxx”
      +- vhicle.xxx.xxx_xxx
   +- Imagesets
      +- train.txt   # same format as img_list.txt in “$ROOT_PATH/dataset/record2020_xxxx_xxxx”
      +- test.txt
      +- val.txt
   +- training
      +- calib  # same as calib00 in “$ROOT_PATH/dataset/record2020_xxxx_xxxx”
          +- 0000.txt
          +- 0001.txt
      +- image_2   # same as image00 in “$ROOT_PATH/dataset/record2020_xxxx_xxxx”
          +- 0000.jpg
          +1 0001.jpg
      +- label_2   # same as label00 in “$ROOT_PATH/dataset/record2020_xxxx_xxxx”
          +- 0000.txt
          +- 0001.txt
      +- velodyne   # same as velodyne in “$ROOT_PATH/dataset/record2020_xxxx_xxxx”
          +- 0000.bin
          +- 0001.bin

With the above data structure, run the following:
cd ./PCDet/pcdet/datasets/kitti
python3 preprocess.py create_kitti_infos record2020_xxxx_xxxx vhicle_id
This will create several info files for training and testing in PCDet. More details can be found at: https://github.com/open-mmlab/OpenPCDet


Again Oracle Create Procedure Throws Invalid SQL Statement

System information:

Operating system (distribution) and version Windows 10
DBeaver version 7.3.3
Additional extensions

Connection specification:

Database name and version Oracle 19i
Driver name ojdbc8 <IP_ADDRESS>
Do you use tunnels or proxies (SSH, SOCKS, etc)? No

Describe the problem you're observing:
There is again a bug with Oracle Database CREATE PROCEDURE. In DBeaver 7.3.3 the code from issue #3742 throws:
SQL Error [900] [42000]: ORA-00900: invalid SQL statement
ORA-00900: invalid SQL statement

But it executes without a problem on the Oracle SqlDeveloper Client.
Steps to reproduce, if exist: Like in https://github.com/dbeaver/dbeaver/issues/3742
Include any warning/errors/backtraces from the logs

Another example
CREATE OR REPLACE PROCEDURE DROPPRIMCONSTRAINT(ATABLENAME IN VARCHAR2,
AOWNER     IN VARCHAR2)
AS
CURSOR CUR IS
SELECT CONSTRAINT_NAME
FROM ALL_CONSTRAINTS
WHERE CONSTRAINT_TYPE = 'P'
AND OWNER = AOWNER
AND TABLE_NAME = ATABLENAME;
BEGIN
FOR I IN CUR
LOOP
--DBMS_OUTPUT.PUT_LINE('ALTER TABLE ' || AOWNER || '.' || ATABLENAME || ' DROP CONSTRAINT ' || I.CONSTRAINT_NAME || ' DROP INDEX');
EXECUTE IMMEDIATE 'ALTER TABLE ' || AOWNER || '.' || ATABLENAME || ' DROP CONSTRAINT ' || I.CONSTRAINT_NAME || ' DROP INDEX';
END LOOP;
END DROPPRIMCONSTRAINT;
/

Another example
CREATE OR REPLACE PROCEDURE DROPPRIMCONSTRAINT(ATABLENAME IN VARCHAR2,
AOWNER     IN VARCHAR2)
AS
CURSOR CUR IS
SELECT CONSTRAINT_NAME
FROM ALL_CONSTRAINTS
WHERE CONSTRAINT_TYPE = 'P'
AND OWNER = AOWNER
AND TABLE_NAME = ATABLENAME;
BEGIN
FOR I IN CUR
LOOP
--DBMS_OUTPUT.PUT_LINE('ALTER TABLE ' || AOWNER || '.' || ATABLENAME || ' DROP CONSTRAINT ' || I.CONSTRAINT_NAME || ' DROP INDEX');
EXECUTE IMMEDIATE 'ALTER TABLE ' || AOWNER || '.' || ATABLENAME || ' DROP CONSTRAINT ' || I.CONSTRAINT_NAME || ' DROP INDEX';
END LOOP;
END DROPPRIMCONSTRAINT;
/

Duplicate of #11510


Improvement idea: Use specific type for exceptions

Hi,
It would be very useful for exception handling if the following utils:
Utils.QL_REQUIRE
throws a dedicated type of exception instead of the generic System.Exception. C#6 and C#7 have lots of goodies to handle exceptions by type and QLNet ones cannot be differentiate easily.
Change is minor:

Create an exception class that inherits System.Exception
Change the "Utils.QL_REQUIRE" code to generate this new type instead

What do you think?
PS: having two new Exception types would be even better. One for QL_REQUIRE and another for QL_FAIL.

+1 for the 2 exception

I also support this, but I think the exceptions should be more granular than just QL_REQUIRE and QL_FAIL. For example, an exception for missing / wrong inputs. Another exception for calculation error. etc.

Totally agree on the "more granular" idea. Also, exception could be filled with context property like strike, dates, ... so that we do not only rely on the message it self to get the information.
But I'd say: step by step.

https://github.com/amaggiulli/QLNet/commit/39d4390d8c11cb2d62c280a78807cefba1c99e27