[UNDERTOW-1828] NullPointerException occurred if a servlet calls Http…

issue: https://issues.redhat.com/browse/UNDERTOW-1828
2.1.x PR: https://github.com/undertow-io/undertow/pull/1021
2.0.x PR: https://github.com/undertow-io/undertow/pull/1022

Is it a correct behavior to treat contextPath as "/" if deploymentInfo == null?
Similar NullPointerException can happen on other methods in ServletContextImpl, too.
What about adding a check logic for deploymentInfo? Then, if deploymentInfo == null,  throwing IllegalStateException with a message like this context has been already destroyed?

Is it a correct behavior to treat contextPath as "/" if deploymentInfo == null?
Similar NullPointerException can happen on other methods in ServletContextImpl, too.
What about adding a check logic for deploymentInfo? Then, if deploymentInfo == null,  throwing IllegalStateException with a message like this context has been already destroyed?

@msfm

There was already a check  if (contextPath.equals("/")), however server stops on ctrl+c so it will remain on same context, that is the reason I have  given contextPath as "/" if deploymentInfo == null.
If there are other locations then we can add it there too but dont have any test case to reproduce it aS NullpointerException.
If you think throwing Exception is the best option thyen I will add it.

Thanks.

There was already a check if (contextPath.equals("/"))

I think that check exists only for the different purpose. That logic exists for the following statement described in the API Doc of ServletContext#getContextPath():

Returns:
The context path of the web application, or "" for the default (root) context

If there are other locations then we can add it there too but dont have any test case to reproduce it as NullpointerException.

Similar NullpointerException can happen on the following methods:

context.getServerInfo()
context.getMajorVersion()
context.getMinorVersion()
context.getEffectiveMajorVersion()
context.getEffectiveMinorVersion()
context.getResourcePaths(path)
context.getResourcePath(path)

For example, modify the reproducer which is attached in JBEAP-20706 as shown below:
    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        log.infof("# start");
        try {
            Thread.sleep(10000L);
        } catch (ignore e) { }
        ServletContext context = request.getServletContext();
        System.out.println("# context.getServerInfo() = " + context.getServerInfo());
    }

and follow the same steps to reproduce.
Then, you will see similar NullpointerException happens at io.undertow.servlet.spec.ServletContextImpl.getServerInfo(ServletContextImpl.java:417).

There was already a check if (contextPath.equals("/"))

I think that check exists only for the different purpose. That logic exists for the following statement described in the API Doc of ServletContext#getContextPath():

Returns:
The context path of the web application, or "" for the default (root) context

If there are other locations then we can add it there too but dont have any test case to reproduce it as NullpointerException.

Similar NullpointerException can happen on the following methods:

context.getServerInfo()
context.getMajorVersion()
context.getMinorVersion()
context.getEffectiveMajorVersion()
context.getEffectiveMinorVersion()
context.getResourcePaths(path)
context.getResourcePath(path)

For example, modify the reproducer which is attached in JBEAP-20706 as shown below:
    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        log.infof("# start");
        try {
            Thread.sleep(10000L);
        } catch (ignore e) { }
        ServletContext context = request.getServletContext();
        System.out.println("# context.getServerInfo() = " + context.getServerInfo());
    }

and follow the same steps to reproduce.
Then, you will see similar NullpointerException happens at io.undertow.servlet.spec.ServletContextImpl.getServerInfo(ServletContextImpl.java:417).

There was already a check if (contextPath.equals("/"))

I think that check exists only for the different purpose. That logic exists for the following statement described in the API Doc of ServletContext#getContextPath():

Returns:
The context path of the web application, or "" for the default (root) context

If there are other locations then we can add it there too but dont have any test case to reproduce it as NullpointerException.

Similar NullpointerException can happen on the following methods:

context.getServerInfo()
context.getMajorVersion()
context.getMinorVersion()
context.getEffectiveMajorVersion()
context.getEffectiveMinorVersion()
context.getResourcePaths(path)
context.getResourcePath(path)

For example, modify the reproducer which is attached in JBEAP-20706 as shown below:
    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        try {
            Thread.sleep(10000L);
        } catch (Exception ignore) { }
        ServletContext context = req.getServletContext();
        System.out.println("# context.getServerInfo() = " + context.getServerInfo());
    }

and follow the same steps to reproduce.
Then, you will see similar NullpointerException happens at io.undertow.servlet.spec.ServletContextImpl.getServerInfo(ServletContextImpl.java:417).

What do you suggest now?

There was already a check if (contextPath.equals("/"))

I think that check exists only for the different purpose. That logic exists for the following statement described in the API Doc of ServletContext#getContextPath():

Returns:
The context path of the web application, or "" for the default (root) context

If there are other locations then we can add it there too but dont have any test case to reproduce it as NullpointerException.

Similar NullpointerException can happen on the following methods:

context.getServerInfo()
context.getMajorVersion()
context.getMinorVersion()
context.getEffectiveMajorVersion()
context.getEffectiveMinorVersion()
context.getResourcePaths(path)
context.getResourcePath(path)

For example, modify the reproducer which is attached in JBEAP-20706 as shown below:
    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        try {
            Thread.sleep(10000L);
        } catch (Exception ignore) { }
        ServletContext context = req.getServletContext();
        System.out.println("# context.getServerInfo() = " + context.getServerInfo());
    }

and follow the same steps to reproduce.
Then, you will see similar NullpointerException happens at io.undertow.servlet.spec.ServletContextImpl.getServerInfo(ServletContextImpl.java:417).

What do you suggest now?

My suggestion does not change. What about adding a check logic for deploymentInfo before using deploymentInfo inside ServletContextImpl? Then, if deploymentInfo == null, throwing another exception (e.g. IllegalStateException) instead of NullPointerException with a meaningful message (e.g. this context has been already destroyed, etc).

My suggestion does not change. What about adding a check logic for deploymentInfo before using deploymentInfo inside ServletContextImpl? Then, if deploymentInfo == null, throwing another exception (e.g. IllegalStateException) instead of NullPointerException with a meaningful message (e.g. this context has been already destroyed, etc).

My suggestion does not change. What about adding a check logic for deploymentInfo before using deploymentInfo inside ServletContextImpl? Then, if deploymentInfo == null, throwing another exception (e.g. IllegalStateException) instead of NullPointerException with a meaningful message (e.g. this context has been already destroyed, etc).

@msfm changes modified, please review.
Thanks.

My suggestion does not change. What about adding a check logic for deploymentInfo before using deploymentInfo inside ServletContextImpl? Then, if deploymentInfo == null, throwing another exception (e.g. IllegalStateException) instead of NullPointerException with a meaningful message (e.g. this context has been already destroyed, etc).

@msfm changes modified, please review.
Thanks.

@moulalis  I think it is best if we add a check in a single method, and we add the exception message to logger. I'll be updating your PR within the next minutes to make sure it is included in the next release. Thanks for your PR!

@moulalis  I think it is best if we add a check in a single method, and we add the exception message to logger. I'll be updating your PR within the next minutes to make sure it is included in the next release. Thanks for your PR!

@moulalis I think it is best if we add a check in a single method, and we add the exception message to logger. I'll be updating your PR within the next minutes to make sure it is included in the next release. Thanks for your PR!

@fl4via I have changed it accordingly. Thanks

@moulalis I think it is best if we add a check in a single method, and we add the exception message to logger. I'll be updating your PR within the next minutes to make sure it is included in the next release. Thanks for your PR!

@fl4via I have changed it accordingly. Thanks

It's just nit-picking..., but the indentations don't seem to be aligned.

It's just nit-picking..., but the indentations don't seem to be aligned.


#16 Add Magento 2.4.0 compatibility

https://github.com/integer-net/magento2-session-unblocker/issues/16

Hi @wigman,
I tested - this version at least successfully installs on Magento 2.4.
Could you please merge this PR and add stable version?

Thanks for confirming @ihor-sviziev! I've merged it, and asked @schmengler if he wants to tag a new release.


fix: additional-css UNIX style path normalization

Clone of #162 with fixes/ci applied
Awaiting response from original author, review 21/12/2023

The merge is coming...


add command docker stack services STACKNAME

Implement #24042
- What I did
1.Add command docker stack services STACKNAME;
2.export functions of service for this command;
3.add a test case.
- How I did it
*- How to verify it
root@10-11-17-53:~# docker stack services root
ID            NAME      REPLICAS  IMAGE                                                                                COMMAND
4kuqna4wyf0g  root_web  1/1       javajudd/nuez@sha256:fa49b331c07aa70e3fa63a7fd0cebfeb26a0c9ddfcecb5237677f09234e52f51
a7hbfin1aaor  root_db   1/1       javajudd/nuez-db@sha256:e1c7c863670e0938ee994c98f0c34bbe4c5fe0c6fc572270e414d3eca1f81950

Signed-off-by: allencloud<EMAIL_ADDRESS>

Any update with the function duplication? @dnephin

Design looks good to me, moving to code review.

@allencloud What do you need from @dnephin? It seems he makes a valid point about duplicated code. Let us know if it's unclear!

I exported two functions in service/list.go to avoid duplication.
@dnephin

Works for me if it works for Daniel ;-) LGTM

LGTM

Actually, I found previously the integration test only has one test. So I rebased the code, ran the normal 8 tests. Unfortunately, There is one failing.
@icecrime @dnephin @LK4D4

CI is green now, and tested locally
LGTM, thanks! I'll do a follow up to add some docs to the CLI reference

opened https://github.com/docker/docker/pull/24852 for the reference docs

PrintNotQuiet? Could we please start reviewing PRs?


Bump rules_nodejs back to 5.8.4

Type of change
https://github.com/aspect-build/rules_js/pull/1311 reverted the rules_nodejs version back to 5.8.2. This drops support for Node.js v18.17.0. This PR re-reverts the rules_nodejs version to 5.8.4
Test plan

Covered by existing test cases

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@alexeagle I'm unable to assign this PR or request a review. Can you please take a look? This should be a quick fix. Thanks!


Add homeassistant-core container 🎁

Hi @dusty-nv !
This is the last PR from me this month 🤞 🙈 (surprise!)

Add new homeassistant-core container;
homeassistant-core container available to build using the latest or specific version, eg: 2024.03.1;
Add config.py to python container together with ppa:deadsnakes/ppa logic to allow installation of python 3.11, python 3.12, etc...

Have fun!

Thanks @ms1design, amazing!  👍
This is a great start to HomeAssistant integration, thank you.   That is also great what you did with the python container and deadsnakes ppa.  Eventually we can clean up that dockerfile by just making an install_python.sh script so there isn't so much cludgery in the dockerfile RUN commands (I suspect most of the repo will end up migrating to a similar approach to not only save layers, but make them more clean/editable and also those scripts can be reused by other ppl outside of containers).  But this is great for now!

You're welcome @dusty-nv 👍
Bare in mind that homeassistant-core requires python 3.11 - but this we could fix later. I'm 100% with you on the creating some sh scripts for such purposes as installing specific versions of dependencies, cleanups, etc.

Updated, but now I'm trying to remember if I had it set to off because of this: https://github.com/pypa/pip/issues/2897 or not 🤔


Maven multi modules

I think it's ready to merge.
This is a big change, I think you could consider releasing this as the next major release, even if API is still compatible with current codebase.

All features have been splitted into maven modules.

Every piece of random data is now generated through the Randomizer interface. It ensure consistency between randomizers used internally by jPopulator and those writen by the user.

Internal randomizers are now declared in RandomizerRegistry implementations. Each registry is a set of Randomizers, and is automatically registered in PopulatorBuilder using Java Provider API (ServiceLoader + META-INF/services/io.github.benas.jpopulator.api.RandomizerRegistry file). Currently, there's 3 registries : InternalRandomizerRegistry, JodaRandomizerRegistry and ValidationRandomizerRegistry

PopulatorBuilder now supports a registerRandomizerRegistry method to add custom randomizer registries. Also, disableDefaultRegistries allow disabling of default registries provided by Java Provider API.

To skip a value, a Randomizer should now throw RandomizerSkipException instead of returning null. As a consequence, a Randomizer returning null will be effective and the null value will be set on the requested field. (required for @Null validation randomizer).

An EnumRandomizer interface has been added to the API to support future customization of enum random values.

Mahmoud Ben Hassine » jPopulator #103 SUCCESS
This pull request looks good
(what's this?)

Mahmoud Ben Hassine » jPopulator #104 SUCCESS
This pull request looks good
(what's this?)

To continue this work, I need a refactoring because some parts of the code is coupled and make some class moving note possible.
Main problem is in PopulatorImpl#populateSimpleType method : architecture for custom randomizer is good and use one Randomizer implementation per type, but default supported types are implemented in two classes : DefaultRandomizer and BeanValidationRandomizer.
Those two classes adds coupling on joda-time dependency for DefaultRandomizer, and javax-validation for BeanValidationRandomizer.
So in need to refactor those two classes using Randomizer implementations, and register them like custom randomizers in PopulatorImpl class when the jpopulator-joda-time/jpopulator-validation module is available.

I see the trick. That's fine, go for refactoring those classes. thx

Mahmoud Ben Hassine » jPopulator #105 SUCCESS
This pull request looks good
(what's this?)

Mahmoud Ben Hassine » jPopulator #106 SUCCESS
This pull request looks good
(what's this?)

Mahmoud Ben Hassine » jPopulator #107 SUCCESS
This pull request looks good
(what's this?)

@benas any about about this and #33 ?

Hello Rémi,
Sorry for being late, there are a lot of changes and I did not got time to check them all.
Everything looks good. I'll merge your PR as soon as I can.
Kind regards
Mahmoud

Hi Rémi, I finally got time to review your PR, sorry for being late.

All features have been splitted into maven modules.

That's great!

Every piece of random data is now generated through the Randomizer interface. It ensure consistency between randomizers used internally by jPopulator and those writen by the user.

That's fantastic!

Internal randomizers are now declared in RandomizerRegistry implementations. Each registry is a set of Randomizers, and is automatically registered in PopulatorBuilder using Java Provider API (ServiceLoader + META-INF/services/io.github.benas.jpopulator.api.RandomizerRegistry file). Currently, there's 3 registries : InternalRandomizerRegistry, JodaRandomizerRegistry and ValidationRandomizerRegistry

J'achète :-)

PopulatorBuilder now supports a registerRandomizerRegistry method to add custom randomizer registries.

The concept of registry is a clean way to logically group randomizers together :+1: . But IMO this should not be a public API, the user can supply custom randomizers at construction time (which will be registered behind the scene in a UserRandomizersRegistry), but should not bother with registries.

In regards to priorities, of course we can define it via annotation and so on, but this introduces a unnecessary complexity, I like to KISS.. I mean, to keep it simple :-). In reality, user defined randomizers may override default ones, that's it. There are only two priorities.

Also, disableDefaultRegistries allow disabling of default registries provided by Java Provider API.

With the new Service Loader architecture, disabling a default registry is as simple as not putting its jar in the classpath, which is the most elegant part pf the pattern. So this option is redundant.

To skip a value, a Randomizer should now throw RandomizerSkipException instead of returning null.

I'm not sure this is the right place to decide if the value should be skipped or not. Imagine I want to implement the Randomizer interface. When should I throw a ``RandomizerSkipException` ? I don't have information of the context in which the randomizer is used to decide if the value should be skipped or not. It is the caller of the randomizer that can decide if the value should be skipped or not (ie, the field is annotated with @Exclude or not). Do you see?
So Randomizer should simply generate random values for a given type.

As a consequence, a Randomizer returning null will be effective and the null value will be set on the requested field. (required for @Null validation randomizer).

We can imagine a "NullRandomizer" in the io.github.benas.jpopulator.randomizers.validation (Null Object pattern) or simply set the field to null when the field is annotated with @Null.

An EnumRandomizer interface has been added to the API to support future customization of enum random values.

That's fine, but do we really need a separate interface for that? Why not simply a class EnumRandomizer implements Randomizer<E extends Enum> that will generate a random value from the values set of the enum E?


please tell if I can use it for personal use in some way, as Readme.md file doesn't having any instruction, per se

After a recent PR from @mehedi-shafi, instructions are added.


CustomUserService signin querystring

For the LocalRegistrationController in the CustomUserService sample, the register controller takes a string signin for both Get and Post actions.
The registration works for me but then when it redirects to the login page the signin querystring is blank. Where does the value for signin come from and what is it meant to be. From the other samples I can tell it's some kind of token used to identify the client but I can't tell how the value should be set in this sample so the login page gives an error.
Any hints would be much appreciated.

It should be passed back since in the view it's using  Html.BeginForm() to build the form.

Thank you, I added an MVC client application to the sample and it redirected to the login page from which I could access the register link with the correct signin querystring.


Fix UI bug with accordion closing when updating some of its properties

I cannot fully recall/reproduce the issue, but both @zonia3000 and @jluethi noticed it.
It is then fixed with #518

Closed with #518


Support for JPDB Mining

would be awesome if you could add support for JPDB, if possible.

Explicit JPDB support likely won't be implemented. Though JPDB-connect seems to plan to add JL support.

sounds good, thank you.

So I just started working on this and noticed that it already works without changes 😄
it probably has been working for half a year already
What was a bit finicky was setting up JL to send to a port other than the default. I don't know if I was too dumb to do it properly but setting JL to send to port 3030 and then hitting refresh resulted in an error with a fresh setup.
I then changed jpdb-connect to listen to the default JL port, did the initial card setup that way and was able to change the port back after.

it already works without changes

That's great to hear!

setting JL to send to port 3030 and then hitting refresh resulted in an error with a fresh setup

Preferences aren't validated and applied until you hit the "Save" button. So when you change the "AnkiConnect server address" and hit the refresh button, it will not use the newly entered address since it hasn't been validated and saved yet. I guess this is nonintuitive and an oversight on my part. I will try to improve this in the version of JL.


Implement persistent locks and bans

Write locks and bans to a .csv file to be loaded upon server
start/restart

Why pipe-delimited values? We should either use JSON or stick to CSV as other users features do.

I'd prefer TSV in the future; CSV ends up being sort of dumb in my experience.

Donovan, better use json for server-side objects (check how rooms are stored). Also, remember our conversation? Perhaps add a timer to back bans and locks up once every... 12 hours? What do you think, @Zarel?

I don't like JSON. We use it for rooms only because rooms are too complicated for TSV/CSV.
On second thought, let's go with CSV.
How often we backup bans/locks depends on how slow the backup process is.

I've updated the code to delete expired locks and bans each time the file is loaded and also added the backup interval, currently set to 12 hours. It writes a 110KB file (total of 1000 entries) in 4ms (for reference, we've had about 350 locks and bans in the past 3 days).
I can't easily replicate the server's load so I don't know how big of a difference there is between that and running it locally, but do you think the backup should run more/less frequently?

12 hours is fine for a 4ms runtime.

I've updated the code.

Are there any other changes that need to be made?

We were discussing a week-ban feature as well for serious cases, as an intermediate between the standard ban and permaban, so maybe you could toss that in somewhere.

I think that belongs in a separate pull request with the other timed punishments, which I'll make after this is implemented.

Yeah, just hold on, I'll have time for this eventually.

bump

bump

This needs to be rebased, which I'll do since I have a few other changes I need to make to it, too.

Unlike the other long-running pullreqs, I don't need to optimize main process RAM usage too much for this, so I should be able to get to this soon.

"We can’t automatically merge this pull request. "
:(

Just as well; I needed to manually merge it either way.

knock knock

is still on my list

I've recently started optimizing, too! I told you about that.

Yeah, sorry, I thought this was ready for manual merge.

:^)

Still on my to-do list, I haven't forgotten about it.

one day... one day

I check Issues all the time, so no, I haven't forgotten this.


StatusLight in TableView column looks misaligned because of padding

Expected Behaviour
A  StatusLight in a Cell in a TableView Column, which is by default left-aligned, should be capable of being aligned with its header. I should be able to remove the left padding without resorting to UNSAFE_style. (Likewise for a right-aligned column I might want to remove the right padding).
Actual Behaviour
StatusLight has padding-left so it isn't aligned correctly. Setting paddingStart="0" is not supported.
Reproduce Scenario (including but not limited to)

Steps to Reproduce
Platform and Version
"@adobe/react-spectrum": "3.14.0"
Sample Code that illustrates the problem
<Cell>     <StatusLight variant="negative">        Error     </StatusLight> </Cell>
Logs taken while reproducing problem

Looks like the front padding was removed from the StatusLight in spectrum-css: https://github.com/adobe/spectrum-css/pull/1332, we'll explore ways to do this in RSP.

Closing for the same reason as https://github.com/adobe/react-spectrum/issues/2212#issuecomment-2272586026


Pages from data, take 5

There are existing issues about this, but I prefer to start fresh when I have new ideas on a subject. I thought about this again when having my hair washed at the hairdresser today. Maybe the hair massage helped.
I think I have been too hung up in the technical challenges of this (remote adapters, how to effectively do partial updates etc.), making the whole issue too big to start with. What we have talked about earlier have also been "something different on the side of what we already have".
But what we have is:

A virtual /content directory that can be composed via Hugo Modules (with overrides on file level)
Virtual mounts support (you can mount any directory or file, even from remote GitHub repos into /content)
A front matter based metadata model with cascade keyword etc.
Partial server updates based on filesystem events.
...

With that I mind, I thought about adding a new reserved filename in /content starting with _content.
Given the example below:

_content.json, _content.toml (and YAML) would be fairly straight forward, i.e. metadata + content. We should probably support a tree structure somehow, so you can build a complete content structure from one root _contentfile. We should probably support multiple files per directory so you could do _content_products.json etc.
_content.go would represent the dynamic content, some kind of content adapter, possibly remote. This is obviously the area with most open questions (_content.js would be a thought), but having the naming in place is a start.

content
├── _content.json
├── _index.md
├── blog
│   ├── _content.toml
│   ├── _index.md
│   ├── image.png
│   └── post1.md
└── docs
    ├── _content.go
    └── _index.md

/cc @regisphilibert @onedrawingperday @digitalcraftsman @budparr @moorereason @kaushalmodi and gang.

Yes. This is clearer than using the section's _index.md.

We should probably support multiple files per directory so you could do _content_products.json etc.

If we need to create another file, might as well create the directory: /products/_content.json

If we need to create another file, might as well create the directory: /products/_content.json

Yes, probably.
Also thinking, I think I'm going to restrict this to JSON in its first iteration, as that is the only format supporting stream decoding.

Also thinking, I think I'm going to restrict this to JSON in its first iteration

And possibly YAML:
https://github.com/go-yaml/yaml/issues/4

That's great. Now I'm seeing those _content.yaml file with lots of information about the meta data. Which key goes where in a "page" object.
But I was also under the impression that Hugo would fetch the data based on some parameters (endpoints, pagination, etc...) which might have been a bit optimistic.
With _content.go? Does this mean, we'll be able to write our own data fetcher/parser in Go or Javascript and if so, can't the metadata/front_matter be addressed from there?

With _content.go?

So, there is 2 stories to this issue.
The main story being that we ned to break this down into smaller pieces to be able to grasp it and possibly also implement it in iterations.
So:

_content.yaml is raw page data just wrapped in a more "data like" format than a markdown file with frontmatter. The big benefit being that you can create thousands of pages in one file.
_content.go would be the "create those thousands of articles by some kind custom scripting towards and Hugo API" (which would handle all the caching/partial update logic etc).
_content.wordpress(and now I'm just making stuff up) would use a built-in Hugo adapter to pull in those thousand articles.

Got it! Thanks for clarifying.

Oh and, if all Hugo need from a _content.go is to produce an array of item, we could use Go Template and return.
That would allows us to use some partials and other familiar Hugo stuff to parse de data grabbed from GetJSON or else...

Oh and, if all Hugo need from a _content.go is to produce/return an array of items, we could use Go Template and return.

Sure, but what I had in mind, which I admit needs some more thoughs, was something more powerful. Something in the line of what @natefinch suggested here some time ago, a more or less "full Go" plugin type of thing.

I have done some experiments with @natefinch 's https://github.com/starlight-go/starlight today (his library wraps https://github.com/google/starlark-go/ by Google), and I'm very impressed and think it would be a good fit for the above (and also other uses).
It will be yet another thing to learn for Hugo users (it's a Python dialect), but I think well worth it.
It integrates very well with the Go side of the fence. It returns all the variable definitions when evaluating a script, even functions, so it should be possible to define "plugin interfaces" with default implementations, and implement whatever needed in the _content.py file, e.g:
type interface DataGetter {
   GetDataStream() io.ReadCloser
}

Would be implemented in _content.py as:
def GetDataStream():
	return http.Get(site.Param("wordPressAPI"))

See my previous comments, and push the appropriate button below whether you think adding Starlark (a Python dialect) as a scripting language in Go. First as a way for users to write custom "source adapters" for content, but we will most llkely find other use cases, eventually (@natefinch did a PR some time ago with custom template functions in Python).

I think the learning curve of Go Template is hard enough for many new users, letting them know that learning Python is a requisite for Data source sounds a bit harsh.
I'd be willing to invest time into learning some Go, but Python, not so excited. Did you drop JS because of speed?

I'd be willing to invest time into learning some Go, but Python, not so excited. Did you drop JS because of speed?

The options that I have evaluated are Python and Lua. To my knowledge there are no solid solid and embedded JS implementation in Go. But I assume some day it will happen.

And for my own curiosity, why not Go?

And for my own curiosity, why not Go?

Mostly security related.

I think the learning curve of Go Template is hard enough for many new users, letting them know that learning Python is a requisite for Data source sounds a bit harsh.

+1

Mostly security related.

Note that that remark was about "compiled Go" (not Go templates).

I trust you in choosing the fastest most reliable way of letting coders build their own data parsers.
So I'll mention Go Template one last time.
With returning partials, merge, transform.Unmarshal, Hugo's got really better at handling data.
With getJSON (might need improvements and complementing methods) and the new features mentioned above, I know I can build that parser with Go Template. Is it a bad idea?

The main problem/challenge with using Go templates for this is that it's procedural, it's "one script per file", e.g. one method.
I will try to think of a better example, but in the context we're talking about (plugins), it becomes hard/ugly to then create plugin APIs with life cycle methods, e.g.:
def PluginType():
  return "source"

def ShouldUpdate(after):
  return true

def GetDataStream():
	return http.Get(site.Param("wordPressAPI"))

In the above, Hugo could look at the plugin and say "Oh, it supports JSON via a reader (stream), we can optimize for that".
With Go templates, parts of the  above may look like:
{{ if .LastUpdate.After ... }}
{{ .Result.NotModified }}
{{ else }}
{{ .Result.Set getJSON "foo" }}
{{ end }}

Note again that I'm not saying that the above represents a "real plugin interface", but I'm fairly sure that most real plugin scenarios would require some level of "branching". And it would be good if we could write those plugins in something that doesn't look like code from the 80s.
Also note that Starlark is a Python dialect, a sub-set of Python built for this particular purpose (and as an embedded scripting option it is, in my eyes, done very well -- supporting both Go's garbage collection and multithreading).

Starlark is a small and simple language with a familiar and highly readable syntax. You can use it as an expressive notation for structured data, defining functions to eliminate repetition, or you can use it to add scripting capabilities to an existing application.

For me, my Python skills are just about on par with my JavaScript skills, but I still think I would prefer Python for the use cases above if I were a Python newbie. I only think JS would make sense if you also bring in all the (in)sanity of NPM.
Also, being able to define these plugin interfaces (as proper interfaces), we can also provide a set of implementations in Go which you then can configure from your plugin, e.g. (and again, I'm just quickly making this stuff up):
def SourcePlugin():
  # One of "many" supported adapters with implementation in Go.
  return "wordpress"

def PluginConfig():
  return site.Param("myWordPressConfig")

The above is, I think, valid Python. Most editors will provide syntax highlighting for it (if you suffix the file "*.py") and is, in my head, much easier to explain to people than some Go template conditional spaghetti.
I will also add that, if you think the above is hard and you still want/need to use it (people have happily lived without the above for a long time), Hugo Modules allows people to borrow from other people's work. This will be even more true if we extend this to writing plugins that gets exposed as template functions (see @natefinch 's PR).

This may also be an interesting option that I will have a look at:
https://github.com/d5/tengo

Most editors will provide syntax highlighting for it (if you suffix the file "*.py") and is, in my head, much easier to explain to people than some Go template conditional spaghetti.

yes, yes! Let Go Templates do the templating ... and other interfaces for the more ambiguous and likely-to-branch-and-to-error data generation.  Since Go is 1) a security risk and 2) rather esoteric then Python and JS seem like robust and highly used options (with Python being particularly well crafted and good for beginners).  That being said, there would be some aesthetic nicety of having an "all-GO-based" solution .

I’m convinced and cast my vote. :). Can’t wait to see some parser example. I guess I wanted an excuse to learn Go.

This might be a an example that Hugo users can more easily rely to:
https://github.com/bazelbuild/starlark/blob/master/README.md#tour
This is pretty exciting actually. Give so much control on a directory’s data, remote or otherwise

I guess I wanted an excuse to learn Go.

Still plenty of reasons to learn Go ...
For completeness, my previous answer about "Go as plugin" assumed some kind of "compile on the fly" and communication via "os/exec". Go has a "plugin package" built-in that could have worked for us, but it's Linux and macOS only and considered very experimental and buggy (and no-one have worked on it for years).

The options that I have evaluated are Python and Lua.

Lua is commonly used for such things, e.g. Pandoc uses it for custom filters, but considering that Starlark is implemented in Go and looks as 'python-lite', I believe it's a better option than full-fledged Lua  being easy and/or simple-enough.

To my knowledge there are no solid and embedded JS implementation in Go.

I won't cry because of that. :-)

Is this issue regarding pages from data still relevant? The discussion seems to have stalled.
Anyway, I used https://github.com/avillafiorita/jekyll-datapage_gen for Jekyll site in the past and maybe it could be used for inspiration for a Hugo equivalent?

I definitely think it's still relevant.
I wrote a couple of small scripts in golang that are doing the job for me, currently, but I would love to have this feature baked-in.

Just chiming in to say I have a number of cases where having a way for Hugo to consume data via json from an external provider or a collection of json files stored as data would be hugely beneficial. Currently having to write solutions that create and commit md to a repo or script the creation of hundreds of markdown files that reference a json data file are cumbersome and undesirable complexity. If I am following this thread (if) seems like having a way to define external or data driven content sources within the current content tree and if needed have a scripting language to define your transform sounds excellent. So would like to see something like this happen.

Just chiming in to say I have a number of cases where having a way for Hugo to consume data via json from an external provider or a collection of json files stored as data would be hugely beneficial.

100% agreed. We are developing a CMS for JSON files and as a consequence have done lots of research in that area. Most static page generators are severly lacking in that aspect. While some (including Hugo) can load data files into certain predefined pages, creating dynamic pages and/or dynamic navigations is mostly somewhere between cumbersome to impossible. So I think that having good support for arbitrary structured data sources could be beneficial for Hugo.
One other project that was introduced recently is Nuxt Content which is a plugin for Nuxt.js (a framework for server-side rendered or statically generated applications). It allows loading different types of files and create content dynamically based on them. For example, you can create dynamic pages/routes and/or create a dynamic navigation. We've done a simple test project with it and it seems to be well thought out and very flexible. Maybe there is some inspiration in that project for Hugo.

https://github.com/gohugoio/hugo/issues/5074#issuecomment-717564911

I can see this thread is fairly old now but I just wanted to ask, given I kinda share the same sentiment as this old comment:

I think the learning curve of Go Template is hard enough for many new users, letting them know that learning Python is a requisite for Data source sounds a bit harsh.

I'd be willing to invest time into learning some Go, but Python, not so excited. Did you drop JS because of speed?

is it not possible to use a scripting language that is at least very similar, syntactically, to Go?
I am not a security expert by any means but can the security concern/s not be mitigated somehow?
if a project like https://github.com/goplus/gossa  exists, what are they doing security-wise which is different?
...just for the record, I am in no way trying to discredit the starlight project, which is obviously great work!

I have a build script  in python that converts json entries to files. You can find it here: https://github.com/brunoamaral/gregory/blob/main/build.py#L136-L189
Hope it helps.

As the end of year comes near, I'd like to reflect a bit on this popular ticket.
Having dealt this year with a lot of pre-build (write your own files) workarounds, I feel like this could go into an early stage using Go Template and returning partials to feed Hugo a simple list of stings or maps.
We know Hugo is already building pages without content files for taxonomy terms. And all it can work with is a string (Museum, Food And Entertainment etc...) and it's pretty good at building a URL from that and publishing the page. I personally have be using this to build page from data with Hugo (or remote, it eases up the pre-build work: only two files to write instead hundreds of md).
What would already be a big leap to achieve the goals this ticket is aiming for is to give Hugo a simple list of entries to build pages for a given section. (akin to a simple list of taxonomy terms)
This can be as simple as a returning partial whose returned value could be fed to Hugo. It could come from a data file or a getJSON call.
A simple slice of strings (no data structuring)
It could even be a simple lists of strings (exactly like taxonomy terms).
{{/* content/products/_content.html */}}
{{ $products := slice }}
{{ with site.Data.products }}
  {{ range . }}
    {{ $products = append .Name }}
  {{ end }}
{{ end }}
{{ return $products }}

And, users could build up their own logic to fetch the data for each page using a data file or else. (Like when using the taxonomy hack)
{{/* layouts/products/single */}}}
{{ with index site.Data.products .Title }}
  <h1>{{ .Name }}</h1>
  price: {{ .TaxPrice }}
{{ end }}

As a slice of maps (data structuring is supported)
We could envision a slice of maps. This would potentially take care of the data structuring. It seems the only required key would be title (to generate the URL). All the rest could be parsed as is (reserved keys preserved, unreserved keys under would be held in the .Params object in templates).
That's how this returning partials could look like:
{{/* content/products/_content.html */}}
{{ $products := slice
{{ $remote_entries := site.Data.products }}
{{/* or $remote_entries := getJSON "https://myapi.com/products"  */}}

{{ with $remote_entries }}
  {{ range . }}
    {{ $products = $products | append (dict
      "title" .Name
      "price" .TaxPrice
    ) }}
  {{ end }}
{{ end }}
{{ return $products }}

If I'm not wrong and this is not such a big leap from the code base, this would make this early stage "building pages from data" easy enough that any body could jump in and test it. The more feedback the more robust the next stages will be.
I understand plugins could be used, and yeah maybe it would be great in the long term to support  heavy old dated structure like WordPress or Drupal. But do we need this from the get-go? I'm very excited about this prospect and I, personally, would love to jump into a world where Hugo can build pages from data, even if it is as limited as my suggestion can be.

@corinnefreer  https://www.thenewdynamic.com/article/toward-using-a-headless-cms-with-hugo-part-2-building-from-remote-api/

I've stumbled upon this problem yesterday and found this thread.  I'd like to propose an extension of the initial solution proposed by @bep that uses the content directory structure as a routing mechanism and a JSON query library file to filter the source data for each route.
For example, let's imagine we're building a movie database website and we want automatically generated pages for movies by title, by year, by genre, and by actor, all generated from a single JSON file (or external source) like this one.
The directory structure would appear as follows:
content
├── _index.md
├── movies
│   ├── query.toml
│   ├── _index.md
│      ├──year
│            ├── _index.md
│            ├── :id
│               ├── query.toml
│               ├── _index.md
│      ├──actor
│            ├── _index.md
│            ├── :id
│               ├── query.toml
│               ├── _index.md
...etc

The query.toml would include the data source and the JSON query to filter the data.  JSON queries could be executed using a Go library like jsonql, so a sample query file for the movies folder (which, for example, is there to lists all published movies) could look like this:
source = "movies.json"
query =  "published is true"

This setup would look at movies.json, find all entries where published is true, and then make that data available via .Page.Data variable in the template for looping purposes.
A special :id directory tells Hugo to build one page for each query result.  The query file in that directory must include the :id variable that denotes the key which will be used to build pages.  Looking at the year folder example, our intention here is to create one page for each year found in the JSON file, i.e:
/movies/year/1986
/movies/year/1999
/movies/year/2004
...etc.

This is why we would create an :id directory under the year folder, with following query.toml
source = "movies.json"
query =  "year = :id"

Hugo would recognize that year is the key we want to build pages around and execute accordingly.
This proposal is still rough around the edges but there are a few clear benefits:

Definining routes is typical for most frameworks and this would ease the adoption of this feature for newcomers
The data is separate from content and still gets to sit in the data directory, which allows for reusability via .Site.Data
We could support external queries via config files in the data directory
Using JSON query to filter the source data is extremely flexible and supports a variety of scenarios and use cases

Hope this helps.


Fix: incorrect translations

fix incorrect translations

@chensixian is this correction still valid ? let me know and I will merge it asap

@cyntss yes, it's valid.


Slack provider ignores username field

Running two clusters each with their own gotk deployments and notifying to the same Slack channel, I'd hoped to distinguish between the two by setting an appropriate username in the Provider resource's definition. Unfortunately, these aren't honoured by either the notification controller, or Slack itself.
My Slack bot has the following manifest:
_metadata:
  major_version: 1
  minor_version: 1
display_information:
  name: Flux v2 integration
  description: FLux v2 integration for displaying notifications
  background_color: "#da3a79"
features:
  bot_user:
    display_name: Flux
    always_online: true
oauth_config:
  scopes:
    bot:
      - channels:read
      - chat:write
      - chat:write.public
      - incoming-webhook
      - chat:write.customize
settings:
  org_deploy_enabled: false
  socket_mode_enabled: false
  token_rotation_enabled: false

The provider definition looks like this:
apiVersion: notification.toolkit.fluxcd.io/v1beta1
kind: Provider
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: <snipped>
    kustomize.toolkit.fluxcd.io/checksum: 868a205a9b3c23e6173d439e14835133da776b49
  creationTimestamp: "2021-07-07T15:11:10Z"
  generation: 4
  labels:
    kustomize.toolkit.fluxcd.io/name: flux-system
    kustomize.toolkit.fluxcd.io/namespace: flux-system
  name: slack
  namespace: flux-system
  resourceVersion: "75539776"
  selfLink: /apis/notification.toolkit.fluxcd.io/v1beta1/namespaces/flux-system/providers/slack
  uid: addb1946-3697-4cb2-9bd2-2519ae053945
spec:
  channel: devops-notification
  secretRef:
    name: slack-url
  type: slack
  username: Dev/Test Fluxbot

(and similar for prod, just a different username).
Sending to slack itself works fine, it's only issues with a custom username.

Ah, digging through the Slack documentation shows the following:

You cannot override the default channel (chosen by the user who installed your app), username, or icon when you're using Incoming Webhooks to post messages. Instead, these values will always inherit from the associated Slack app configuration.

(src: the end of this block
I suppose I'm out of luck until it's set up as a Slack App rather than a webhook, as with #245


Tweak implementation of throwaway HTTP sessions

What does this PR do?

Our RequestsWrapper has a persist_connection switch. When it's off (the default), we use a throw-away session for each request, instead of reusing a shared Session.
This PR changes this throw-away mechanism from using the top-level requests namespace to using the same session than we'd use when persisting connections. This is done via a ._create_session() method shared between each code path (persist / don't persist).
Motivation

I'm working on adding Unix Domain Socket support to our RequestsWrapper (marked as TODO in the docs, plus I'm going to need it for IoT Edge #7465).
Using requests-unixsocket this requires mounting an adapter onto a session, eg session.mount("unix://", UnixAdapter()), and so in the "don't persist" code path we'll need to create a dedicated session rather than the top-level requests library.
With this new shared ._create_session() logic, adding UDS support will basically amount to editing _create_session() so that it mounts the adapter. This will apply to both "persist" and "don't persist" use cases.
In any case, using the exact same logic in each code path seems like the best way forward!
Additional Notes

Review checklist (to be filled by reviewers)

[ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
[ ] PR title must be written as a CHANGELOG entry (see why)
[ ] Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
[ ] PR must have changelog/ and integration/ labels attached

Closing, we'll come back to this later.


Add ability to select gun classifications

The threshold of .7 was determined empirically for this dataset

Code logic LGTM. Can you push this to a live version of the app to review?

It's live, feel free to test. I think there is a heisenbug somewhere in there, haven't been able to reproduce.

Would be nice to add a bit of space (15px) below the search control to even out the spacing, otherwise LGTM!

Done!


release

Codecov Report

Merging #4 into develop will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           develop     #4   +/-   ##
======================================
  Coverage      100%   100%
======================================
  Files            6      6
  Lines           41     41
  Branches         7      7
======================================
  Hits            41     41

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 71aa36e...08fda5f. Read the comment docs.


The use of open-source LLM

Hi! Have you tried using open-source LLMs for your results? If so, would you be willing to share some instructions on deploying an open-source LLM, such as Llama3?

We did not use open-source LLMs in this paper. You can check  which try LLaMa 2 on the dataset.


Using artifact for design

I'm planning to give https://github.com/vitiral/artifact a try. 2.0.0 is just around the corner, and git version is building for me just fine.

I never really had a time...


Sync rhs/ceph:devel with ceph/ceph:devel

blkdiscard cmd discards all data on the block device which
is not desired. Hence, return unimplemented code if the
volume access mode is block.
Signed-off-by: Rakshith R<EMAIL_ADDRESS>/cc @agarwal-mudit @Madhu-1 @humblec @red-hat-storage/ceph-csi-admins

/cherry-pick release-4.10

/lgtm
/approve

lgtm.. thanks


Repo class refactoring [discussion]

Here are some ideas on repo refactoring and logic splitting. Imho. Discussion is welcomed :)

conceptually, the repo should become a container class, that is not aware how the remote data (and maybe local data too) is retrieved. Then it will be a simple class, lightweight and testable. And it should be one class, screw the sub-classing. related to #2216
repo settings (meta data) - a separate class, convenient getters/setters, decoupled from persistence (for testability)
state calculation - a separate class. in particular this would allow to manage fetching and serialization more independently. E.g. to run remote fetch and serialization in parallel, or being able to view "just local" changes without accessing remote (e.g. to quickly check if everything is committed without entering credentials, or to check WIP diffs) - ref #2664, #3008
remote access strategy (connectors) - zip or http. Something I proposed in #2216.

Problem ! Strategies will have common functions (for repo) and may have specific functions (e.g. listing branches). How to access such functions having a repo instance ? This is not completely clear. Option is to repo->get_remote_accessor( ) or duplicating some of function in repo (bad imho) or unifying the features of zip and http (in fact could be OK, because obviously HTTP accessory will be mostly used and ZIP can raise exceptions on unsupported features).

alternatively, the access strategies could be kind of "factories" to fill internal repo attributes. Not sure which is better, but I have a feeling that this approach could be more flexible and transparent:

* accessor
lo_repo->get_remote( )
* ---
method zcl_repo~get_remote.
  if ii_remote_connector is bound.
    rt_data = ii_remote_connector->fetch( ).

* factory
data lo_fetcher type zcl_http_remote.
new lo_fetcher( )->fetch( lo_repo ).
*---
method zcl_http_remote~fetch.
  io_repo->mt_remote = me->cool_private_http_fetcher( io_repo->get_url( ) ).

maybe local serializer is a separate class too. For testability.
deserializer? again for testability. And features like deployment (#3705)
logger, maybe it should be assigned to repo rather than created inside. Or maybe the log should be just be mergeable with another log.

any low hanging issues which can be started as first steps?

Low hanging won't get us far. I'm all for refactoring, but something major like this should go into an AG 2.0 branch.

@larshp low hanging - not sure.

Certainly I'd create a package for all repo stuff. And a an interface with miiinimal set of features, e.g. get_key, get_name, is_offline ... as a wireframe, it is not clear what should be there at this point
Then either try to rearrange the status method (remove it from repo - it is already a separate class in fact) or try to decouple settings (repo_meta)
then revive #2221 in some way

cover all with at least basis UTs on each step ...
I also have a feeling that it will need "2nd-pass"


Allocation bug in sv_* when running out of memory

How to reproduce:

System with low memory without swap enabled (e.g. 128, 256, or 512MB RAM system)
Run the "counter" example with "4 0" input parameters, e.g. ./counter 4 0 < input

It crashes because accessing a location that should be allocated, but it is not (according to the allocation logic, it should, but it is not; it looks like a subtle bug). I don't think is going to be the case, but can not be discarded being a bug in the C library (bug found in one 32-bit i686 Linux system with eglibc 2.5 and 512MB of RAM -280MB free-); doing tests in more systems should discard this.

Tests on other platforms:
Linux MIPS 32-bit eglibc 2.13 512MB RAM: wrong behavior
Linux ARMv6 eglibc 2.13 512MB RAM: wrong behavior
Linux x86_64 glibc 2.21 (with max alloc limited to 200MB) : OK
That's not conclusive, but a bug in eglibc can not be discarded.
(related hash: 4c356fbdc72488faba6650aea431f26e4922ecc0)

More tests:
Linux MIPS64 eglibc 2.13 512MB RAM: OK
Thoughts on this:

Due to working properly in both x86-64 and MIPS64, but not on 32-bit platforms (i686, ARMv6, MIPS32), it could be related to a problem of realloc() exhausting virtual address space.
In order to be sure that is really a C library problem and not some bug in the application code, I'll try to repeat the tests using a debugger.
Tests on Win32 could also help on this.

The bug was caused by an arithmetic overflow on size_t type on 32-bit targets, that's the reason of not happening on 64 bit. So my previous "realloc bug" on 32 bit targets hypothesis was wrong.
(bug fix hash: 59baefe07022efea4c4147e84fea28d6e2651b71)


remove common-grpc logic that could be pulled in to gax

This is a follow up to @alexander-fenster's comments:
https://github.com/googleapis/nodejs-bigtable/pull/554/files#r333816049, and
https://github.com/googleapis/nodejs-bigtable/pull/554/files#r333816370
On #554, I suggested that we pull in logic identical to @google-cloud/common-grpc, but work towards refactoring this out in future PRs (perhaps pulling more of the logic into gax).

@kolea2 Possibly a good clean-up project to prioritize in 2022?

The entire decorateStatus.ts file seems to be gone so I am guessing this was cleaned up long ago.


Check Lighthouse test

Lighthouse test is always failing because of performance (due to the map i suppose).
Could you take a look at this test? @janosh

Very busy atm, sorry. Can maybe take a look the weekend after next.

Related issue #54.


EncryptionEventListener not initialised if mongodb.encrypt.silent-decryption-failures is not configured

Our application is configured with encryption type reflection, and silent-encryption-failures left to the default (false).
mongodb.encrypt:
  type: reflection
  keys:
    - version: 1
      key: ...

After upgrading to 2.7.0, I get the following exception:
No converter found capable of converting from type [org.bson.types.Binary] to type [java.lang.String]
org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [org.bson.types.Binary] to type [java.lang.String]

Debug output shows in the condition evaluation report under negative matches:
   EncryptAutoConfiguration#encryptionEventListener:
      Did not match:
         - @ConditionalOnProperty (mongodb.encrypt.[type,silent-decryption-failures]) did not find property 'silent-decryption-failures' (OnPropertyCondition)

The culprit seems to be this line in the auto configuration:
https://github.com/bolcom/spring-data-mongodb-encrypt/blob/44b9f247191b400316eb8eff13d21189c83f7602/src/main/java/com/bol/config/EncryptAutoConfiguration.java#L30
Apparently @ConditionalOnProperty (with no matchIfMissing) only evaluates to true if both properties type and silent-encryption-failures are explicitly configured. I guess that the intention was that one of the properties need to be set.

This also occurs if mongodb.encrypt.silent-decryption-failures is explicitly set to false.

 The idea was that autoconfiguration should only happen if there are any properties set under mongodb.encrypt, but there is just no such option that I could find.
Documentation of @ConditionalOnProperty is really scarce regarding if list elements are considered in and AND or an OR relationship. And in system test, both are set for testing purposes.
Let me check if I can find a solution to this. Until then, a workaround is to explicitly set mongodb.encrypt.[type,silent-decryption-failures].

 The idea was that autoconfiguration should only happen if there are any properties set under mongodb.encrypt, but there is just no such option that I could find.
Documentation of @ConditionalOnProperty is really scarce regarding if list elements are considered in and AND or an OR relationship.

I wouldn't habe know that behaviour either. I think Spring's own code just uses custom condition implementations (mostly as local inner classes in the autoconfiguration class) in such cases.

Correct, that's what I noticed in other projects too. But I really want to keep this library simple and short, so custom wiring annotations falls well outside the scope.
Now I've added a simple change, along with a system test to prove it working. Take a peek if interested.
Also released 2.7.1 with this change.


Can't set a value of "" on goa-dropdown

Replication Steps:

You can replicate by visiting this site, Stackblitz
Select any number of options you want on whichever dropdown pleases you
Hit the reset button
The goa-dropdown with native property and the default HTML dropdown element both select the "" value. But the normal goa-dropdown element does not.

Replication Steps # 2:

Visit this site, Stackblitz
The difference here is that there is no "" value in the Dropdown values
Here the reset button works differently for all three versions of Dropdown. It does nothing for the default goa-dropdown, it resets to the first value in the native goa-dropdown, and it blanks out the selection for the default HTML dropdown.

Acceptance criteria:

You should be able to select any value that is set to the dropdown, even if that value is blank
You should be able to blank out the selection in a Dropdown, just like you can in a default HTML dropdown

Closed as completed


Add Time Preferences

Users can edit observation time in GMT or Local time.
Users can view observation and location time as relative or absolute local or GMT.

Pull Request Test Coverage Report for Build 299

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 44.386%

Totals

Change from base Build 296:
0.0%

Covered Lines:
2565

Relevant Lines:
5456

💛  - Coveralls


Renamed Files List not writing in the Output logs folder

Hi Team,
I found renamed list got gathered logically but not writing the renamedFileList.txt into the output log folder.
Any plan to implement this next releases. Please take this up as we are in need to get this list and process them in our deployment process.
Thanks :-)

Hi @kiran-esh2612 ,
zAppBuild manages the changed, renamed and deleted files. Indeed, deleted files are currently written to the output dir.
Do you plan on any post-build processing for renamed files ? If so, can you please elaborate on the use case a bit?

We need list of renames files so that during deployment we can remove those components from our loadlibs, source libs etc and will avoid referring of renamed code in any case and will maintain our libs with out any unnecessary components.

Hi Kiran,
There is a larger enhancement that we are planning to contribute, which is about documenting the deletions (and now also renaming) as DBB Build Report records, to be able to process the deletions through UCD. Basically, these are deleted source files with their assumed generated outputs.
You can have a preview here.
https://github.com/dennis-behm/dbb-zappbuild/compare/main...dennis-behm:documentDeletions
Would that address your scenario?

I'm glad to hear about the larger enhancement.
Meanwhile we have taken care of writing renameFileList.txt into output log folder.
Thanks :-)


API extension via plugin or module

I want to essentially make a plugin to manage my container's network and also to provide cloud like service discovery.
Does LXD have an API for extending it with plugins, or is it worth a PR?

One thing we've though about in the past and may find acceptable is #3391

"What exactly are you trying to do which can't be done by an API client of our existing API?"
Automatically handle NAT and automatically give IP's for each container started, without modifying iptables manually.

Is there a reason why you can't route your public subnet to the LXD bridge (see ipv4.routes config key) and then feed the containers a static IP through cloud-init data?
That's what most of our users seem to be doing when providing public addresses to their containers (usually on hosting provides like OVH or Hetzner).

In any case, as configuring iptables can happen even after the container has started, it sounds like the lifecycle events we push out right now would be sufficient for such a tool.

Can you provide an example of how to make this work?
I'm fairly new to LXD.

@RevoluPowered https://github.com/lxc/lxd/blob/master/lxc/monitor.go#L96 has an example of a events API client.
That handler is called everytime a relevant message is sent. You can look at messages by running lxc monitor.
With code similar to that above, you can filter for those events you care about (type=lifecycle, action=container-started) like this one:
metadata:
  action: container-started
  source: /1.0/containers/centos4
timestamp: "2018-05-30T13:29:03.323275991-04:00"
type: lifecycle

And then spawn your iptables subcommand, possibly fetching more information about the container through GetContainer and GetContainerState.
https://godoc.org/github.com/lxc/lxd/client has more details to get you started with our Go API.
If not using Go for this, then you'll need a websocket capable HTTP client to hit /1.0/events on the REST API, you'll then receive one message per event with the same format as shown above.

What I'll do is start throwing something together this weekend and see where I get.
I'm probably going to put the code in the game server management software, I can use C# with RestSharp to access your HTTP web end point.
Only question left is, how is auth handled, I think I've read it's password protected?
Thanks for your help @stgraber


Write generalized unit tests in TypeScript for all our SDK service calls

@lucasamonrc can you elaborate on this?

@lucasamonrc can you elaborate on this?

I think @michaeldboyd will do a better job elaborating actually.

Another call for elaboration please? @michaeldboyd
Otherwise, we'll have to close this issue.

This issue is actually related to issue #408. @michaeldboyd recommended that it would be useful to write generalized tests for our node sdk in typescript that then could be referenced as snippets for our v2 reference docs.
Did I understand it correctly @michaeldboyd?

We currently have a unit test for the vaccination demos in each language that we import. We don't have any unit tests that to demonstrate reference SDK calls on our SDK Services reference page. I don't think we need unit tests covering all languages at this point, but having just one will be helpful for developers to get an idea on what SDK calls look like when they're reviewing docs for the first time. Having comprehensive coverage of all our sdk calls in at least one language will also help us know if something changes.

We currently have a unit test for the vaccination demos in each language that we import. We don't have any unit tests that to demonstrate reference SDK calls on our SDK Services reference page. I don't think we need unit tests covering all languages at this point, but having just one will be helpful for developers to get an idea on what SDK calls look like when they're reviewing docs for the first time. Having comprehensive coverage of all our sdk calls in at least one language will also help us know if something changes.

That's part of some of the broader initiatives I'm doing around complete SDK test coverage to verify behavior in all languages. (No edge cases). I don't really unit test the SDKs, so much as integration test them, since unit-of-behavior is the full-stack call. If there are other developments on this, keep me in the loop.


Fix linter

This is an attempt to fix #42

Ok, now the "lint" step passes.
"molecule test -s default --destroy always" still fail, but the error regarding the missing role seems to be resolved.
The new error is:
ERROR! couldn't resolve module/action 'community.docker.docker_container'. This often indicates a misspelling, missing collection, or incorrect module path.
The issue may be explained here: https://github.com/ansible-community/molecule/issues/2755
Either we try to fix the molecule//collections.yml for 2.9 and 2.10. Or we could skip those versions to use ansible 3.1, 4.7 and 5.4 that are out for a while now.
But for people still on Ansible Automation Platform 1, the supported version is the 2.9...

Ok, now it works up utile this:
TASK [ansible-mysqld_exporter : Ensure mysqld_exporter is enabled on boot] *****
fatal: [bionic]: FAILED! => {"changed": false, "msg": "Unsupported parameters for (systemd) module: started Supported parameters include: daemon_reexec, daemon_reload, enabled, force, masked, name, no_block, scope, state, user"}
fatal: [buster]: FAILED! => {"changed": false, "msg": "Unsupported parameters for (systemd) module: started Supported parameters include: daemon_reexec, daemon_reload, enabled, force, masked, name, no_block, scope, state, user"}
fatal: [centos7]: FAILED! => {"changed": false, "msg": "Unsupported parameters for (systemd) module: started Supported parameters include: daemon_reexec, daemon_reload, enabled, force, masked, name, no_block, scope, state, user"}
fatal: [stretch]: FAILED! => {"changed": false, "msg": "Unsupported parameters for (systemd) module: started Supported parameters include: daemon_reexec, daemon_reload, enabled, force, masked, name, no_block, scope, state, user"}
fatal: [fedora]: FAILED! => {"changed": false, "msg": "Unsupported parameters for (systemd) module: started Supported parameters include: daemon_reexec, daemon_reload, enabled, force, masked, name, no_block, scope, state, user"}
fatal: [xenial]: FAILED! => {"changed": false, "msg": "Unsupported parameters for (systemd) module: started Supported parameters include: daemon_reexec, daemon_reload, enabled, force, masked, name, no_block, scope, state, user"}
But I may have  fixed that in PR #40

Mergify now says: The new Mergify configuration is valid
I learned a lot about CircleCI and Github, thanks for that :)
Is there somebody at cloudalchemy who can find the time to review the PR please? @paulfantom maybe?
Many thanks for your roles! They're the first ones I use due to their great quality.

We have a bunch of things to fix in our CI right now. The changes are mostly done in our skeleton/automated updates. But we need to get things pushed out to all of the other projects.
I'm planning to do a bit of a hackathon with @paulfantom sometime soon to get things cleaned up.

Thanks for the reply @SuperQ
I look forward to seeing the new CI setup then :)

Not stale

This role has been deprecated in favor of a the prometheus-community/ansible collection.


Reward Distribution Upon Quest Completion

Ticket: Reward Distribution Upon Quest Completion
Description: Develop functionality to automatically distribute the specified reward (NFT or token) to the user's wallet address upon quest completion. Ensure that rewards are distributed only once per user upon completion of the quest.
Acceptance Criteria:

The contract should automatically distribute the specified reward (NFT or token) to the user's wallet address upon quest completion.
Ensure that rewards are distributed only once per user upon completion of the quest.
Implement checks to prevent multiple distributions of the reward to the same user for the same quest.
Provide functions to verify reward distribution status.

Solidity Smart Contract:
// SPDX-License-Identifier: MIT

pragma solidity ^0.8.0;

import<EMAIL_ADDRESS>
import<EMAIL_ADDRESS>
import<EMAIL_ADDRESS>
contract EventAmbassadorQuest {

    using SafeERC20 for IERC20;

    address public admin;

    struct Quest {

        uint256 eventId;

        uint256 startDate;

        uint256 endDate;

        uint256 requiredInteractions;

        string rewardType; // "NFT" or "token"

        address rewardAddress;

        uint256 rewardTokenId; // Used if rewardType is NFT

        uint256 rewardAmount; // Used if rewardType is token

    }

    struct UserInteraction {

        bool isRegistered;

        uint256 interactionsCount;

        bool isCompleted;

        bool rewardDistributed;

    }

    mapping(uint256 => Quest) public quests;

    mapping(uint256 => mapping(address => UserInteraction)) public userInteractions;

    event QuestCreated(uint256 eventId, uint256 startDate, uint256 endDate, uint256 requiredInteractions, string rewardType);

    event UserRegistered(uint256 eventId, address user);

    event QuestCompleted(uint256 eventId, address user);

    event RewardDistributed(uint256 eventId, address user, string rewardType);

    modifier onlyAdmin() {

        require(msg.sender == admin, "Only admin can perform this action");

        _;

    }

    constructor() {

        admin = msg.sender;

    }

    function createQuest(

        uint256 _eventId,

        uint256 _startDate,

        uint256 _endDate,

        uint256 _requiredInteractions,

        string memory _rewardType,

        address _rewardAddress,

        uint256 _rewardTokenId,

        uint256 _rewardAmount

    ) public onlyAdmin {

        require(_requiredInteractions >= 3, "Required interactions must be at least three");

        require(bytes(_rewardType).length > 0, "Reward type must be specified");

        quests[_eventId] = Quest({

            eventId: _eventId,

            startDate: _startDate,

            endDate: _endDate,

            requiredInteractions: _requiredInteractions,

            rewardType: _rewardType,

            rewardAddress: _rewardAddress,

            rewardTokenId: _rewardTokenId,

            rewardAmount: _rewardAmount

        });

        emit QuestCreated(_eventId, _startDate, _endDate, _requiredInteractions, _rewardType);

    }

    function registerForQuest(uint256 _eventId) public {

        require(quests[_eventId].eventId != 0, "Quest does not exist");

        require(!userInteractions[_eventId][msg.sender].isRegistered, "User already registered");

        userInteractions[_eventId][msg.sender] = UserInteraction({

            isRegistered: true,

            interactionsCount: 0,

            isCompleted: false,

            rewardDistributed: false

        });

        emit UserRegistered(_eventId, msg.sender);

    }

    function submitInteraction(uint256 _eventId) public {

        require(userInteractions[_eventId][msg.sender].isRegistered, "User is not registered for this quest");

        require(!userInteractions[_eventId][msg.sender].isCompleted, "Quest already completed");

        userInteractions[_eventId][msg.sender].interactionsCount++;

        if (userInteractions[_eventId][msg.sender].interactionsCount >= quests[_eventId].requiredInteractions) {

            userInteractions[_eventId][msg.sender].isCompleted = true;

            emit QuestCompleted(_eventId, msg.sender);

            distributeReward(_eventId, msg.sender);

        }

    }

    function distributeReward(uint256 _eventId, address _user) internal {

        require(userInteractions[_eventId][_user].isCompleted, "Quest not completed");

        require(!userInteractions[_eventId][_user].rewardDistributed, "Reward already distributed");

        Quest memory quest = quests[_eventId];

        if (keccak256(bytes(quest.rewardType)) == keccak256(bytes("NFT"))) {

            IERC721(quest.rewardAddress).safeTransferFrom(admin, _user, quest.rewardTokenId);

        } else if (keccak256(bytes(quest.rewardType)) == keccak256(bytes("token"))) {

            IERC20(quest.rewardAddress).safeTransfer(_user, quest.rewardAmount);

        }

        userInteractions[_eventId][_user].rewardDistributed = true;

        emit RewardDistributed(_eventId, _user, quest.rewardType);

    }

    function isRewardDistributed(uint256 _eventId, address _user) public view returns (bool) {

        require(userInteractions[_eventId][_user].isRegistered, "User is not registered for this quest");

        return userInteractions[_eventId][_user].rewardDistributed;

    }

    function getUserInteractionCount(uint256 _eventId, address _user) public view returns (uint256) {

        require(userInteractions[_eventId][_user].isRegistered, "User is not registered for this quest");

        return userInteractions[_eventId][_user].interactionsCount;

    }

}

Acceptance Criteria Validation:

Reward Distribution Functionality:

The distributeReward function automatically distributes the reward (NFT or token) to the user's wallet address upon quest completion.
Ensures that rewards are distributed only once per user by checking and updating the rewardDistributed flag.

Tracking Reward Distribution:

The UserInteraction struct includes a rewardDistributed flag to indicate if the reward has been given to the user.
The isRewardDistributed function allows retrieval of the reward distribution status for any user and quest.

Security and Validation:

Ensures that rewards are only distributed to users who have completed the quest.
Validates that rewards are not distributed multiple times to the same user.

Events:

The RewardDistributed event is emitted upon reward distribution, providing transparency and tracking.

This contract component ensures that the specified reward is automatically distributed to users upon quest completion and that rewards are distributed only once per user.

points: 1, 2, 3, 5, 8, 13, 21
@russlive215
@Fr0z0n3
@noireconnect2024
@wvwatson
@tierra-d
points:


KeyError: 'input_1_ib-0' , memory allocation error

Hi there again!
I got an error when initiating the training, it gives me this output.

These are my codes :

I suspect this has to do something with tensorflow clashing with keras but I'm not sure. Thanks!

I saw your files date is 2 months ago, and I do have a commit about 1 months ago to fit TF 2.6.0. Maybe you are using TF 2.6.0 and didn't get this updated? The key change there is like:
- from tensorflow.python.keras import layers
+ from tensorflow.keras import layers

Thanks for the quick reply! Yeah, the update does fix my problem. Thanks again!!

Hello again! Sorry but I have another issue (and I don't want to open a new thread because might be too many threads) , regarding memory exhaustion. The codes are same as above and here is the details of total images and classes

I'm using my own dataset (around 15k images + CASIA webface) for training and the face_emore as evaluation. My server details are

And I got the error message "tensorflow.python.framework.errors_impl.ResourceExhaustedError: failed to allocate memory [Op:Mul]".
I tried to lower the batch size until 16 and include mixed_precision, but I still get the memory throwback. Do you have any suggestion on what I can do?

In the log, you got 6573531 classes... By default it's using person folder names as classes at data.py#10:
default_image_classes_rule = lambda path: int(os.path.basename(os.path.dirname(path)))

The largest folder name will by used as total classes. If your folder name is not numerical increasing class names, may try change data.py#48:
- image_classes_rule = default_image_classes_rule
+ image_classes_rule = ImageClassesRule_map(data_path)

OK, I just updated using ImageClassesRule_map as default behavior... You can remove the backuped cache [data_path]_shuffle.npz and try again.

Thanks! I pulled the latest code and it works now! The classes number has changed

The machine still gives memory throwback but i fixed it buy lowering the batch size to 128
But I'm a bit worried since the classes should be around 15k based on the aligned_112 dataset

So I'll have to check it out why the number isn't the same.
The reason why I choose Resnet34 is due one of the face verification framework that I'm using  (https://github.com/serengil/deepface) uses your keras implementation and I checked it was resnet34 as base model.
Thank you again sir for your help and I'm making sure to include credits for this repo!

Hello. I'm just wondering if png image format is applicable for training. I'm having concern due to custom data (4k person) that I added to CASIA webface is likely not included since read total class was 10575, which should be 15k. Thanks.

Hello. I'm just wondering if png image format is applicable for training. I'm having concern due to custom data (4k person) that I added to CASIA webface is likely not included since read total class was 10575, which should be 15k. Thanks.

You can refer to: data.py line 9 for more details

As currently it's using ImageClassesRule_map as default rule, not concerning it's png / jpg. So technically total classes should be same with:
import glob2
data_path = /datasets/faces_casia_112x112_folders/'
print(len(glob2.glob(os.path.join(data_path, '*'))))
# 10572

data_path = /datasets/ms1m-retinaface-t1_112x112_folders/'
print(len(glob2.glob(os.path.join(data_path, '*'))))
# 93431

For CASIA dataset, it's 10572, just 3 from yours. Not sure why skipped some in your dataset. Can you check it with glob?
And @whalefa1I , I just changed the default rule, thus data.py L9 is not using now. :)

Sorry, png is excluded, and data.py L9 still works..
png should also work, you may try changing data.py L9:
- default_image_names_reg = os.path.join("*", "*.jpg")
+ default_image_names_reg = os.path.join("*", "*")

Sorry, just excited off work, I'll do some tests...

noted with thanks! Please take your time.

Just updated default image_names_reg rule, you may try again. In my tests, data.py#L60 tf.image.decode_jpeg also works with png images, but tf.image.decode_image throws some error, so I'm keeping tf.image.decode_jpeg there.

Just updated default image_names_reg rule, you may try again. In my tests, data.py#L60 tf.image.decode_jpeg also works with png images, but tf.image.decode_image throws some error, so I'm keeping tf.image.decode_jpeg there.

Sometime it throws error since the size of image is 0

Noted with thanks!


[REEF-223]: Add a project reference to the JAR

This addressed the issue by
Adding a project reference to JAR project.
JIRA: REEF-223
This closes #

@beysims I am done with my pass.


Pirate shuttle update

About the PR
Pirates as a mode isn't really implemented properly right now but on the off chance an admin turns it on or spawns the shuttle for other uses it needs a bit of updating. this is not intended to actually refactor pirates, that's out of scope. Just a QoL update of the shuttle. Let me know of any more changes to include.
Media

[x] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog

added tiny fans under the docks
changed access to some airlocks
added an iff computer
added a fax machine and a few handheld radios
replaced the oxygen tanks with a tank dispenser for our slime friends
replaced the toolboxes with a youtool
brought the grav gen inside

:cl: checkraze

tweak: minor update to the pirate shuttle

I finally got a few more responses from admins in game about it. Just a few more changes to it tomorrow and I'll mark as ready for review


For segment metadata push, don't copy the segment tarball when it's already local

For segment Metadata push, we don't copy the tarball if it's already on local disk.

Codecov Report

Merging #10033 (95854fa) into master (3fb2118) will decrease coverage by 50.67%.
The diff coverage is 0.00%.

@@              Coverage Diff              @@
##             master   #10033       +/-   ##
=============================================
- Coverage     64.28%   13.61%   -50.68%
+ Complexity     5670      176     -5494
=============================================
  Files          1939     1939
  Lines        105042   105044        +2
  Branches      16045    16048        +3
=============================================
- Hits          67531    14300    -53231
- Misses        32618    89619    +57001
+ Partials       4893     1125     -3768

Flag
Coverage Δ

unittests1
?

unittests2
13.61% <0.00%> (+0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

 Learn more


[New game]: Guess the valorant agent

🎮 Game Request
Game logic and basic description
there would be a valorant agent on the left and 80 % of the image would be hidden we have to guess the agent
Point down the features
Game points
1st guess:50 points
2nd guess:30 points
3rd guess:10 points
Select program in which you are contributing
GSSoC24
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

Hey @manasdeshpande !
Thank you for raising an issue 💗
You can self assign the issue by commenting /assign in comment 😀
Make sure you follow CODE OF CONDUCT and CONTRIBUTING GUIDELINES 🚀
Don’t Forget to ⭐ our GameZone🎮
Make sure you join our Discord🕹

/assign


Unable to Arm any systems

Hi Colin - First up... Fantastic work. Your plugin has been a great help since IFTT and Life360 stopped supporting location triggers.
Unfortunately, something major has gone wrong with my setup.
I've got 4 separate blink systems running in the Homebridge with the homebridge-blink-for-home plugin.
All was working until recently, I realised systems were not arming when leaving the house.
It seems I'm getting this error in Homebridge - that's causing arm to fail.
[12/8/2020, 7:40:56 PM] [blink-xxxx] Error [ERR_HTTP2_STREAM_ERROR]: Stream closed with error code NGHTTP2_ENHANCE_YOUR_CALM
    at ClientHttp2Stream._destroy (internal/http2/core.js:2115:13)
    at ClientHttp2Stream.destroy (internal/streams/destroy.js:38:8)
    at ClientHttp2Stream.[kMaybeDestroy] (internal/http2/core.js:2131:12)
    at Http2Stream.onStreamClose (internal/http2/core.js:513:26) {
  code: 'ERR_HTTP2_STREAM_ERROR'
}
[12/8/2020, 7:40:56 PM] [blink-xxxx] Error [ERR_HTTP2_STREAM_ERROR]: Stream closed with error code NGHTTP2_ENHANCE_YOUR_CALM
    at ClientHttp2Stream._destroy (internal/http2/core.js:2115:13)
    at ClientHttp2Stream.destroy (internal/streams/destroy.js:38:8)
    at ClientHttp2Stream.[kMaybeDestroy] (internal/http2/core.js:2131:12)
    at Http2Stream.onStreamClose (internal/http2/core.js:513:26) {
  code: 'ERR_HTTP2_STREAM_ERROR'
}
[12/8/2020, 7:41:01 PM] [blink-xxxx] Error: getaddrinfo ENOTFOUND rest-prde.immedia-semi.comundefined.jpg
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26) {
  errno: 'ENOTFOUND',
  code: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'rest-prde.immedia-semi.comundefined.jpg'
}
(node:2846) UnhandledPromiseRejectionWarning: Error: getaddrinfo ENOTFOUND rest-prde.immedia-semi.comundefined.jpg
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26)
(node:2846) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 7)
(node:2846) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
[12/8/2020, 7:43:07 PM] [blink-xxxx] Error [ERR_HTTP2_STREAM_ERROR]: Stream closed with error code NGHTTP2_ENHANCE_YOUR_CALM
    at ClientHttp2Stream._destroy (internal/http2/core.js:2115:13)
    at ClientHttp2Stream.destroy (internal/streams/destroy.js:38:8)
    at ClientHttp2Stream.[kMaybeDestroy] (internal/http2/core.js:2131:12)
    at Http2Stream.onStreamClose (internal/http2/core.js:513:26) {
  code: 'ERR_HTTP2_STREAM_ERROR'
}
[12/8/2020, 7:43:07 PM] [blink-xxxx] Error [ERR_HTTP2_STREAM_ERROR]: Stream closed with error code NGHTTP2_ENHANCE_YOUR_CALM
    at ClientHttp2Stream._destroy (internal/http2/core.js:2115:13)
    at ClientHttp2Stream.destroy (internal/streams/destroy.js:38:8)
    at ClientHttp2Stream.[kMaybeDestroy] (internal/http2/core.js:2131:12)
    at Http2Stream.onStreamClose (internal/http2/core.js:513:26) {
  code: 'ERR_HTTP2_STREAM_ERROR'
}
[12/8/2020, 7:43:19 PM] [blink-xxxx] Error: getaddrinfo ENOTFOUND rest-prde.immedia-semi.comundefined.jpg
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26) {
  errno: 'ENOTFOUND',
  code: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'rest-prde.immedia-semi.comundefined.jpg'
}
(node:2846) UnhandledPromiseRejectionWarning: Error: getaddrinfo ENOTFOUND rest-prde.immedia-semi.comundefined.jpg
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26)
(node:2846) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 14)

Versions are:
Raspbian GNU/Linux Buster (10)
Homebridge: 1.2.4 (although the issue was originally happening on 1.2.3 - I've since updated to see if that cured it)
homebridge-config-ui-x v4.36.0
homebridge-blink-for-home v3.6.2

Oh, I just opened another case - #34 - with these same errors. I didn't realize this impacted arming, but I just tested and it does appear arming doesn't work. I assumed something is wrong with the live view function since  a) I have liveview disabled in the plugin and b) no one is trying to view these cams when the errors occur.
FWIW I also did find out that this plugin does not work on the Homebridge 1.3 Beta. I'm running 1.2.4 as well.

sorry for long comms silence. can you verify that in v3.7 that these issues have been addressed?

@colinbendell - tried v3.7.1 and am hitting this issue: https://github.com/colinbendell/homebridge-blink-for-home/issues/56#issuecomment-783626996


Libvirt provider: Molecule destroy does not work well when provider parameters are wrong

Molecule destroy does not work well when libvirt provider parameters are wrong or not supported by the machine.
For instance, when I specify "graphics_type: gtk", i get the following error:

Error while creating domain: Error saving the server: Call to virDomainDefineXML failed: unsupported configuration: unknown graphics device type 'gtk'

Then i execute molecule destroy and nothing happens. If I try to execute molecule again(molecule test, or molecule create), I get the following error:

Volume for domain is already created. Please run 'vagrant destroy' first.

So while using libvirt and setting wrong or unsupported parameters, I have to remove the volume manually. I do not know whether this happens while using vagrant.
I have observed this section in comands.py:
class Create(AbstractCommand):
    def execute(self):
        """
        Creates all instances defined in molecule.yml.
        Creates all template files used by molecule, vagrant, ansible-playbook.

        :return: None
        """
        if self.static:
            self.disabled('create')

        self.molecule._create_templates()
        try:
            self.molecule._provisioner.up(no_provision=True)
            self.molecule._state['created'] = True
            self.molecule._write_state_file()
        except CalledProcessError as e:
            print('ERROR: {}'.format(e))
            sys.exit(e.returncode)

I would suggest to introduce a vagrant destroy call inside the except block.

@oserde Is this still an issue with the current libvirt work that was merged in #119?

Closing.  Please re-open if still an issue.


Rename "wallets" to "keys" in embedded wallet

The Chore

Tasks

[ ] Change tab name "Wallets" to "Keys"
[ ] Change header from "Wallet" to "My Keys"

Additional details / background info

Done


open_read_retry_time not being honored in ATS 9.1.3

While using the default configuration regarding Open Read Retry Timeout (and RWW disabled):
$ traffic_ctl config match open_read
proxy.config.http.cache.max_open_read_retries: -1
proxy.config.http.cache.open_read_retry_time: 10

the expected behavior according the documentation available in https://docs.trafficserver.apache.org/admin-guide/configuration/cache-basics.en.html#open-read-retry-timeout:

The configurations are (with defaults):

CONFIG proxy.config.http.cache.max_open_read_retries INT -1
CONFIG proxy.config.http.cache.open_read_retry_time INT 10
The defaults are such that the feature is disabled and every connection is allowed to go to origin without artificial delay. When enabled, you will try max_open_read_retries times, each with an open_read_retry_time timeout.

But we're occasionally (p999) seeing that the cache read time goes up to 1 second:

same behavior happens with the following values:
CONFIG proxy.config.http.cache.max_open_read_retries INT 1
CONFIG proxy.config.http.cache.open_read_retry_time INT 10

Cache read time is being measured via logging with the following expression: CacheReadTime:%<{TS_MILESTONE_CACHE_OPEN_READ_END-TS_MILESTONE_CACHE_OPEN_READ_BEGIN}msdms>
We've also noticed that requests with a high cache read time happen every 9 minutes (reported by cqtt even if the duration of the connection doesn't match between them (reported by ttms or TS_MILESTONE_SM_FINISH-TS_MILESTONE_SM_START):
$ cut -f1-2 -d' ' readtime.log |sort |uniq -c
    215 Date:2022-10-04 Time:09:24:52
     65 Date:2022-10-04 Time:09:33:51
    144 Date:2022-10-04 Time:09:33:52
     66 Date:2022-10-04 Time:09:42:51
    147 Date:2022-10-04 Time:09:42:52
    163 Date:2022-10-04 Time:09:51:51
     41 Date:2022-10-04 Time:09:51:52
    169 Date:2022-10-04 Time:10:00:51
$ cut -f1-2,7,9 -d' ' readtime.log |head -10
Date:2022-10-04 Time:09:24:52 CacheReadTime:712 TotalSMTime:712
Date:2022-10-04 Time:09:24:52 CacheReadTime:420 TotalSMTime:421
Date:2022-10-04 Time:09:24:52 CacheReadTime:176 TotalSMTime:177
Date:2022-10-04 Time:09:24:52 CacheReadTime:545 TotalSMTime:545
Date:2022-10-04 Time:09:24:52 CacheReadTime:513 TotalSMTime:513
Date:2022-10-04 Time:09:24:52 CacheReadTime:727 TotalSMTime:727
Date:2022-10-04 Time:09:24:52 CacheReadTime:460 TotalSMTime:461
Date:2022-10-04 Time:09:24:52 CacheReadTime:460 TotalSMTime:460
Date:2022-10-04 Time:09:24:52 CacheReadTime:686 TotalSMTime:687
Date:2022-10-04 Time:09:24:52 CacheReadTime:111 TotalSMTime:111

This could be potentially related to #8598

As pointed out by @zwoop on Slack, the 9 minutes periodicity is related to a cache_dir_sync operation. The undocumented configuration setting proxy.config.cache.dir.sync_frequency is set to its default value of 60 seconds but this cannot be honored because the operation takes 8 minutes so effectively a cache_dir_sync happens every 9 minutes as its being shown by debug logging:
[Oct  5 10:09:15.541] [ET_NET 11] DEBUG: <CacheDir.cc:1099 (mainEvent)> (cache_dir_sync) sync started
[Oct  5 10:09:15.541] [ET_NET 11] DEBUG: <CacheDir.cc:1111 (mainEvent)> (cache_dir_sync) Dir disk.0 294912:195351687: waiting for agg buffer
[Oct  5 10:09:15.542] [ET_AIO 5:166] DEBUG: <CacheDir.cc:1099 (mainEvent)> (cache_dir_sync) sync started
[Oct  5 10:09:15.542] [ET_AIO 5:166] DEBUG: <CacheDir.cc:1118 (mainEvent)> (cache_dir_sync) pos:<PHONE_NUMBER>72 Dir disk.0 294912:195351687 dirty...syncing to disk
[Oct  5 10:17:15.702] [ET_NET 11] DEBUG: <CacheDir.cc:1054 (mainEvent)> (cache_dir_sync) sync done
[Oct  5 10:18:15.728] [ET_NET 11] DEBUG: <CacheDir.cc:1099 (mainEvent)> (cache_dir_sync) sync started

We're using Dell Ent NVMe AGN MU AIC 1.6TB as disk storage and 8 threads per disk, any suggestions on how this could be mitigated?

@jvgutierrez Sounds like this issue is the same as #9124.  Can we close this issue and track it in #9124.

Sure! let's focus on #9124


Upload codecov for migration tests

Description
This makes a couple changes to our codecov config:

Make sure we are uploading codecov reports for our migration tests.
Switch to using exclude_also a new config option for coverage that preserves the built in exclusions https://coverage.readthedocs.io/en/7.2.2/config.html#report-exclude-also
Add a couple more exclusions:

...
pass
raise NotImplementedError

Motivation and Context
@RishiDiwanTT made me realize that we aren't currently uploading codecov reports for migration tests in this comment:
https://github.com/ThePalaceProject/circulation/pull/1189#discussion_r1227636460
This should make the codecov for #1189 look better once its merged @RishiDiwanTT.
How Has This Been Tested?
I did a bit of testing locally to make sure the regex exclusions seemed like they were working.
Checklist

[X] I have updated the documentation accordingly.
[X] All new and existing tests passed.

Closing this for same reason as https://github.com/ThePalaceProject/circulation/pull/1197. I see the issue now. Will make another new pr to test 😓.


Allow adding instructions dynamically using Java SPI

Depends on #106
WILL cause merge conflicts with #107
GitHub seems to be confused about the changes made to the *InstructionStore classes.
The only change for all of them is:

change NamedInstructionStore from an abstract class to an interface
replace addInstruction calls with Arrays.asList
The Instructions themselves did not change.

What this does
This change allows users to add named instructions without compiling aya itself.
Adding named instructions is done by placing libraries in <ayaDir>/libs/.
Here is an example project that registers 2 additional NamedInstructionStores.
AyaSpiExample.zip
(simply run mvn package and place the jar in the libs directory)
The added instructions behave like any other instruction:

Instruction Name Conflicts
Conflicts (instructions with the same name) are currently handled by using the last registered instruction.
So libraries can override builtin instructions (not sure if that's a good thing)
Aya will print a warning like this:
NamedInstruction 'json.loads' has multiple implementations:
  aya.ext.json.LoadJSONInstruction
  example.spi.ExampleInstructionStore$1

Future TODO
Publishing Aya to a Maven repository
Currently, to reference Aya in a libraries' pom.xml (like this:)
<dependency>
    <groupId>co.npaul.aya</groupId>
    <artifactId>aya</artifactId>
    <version>4.0.0</version>
</dependency>

you have to install aya to the local repository by running mvn install.
Ideally, releases would be available from a public repository.
For example: the central repository
Dependency Management
If Aya projects start depending on libraries, this could cause some frustrating Syntax Error: Named instruction :{...} does not exist errors.
There should be some way for Aya projects to declare library dependencies.
E.g. by adding String getLibraryName() and ComparableVersion getVersion() to the NamedInstructionStore interface.
Then projects could reference dependencies and possibly version ranges similar to this

Alternative Solutions / Extensions
It would also be possible to load .jar libraries at runtime.
E.g. "/path/to/lib.jar" :{library.load}
which could be implemented just like this
That might already solve the 'Dependency Management' issues, since you'd get a more descriptive "file not found" error.

Merged with modifications from this PR: #125