Concrete go + hugo modules conflict proposal: Add modfile property also for hugo

I'm happy to try to work on this as a PR, but want first your opinion, especially @bep.
As stated in #6115 both go and hugo think they're the owner of the go.mod file. As commented in https://github.com/gohugoio/hugo/issues/6115#issuecomment-521823167, one using hugo should follow its spec (This is not the place to discuss this decision). Learned how hugo mod works and knowing how go mod works, this issue is very possibly solvable.
Since go1.14 they added the -modfile property allowing you to decide which go.mod file to use. How about hugo modules config get a property with the exact same meaning and propagates that to go commands and further hugo module specific functions?
# config.yaml
# ...
module:
  modfile: hugo.mod # default: go.mod

Result: The developer can opt in to use a separate go mod file for hugo and thus keeping the default go.mod for his other purpose.

Since go1.14 they added the -modfile property allowing you to decide which go.mod file to use.

So, this is a great idea, but are you sure there is such thing as a universal -modfile property in Go?
A quick scan of the Go source code shows lots of hardcoded "go.mod" strings:
~/d/g/m/g/src ❯❯❯ find . -name "*.go" | grep -v "test" | xargs grep "\"go.mod\"" | wc -l                       master
      45

I am not sure, but according to the release notes linked in https://github.com/gohugoio/hugo/issues/6115#issuecomment-571426244 it sounds to be their aim

-modfile=file is a new flag that instructs the go command to read (and possibly write) an alternate go.mod file instead of the one in the module root directory. A file named go.mod must still be present in order to determine the module root directory, but it is not accessed. When -modfile is specified, an alternate go.sum file is also used: its path is derived from the -modfile flag by trimming the .mod extension and appending .sum.

That explains why there are still (some) hard go.mod strings. Unsure about vendoring but in a quick glance hugo has also the custom _vendor folder (but don't know full context yet). Also that option should only be used if such "main" go.mod exists.
I'll have a check in the coming days and crosscheck with the mod client in hugo. But it sounds like if there is a way it's worth a try. And if we/I find a need for something, we can voice our use case in the upstream issue.

That explains why there are still (some) hard go.mod strings. Unsure about vendoring but in a quick glance hugo has also the custom _vendor folder (but don't know full context yet).

Vendoring should be fine, I guess.

Also,
We need to consider dependencies in all of this, so I don't think adding this as a config option would make much sense, but if we could:

Check for, in order, hugo.mod and go.mod and use the first found

We need to consider dependencies in all of this

I hadn't had that on the radar. If you mean by that a hugo module, yes they might be forced to use go.mod, that might be hardcoded. I use a custom goproxy and can see the requests there. My aim in this rewuest was the end site repo module management, but I'll have a look into it as well.

Also, and I don't think it will solve the above problems, we do have a modules.workspace setting that allow you to use e.g. a hugo.work file.

So I couldn't wait :D
Workspaces seem to be aimed to simplify local development (and I have no experience with it yet). So *.work files are technically/probably not part of repos(?). This feature request is aimed for a endsite monorepo where go and hugo can work together independent of local-machine-only setup.
Overall to my knowledge there is one big property hugo modules differ from go modules

Hugo needs access to all files without them beeing in a go package. Once hugo builds, it needs to be able to access all files within the go pkg cache and same rule applies for vendoring (here go can strip out files)

I made a similiar workaround when working with protobuf files to still be able to vendor them out during compilation
And IMO these are the four main tasks during a hugo module build handling
Preamble: I have a go.mod to be "in the way" when handling hugo.mod*
$ cat go.mod
module myproject

go 1.19

require github.com/gohugoio/hugo v0.111.2

1) Initializing hugo.mod & discovery and registering the module in a hugo.mod
$ go mod init -modfile=hugo.mod example.com/m
go: creating new go.mod: module example.com/m
go: to add module requirements and sums:
        go mod tidy

$ go get -modfile=hugo.mod github.com/McShelby/hugo-theme-relearn
go: added github.com/McShelby/hugo-theme-relearn v0.0.0-20230303193005-e619dabc51a1

$ cat hugo.mod hugo.sum
module example.com/m

go 1.19

require github.com/McShelby/hugo-theme-relearn v0.0.0-20230303193005-e619dabc51a1 // indirect
github.com/McShelby/hugo-theme-relearn v0.0.0-20230303193005-e619dabc51a1 h1:hGJYI84Tqtt5KMJlzyXH08ctE3M2Ygh1Tk4Mr0uYhfs=
github.com/McShelby/hugo-theme-relearn v0.0.0-20230303193005-e619dabc51a1/go.mod h1:mKQQdxZNIlLvAj8X3tMq+RzntIJSr9z7XdzuMomt0IM=

2) Downloading hugo module in the go pkg cache
$ go mod download -modfile=hugo.mod -json
{
        "Path": "github.com/McShelby/hugo-theme-relearn",
        "Version": "v0.0.0-20230303193005-e619dabc51a1",
        "Info"<EMAIL_ADDRESS>        "GoMod"<EMAIL_ADDRESS>        "Zip"<EMAIL_ADDRESS>        "Dir"<EMAIL_ADDRESS>        "Sum": "h1:hGJYI84Tqtt5KMJlzyXH08ctE3M2Ygh1Tk4Mr0uYhfs=",
        "GoModSum": "h1:mKQQdxZNIlLvAj8X3tMq+RzntIJSr9z7XdzuMomt0IM="
}

-json is just for visibility that it actually only read hugo.mod, you can always use -f to only output a specific property
3) Accessability of module contents
a) via go pkg cache
$ go list -m -json -modfile=hugo.mod ...
{
        "Path": "example.com/m",
        "Main": true,
        "Dir": "/home/dionysius/Projects/myproject",
        "GoMod": "hugo.mod",
        "GoVersion": "1.19"
}
{
        "Path": "github.com/McShelby/hugo-theme-relearn",
        "Version": "v0.0.0-20230303193005-e619dabc51a1",
        "Time": "2023-03-03T19:30:05Z",
        "Indirect": true,
        "Dir"<EMAIL_ADDRESS>        "GoMod"<EMAIL_ADDRESS>        "GoVersion": "1.18"
}

-json is just for visibility that it actually only read hugo.mod, you can always use -f to only output a specific property
$ ls -1 $(go list -m -f '{{.Dir}}' -modfile=hugo.mod github.com/McShelby/hugo-theme-relearn)
archetypes
assets
config.toml
exampleSite
frontmatter.json
go.mod
hugo-theme-relearn.code-workspace
i18n
images
layouts
LICENSE
README.md
static
theme.toml
vscode-frontmatter

In the go pkg cache there seems to be always the full repo checked out.
b) via vendoring
Here it gets tricky, thats maybe why hugo has its own vendoring functions. go vendor does only checkout out go packages, that means any folder containing go files (but then all files). The best I could get is ignoring errors with -e to continue the command
$ go mod vendor -v -e -modfile=hugo.mod -o _vendor
example.com/m imports
        github.com/gohugoio/hugo: no required module provides package github.com/gohugoio/hugo; to add it:
        go get github.com/gohugoio/hugo
# github.com/McShelby/hugo-theme-relearn v0.0.0-20230303193005-e619dabc51a1
## explicit; go 1.18

-v is just for visibility
And we have the modules.txt, but no checkout out data:
$ ls -1 _vendor/
modules.txt
$ cat _vendor/modules.txt
# github.com/McShelby/hugo-theme-relearn v0.0.0-20230303193005-e619dabc51a1
## explicit; go 1.18

Because this hugo module does not have any go packages/go files. But we can just copy the files from the go pkg module cache, since it contains all files:
# for each hugo module (if vendoring is not disabled by config)
$ mkdir -p _vendor/github.com/McShelby/hugo-theme-relearn
$ cp -rT $(go list -m -f '{{.Dir}}' -modfile=hugo.mod github.com/McShelby/hugo-theme-relearn) _vendor/github.com/McShelby/hugo-theme-relearn
$ tree -L 4 _vendor/
_vendor/
├── github.com
│   └── McShelby
│       └── hugo-theme-relearn
│           ├── archetypes
│           ├── assets
│           ├── config.toml
│           ├── exampleSite
│           ├── frontmatter.json
│           ├── go.mod
│           ├── hugo-theme-relearn.code-workspace
│           ├── i18n
│           ├── images
│           ├── layouts
│           ├── LICENSE
│           ├── README.md
│           ├── static
│           ├── theme.toml
│           └── vscode-frontmatter
└── modules.txt

11 directories, 8 files

4) Build the hugo site
Hugo can access now the vendor folder if exists (isVendored() from modules/collect.go:543) or fall back to the go pkg cache as already (listAndDecodeModules() from modules/client.go:472), but add -modfile to it.
Some hugo mod commands not reflected above
hugo mod tidy needs indeed to be a hugo implementation as the go variant scans the source code to tidy it - and hugo has no go files. But hugo has them listed as imports in the config and thus already can write them out. Respecting the -modfile switch would let it just write it to that .mod and .sum file.
hugo mod graph can be forwarded to go:
go mod graph -modfile hugo.mod
example.com/m<EMAIL_ADDRESS>
Notes

I'm pretty sure hugo already does those workarounds in some way or another. I just listed the full cycle in pure commands.
Right now I have golang 1.19.4, experience might vary with earlier go version so this needs to be tested as well.
Does hugo verify vendored modules? Because I saw in modules/collect.go:209 (// If both these are true, we don't even need Go installed to build.). Because without go you probably can't verify that the vendored files are matching the checksum of hugo.sum unless hugo has this check reimplemented. Having a vendored build without requiring go would indeed be a plus for many.
And yes, picking hugo.mod or go.mod could probably also be autodiscovered in hugo code.
I haven't yet looked into whether hugo modules could also only deliver hugo.mod files instead of go.mod. If hugo modules can also only provide hugo.mod, we even can switch to hugo.mod fully and use go.mod as compatibility fallback.
Since go 1.18 the -d in go get is always set in module mode and thus not set above
Having no go.mod makes this full process pretty unusable. This is the part where we would need to voice our use case upstream. AFAICS this currently makes the modfile switch only possible if you actually have a separate use of the go.mod file (This is probably due A file named go.mod must still be present in order to determine the module root directory, but it is not accessed. from the release notes).  You can get only this far:

$ go mod init -modfile=hugo.mod example.com/m
go: creating new go.mod: module example.com/m
go: to add module requirements and sums:
        go mod tidy
$ go get -modfile=hugo.mod github.com/McShelby/hugo-theme-relearn
flag provided but not defined: -modfile
usage: go get [-d] [-f] [-t] [-u] [-v] [-fix] [build flags] [packages]
Run 'go help get' for details.

But this was overall the idea behind this concrete feature request on why hugo can make -modfile work for end site repos if chosen to do so by the developer. I'll look into it whether hugo modules can provide only a hugo.mod "technically", but I believe go.mod is an integral part of the go module discovery framework.
What do you think? You have more insight in hugo on what might be an issue to this idea since I still don't know hugo very well.

The possibilities are:

switching to hugo.mod autonomously by choice of hugo devs

new projects: creating empty go.mod if missing
existing projects: migrating go.mod to hugo.mod by renaming it, still create empty go.mod due to technical reasons

switching to hugo.mod by site developer choice by introducing a config property:

either a bool property to use separate .mod file (hugo.mod constant)
or a string property so the full file name can be specified

not offering this option at all and thus the claim stays If you create a Hugo project, that is the spec you need to follow

Thanks for the writeup. The one thing that's not clear from the above, which needs to be tested is:
$ go get -modfile=hugo.mod github.com/McShelby/hugo-theme-relearn
go: added github.com/McShelby/hugo-theme-relearn v0.0.0-20230303193005-e619dabc51a1
$ cat hugo.mod hugo.sum
module example.com/m

go 1.19

require github.com/McShelby/hugo-theme-relearn v0.0.0-20230303193005-e619dabc51a1 // indirect
github.com/McShelby/hugo-theme-relearn v0.0.0-20230303193005-e619dabc51a1 h1:hGJYI84Tqtt5KMJlzyXH08ctE3M2Ygh1Tk4Mr0uYhfs=
github.com/McShelby/hugo-theme-relearn v0.0.0-20230303193005-e619dabc51a1/go.mod h1:mKQQdxZNIlLvAj8X3tMq+RzntIJSr9z7XdzuMomt0IM=

$ go mod download -modfile=hugo.mod -json
{
        "Path": "github.com/McShelby/hugo-theme-relearn",
        "Version": "v0.0.0-20230303193005-e619dabc51a1",
        "Info"<EMAIL_ADDRESS>        "GoMod"<EMAIL_ADDRESS>        "Zip"<EMAIL_ADDRESS>        "Dir"<EMAIL_ADDRESS>        "Sum": "h1:hGJYI84Tqtt5KMJlzyXH08ctE3M2Ygh1Tk4Mr0uYhfs=",
        "GoModSum": "h1:mKQQdxZNIlLvAj8X3tMq+RzntIJSr9z7XdzuMomt0IM="
}

In the above:

The local project has a hugo.mod file
hugo-theme-relearn has a go.mod file which is resolved correctly.

But what if  hugo-theme-relearn had a hugo.mod file`?

Also, the short answer to why Hugo has its own vendoring logic:

Go's vendor tool (and some others) operate on Go packages and and only vendor the Go packages in use (which obviously doesn't work for us)
Also, when Go zips a module, it ignores vendor folders. We respect transitive vendoring, so to speak, so we name the folder _vendor to allow it to survive.

I have pulled the proposal label off of this issue. I'm not sure it's doable, but it's well worth it if it is.

Thank you. Yes ultimatly I tried to find out how you would want it. It sounds like a constistent hugo.mod file across the dependency tree is the wished goal of hugo.
Can you maybe summerize for me how hugo currently accesses the module data? Is it the go pkg cache and the custom _vendor folder as assumed above or are there some more steps I should know of?

Well,
The logic is here: https://github.com/gohugoio/hugo/tree/master/modules
The main gist of it is:

We build the "module tree" based on module imports (transitive)
We use go commands for everything, but have built some wrappers because Go doesn't know about the above. E.g. the Tidy command: https://github.com/gohugoio/hugo/blob/master/modules/client.go#L177 -- running go mod tidy would just remove everything.

I did some quick testing in #10807 -- and it fails on go mod download.
go mod download -modfile=hugo.mod
go: cannot find main module, but -modfile was set.
	-modfile cannot be used to set the module root directory.

See https://github.com/bep/hugo/pull/195
First you had trailing space.
Added a hacky way to temporary add a go.mod.
Now it's hanging in go list, because -modfile needs to come before all. The position of -modfile depends on each command, so you'd have to add it per command specifically.
go: no module dependencies to download
go: no module declaration in go.mod. To specify the module path:
	go mod edit -module=example.com/mod
--- FAIL: TestHugoModFile (0.07s)
    c:\Users\Dionysius\Projects\github.com\dionysius\hugo\modules\integrationtest_builder.go:329:
        error:
          got non-nil error
        got:
          e"failed to list modules: failed to execute 'go [list -m -json all -modfile=hugo.mod]': failed to execute binary \"go\" with args [list -m -json all -modfile=hugo.mod]: go: no module declaration in go.mod. To specify the module path:\n\tgo mod edit -module=example.com/mod\n *errors.errorString"

If you want, I can continue this branch for a bit, since I have a simple testcase available now

Please do, but I suggest you just create a new branch/PR (make it a draft so it gets less noisy) so you can push directly to it. Also, you don't need to spend time on making it "pretty" -- the important part is to demonstrate that it's possible.

Also note that the convincing test case would be if you could get that test case to also handle the hugo.mod change here:
https://github.com/bep/hugo-mod-with-hugodotmod/commit/c3e39ab6de5bb0763a5f5e80f03dc02fdc365eb2
Which would require you to change path to github.com/bep/hugo-mod-with-hugodotmod/v6 and the assertion from bar => baz.

Have a look https://github.com/gohugoio/hugo/pull/10808 and I added my notes there as well

Hello,
I understand that people using Hugo have to play with its rules and I’m absolutely not contesting this fact. However, it seems to me that getting Hugo and Go play nicely together would allow for interesting use cases to be realized. One such use case is a Go project consisting of several assembled modules: one could build the documentation for this project by gathering the documentation included in these modules. The documentation would always be in sync with the code.
I didn’t think that much about a way to solve this use case, but it seems to me that if Hugo could manage a hugo.go file consisting only of the naked imports of the packages containing content, it could solve the conflict with go mod tidy and go mod vendor.

As discussed in #10808 we skip the hard parts and focus on the main project repo first.
In my opinion, as a go developer, a modfile property feels better than a hugo.mod. But my use case would be satisfied with either option. The required folder structure of hugo would not make it possible to have two hugo sites in the same folder anyway.
In the end those options remain to be decided by hugo (from https://github.com/gohugoio/hugo/issues/10801#issuecomment-1458621992, bold are changed/improved):

The possibilities are:

switching to hugo.mod autonomously by choice of hugo devs (if hugo modules are in use at all)

new projects: creating temporary empty go.mod if missing due to technical reasons
existing projects: migrating go.mod to hugo.mod by renaming it, still temporary reate empty go.mod due to technical reasons

probably a sanity check required in case someone still has "workarounded "the hugo spec

switching to hugo.mod by site developer choice by introducing a config property:

either a bool property to use separate .mod file (hugo.mod constant)
or a string property so the full file name can be specified (file extension should probably still be .mod)
we could not do any temporary go.mod: Whoever turns on this feature, knows what they are doing and have read the documentation about this option. Having the temporary go.mod would make it a bit less error prone and for hugo and more convenient for the developer though (less issue reports if used wrongly)

not offering this option at all and thus the previous spec stays that hugo modules should not be combine with go modules

Might be feasable since the noise turned relatively low in the initial issue

Note to myself: I've discovered how/where hugo checks out dependencies, which I tried to ask before https://github.com/gohugoio/hugo/issues/10801#issuecomment-1460237735:

hugo seems to check out dependencies to /tmp/hugo_cache/modules

hugo seems to check out dependencies to /tmp/hugo_cache/modules

See https://gohugo.io/getting-started/configuration/#configure-file-caches
If you set the HUGO_CACHEDIR env var to something outside /tmp, I suspect you will get an easier debug  situation.

In my opinion, as a go developer, a modfile property feels better than a hugo.mod. But my use case would be satisfied with either option.

Yea, we can add a modfile option here, which would make it behave similar to the workspace option -- and make sure to document that it only works for the main Hugo project (which is where you run hugo from).
https://gohugo.io/hugo-modules/configuration/


Error log for v<IP_ADDRESS>.4 build

just errors. idk what it affect for, but hugslib show debug log
starts from 1395 line
https://gist.github.com/HugsLibRecordKeeper/26a175e3b3a557a0d6ace2d1a935a9f7

Update your copy of WorkTab. Numbers needs at least version 3.8.281 which was released 21 Nov 2018. You are on 3.4.185 from 30 Oct 2018.
Get the latest from https://github.com/FluffierThanThou/WorkTab/releases/latest or Steam.


Resin OS Download state

Extend resin.models.os.download() state to provide eta, remaining, etc.
Ideally, we can create a ProgressState class that provides an uniform interface to the state, and we can then make Resin CLI Visuals progress bar accept a ProgressState as an argument.

The implementation of the ProgressState class is complete. I'll move the Resin CLI Visuals progress bar accept a ProcessState as an argument to the relevant repository.


Make install and upgrade retries number configurable

What does this PR do?
This PR enables users of the cluster chart to configure the install and upgrade retries of HelmReleases. During cluster upgrades, HelmRelease upgrade tries are exhausted because of non-ready nodes.
What is the effect of this change to users?
All components using HelmReleases can have different install and upgrade retries
Any background context you can provide?
https://github.com/giantswarm/roadmap/issues/3664
Should this change be mentioned in the release notes?

[x] CHANGELOG.md has been updated (if it exists)

In general I agree with Marcus, but on the other hand this feels like setting timeouts in network apps: Of course you can always just set huge timeouts, but this sometimes hides root causes you want to fix. So I'm not sure if we should rather pick a high value or use -1.
Whichever way we go, having it configurable makes sense to me as you we might end up in an incident where it would be useful to not have it hardcoded one day.

In general I agree with Marcus, but on the other hand this feels like setting timeouts in network apps: Of course you can always just set huge timeouts, but this sometimes hides root causes you want to fix. So I'm not sure if we should rather pick a high value or use -1.

We should still have alerting in place for when these are stuck pending for too long. That doesn't change. But all default apps (what we're talking about here) need to install for a WC to be considered successful. So there's no reason not to keep trying from what I see.

Whichever way we go, having it configurable makes sense to me as you we might end up in an incident where it would be useful to not have it hardcoded one day.

I don't think it adds anything to introduce more complexity "just in case". When we have an actual need, sure, but I don't see that we actually need to configure it right now. We just need it not to timeout.


register beacon return 403, permission_denied error

I built the project on my machine.
I configured my G Dev console:

enabled proximity API
added cred for OAuth2.0 and Client ID. Based on the SHA1 code of my machine.

At the beginning I had wrote the wrong namespace when I added the OAuth2.0 and API key.
but then I fixed it.
I ran the app.
It discovered my android-phone-beacon with status "unregistered".
how can I fix this? the discovery works, but the registration doesn't.

these are the toasts for the discovery and registration (attached files)


Iterate over all the elements in table. How?

Hi,
Thank you for making redb. I am really enjoying working with it, but I hit a wall when wanting to list all the elements in a table.
So, is there a way to list all the elements in a table? I know about range() method but it doesn't seem to implement IntoIterator.

Ya, unfortunately it can't implement Iterator due to lifetime requirements. The object that is returned has a .next() method; you need to call that

Ah yes, looking at this again, I think that might be possible. I'll see if I can do the same


Test Integration: Make process use t.Cleanup

Use t.Cleanup for process cleanup to ensure Cleanup is always performed, regardless of whether the test fails or not.
Also simplifies the framework by removing the Framework struct completely.

/test-version-skew

/ok-to-test

/test-sdk-all


Is there a cleaner way to consume JWT API?

Is there a cleaner way to consume JWT API?
I'm looking for something along the lines of:
https://github.com/Moya/Moya
Thanks!

Please use forum.vuejs.org.
This repository is not for questions.


Not quoting column name when doing a time series

Make sure these boxes are checked before submitting your issue - thank you!

[X] I have checked the superset logs for python stacktraces and included it here as text if any
[X] I have reproduced the issue with at least the latest released version of superset
[X] I have checked the issue tracker for the same issue and I haven't found one similar

Superset version
Commit
1f8fccc0f923de0b119b22303e504e9efeeff417
Expected results
time series should work if column name has a space in it
Actual results
get sql error because column is not properly quoted
Steps to reproduce
Have a table with columns where column names have spaces, including
date columns, for example 'CREATED DATE'
Create a slice and in the Time section select a column with a space in the name.
Also select a 'Time Grain' of 'Time Column'.  When you run the query, you will see that the
generated query is not quoting the column like it should.
I've been debugging this and I think the problem lies in superset/connectors/sqla/models.py
I think the get_timestamp_expression function should return column at end of function
and not literal_column
def get_timestamp_expression(self, time_grain):
        """Getting the time component of the query"""
        expr = self.expression or self.column_name
        if not self.expression and not time_grain:
            return column(expr, type_=DateTime).label(DTTM_ALIAS)
        if time_grain:
            pdf = self.python_date_format
            if pdf in ('epoch_s', 'epoch_ms'):
                # if epoch, translate to DATE using db specific conf
                db_spec = self.table.database.db_engine_spec
                if pdf == 'epoch_s':
                    expr = db_spec.epoch_to_dttm().format(col=expr)
                elif pdf == 'epoch_ms':
                    expr = db_spec.epoch_ms_to_dttm().format(col=expr)
            grain = self.table.database.grains_dict().get(time_grain, '{col}')
            expr = grain.function.format(col=expr)
    return literal_column(expr, type_=DateTime).label(DTTM_ALIAS)

Upon further debugging I think this is the correct fix (below)
I changed if not self.expression and not time_grain: to
if not self.expression and (not time_grain or time_grain == u'Time Column') :
  def get_timestamp_expression(self, time_grain):
      """Getting the time component of the query"""
      expr = self.expression or self.column_name
      if not self.expression and (not time_grain or time_grain == u'Time Column') :
          return column(expr, type_=DateTime).label(DTTM_ALIAS)
      if time_grain:
          pdf = self.python_date_format
          if pdf in ('epoch_s', 'epoch_ms'):
              # if epoch, translate to DATE using db specific conf
              db_spec = self.table.database.db_engine_spec
              if pdf == 'epoch_s':
                  expr = db_spec.epoch_to_dttm().format(col=expr)
              elif pdf == 'epoch_ms':
                  expr = db_spec.epoch_ms_to_dttm().format(col=expr)
          grain = self.table.database.grains_dict().get(time_grain, '{col}')
          expr = grain.function.format(col=expr)
      return literal_column(expr, type_=DateTime).label(DTTM_ALIAS)

I'd be happy to submit a pull request of people think this is correct.
thanks!

Please send the PR!

Issued pull request #4343


FlowRouter.current().route.group.prefix

Objective: Getting the name of the group route(s) which the current route is nested in.
What about FlowRouter.current().route.group.name?
MainRouter = FlowRouter.group({
  prefix: '/blah',
  middlewares: [
    function(path, next){
      console.log('running group middleware');
        next();
    }
  ],
  name: 'mainRoute'
})

Would like to be able to check which parent Route I am currently under in Template helpers.
I can just use {...}.prefix but it would be convenient to do it with  human-named Group Route.

Well, this is not an imperative. I've switched to using explicit paths.


FileSystem -> Create Scene has different behaviour than Scene -> New Scene

Godot version:
3.2 Beta 4
OS/device including version:
Mint 19.2
Linux Linux 4.15.0-72-generic
Issue description:

Steps to reproduce:
Right click in FileSystem to create a new scene and it asks for a name. This should just be removed completely because it makes an empty scene anyways.

Update: The name actually is used, but only when you save the scene

I think that the option should still be removed to be consistent with Menu->Scene->New Scene

Um, it asks for the filename. When you save that empty scene it will create "This goes nowhere!.tscn".
If anything, we could change behavior of empty scenes. When you delete scene root, the tab becomes "empty scene" too, but you can still save it when you add new root.

I would vote for one of two options

File System -> New scene asks for a name, the scene should be created with a named root node and automatically saved to the file system

File System -> New scene is just a shortcut for Scene -> New Scene (Top bar), does not ask for a name. Maybe CTRL S automatically highlights the folder you right clicked in but the name is the root node name

Still valid in 3.2.4 beta4

Should we move this to proposals?


Admin Error

getting this even when running as admin

I honestly have no idea, from what is says it seems you aren't logged as the admin

I managed to install by spamming running it without selecting run as admin. It asked for permission to run as admin after the third time and it work then. It was one of the weirdest thing ever happen to me in windows 10, still can't figure out what's the problem...

After checking my antivirus (Avast) logs, it was blocking the install each time and spamming it somehow let one instance escape from it.


Specify what ip pool to use with the OpenStack hypervisor

Currently it is not possible to specify what ip pool to use when allocating ip addresses. This means that provisioning will fail if the default pool does not exist.

@stefanmortensen the PR put forward for this was closed for inactivity: #619.

Closing due to inactivity.


Update generated Node.js samples

use async/await
fix paginated samples
add Buffer.from()

@fhinkel Please let me know if you need a review on this!


allow to set listening address for nhost dev/start

Is your feature request related to a problem? Please describe.
when running nhost dev the resulting listening addresses that are mapped from the host to the docker containers are all "localhost".
This is perfect if the docker host is the same machine that the development is being done on.
However some users run their docker host within a virtual machine, or some other system container technology, so it would be handy to be able to also set the listening address even when running in development mode.
for instance this would enable the user to be able to set the listening address to the IP address of the VM container on the host or just <IP_ADDRESS> to listen on all.
Describe the solution you'd like
an additional command line flag could be added. e.g.
$ nhost dev --address <IP_ADDRESS> --port 1234 [flags etc]
> Starting your app
> Running a quick health check on services
> Preparing your data

---      -----
Backend  http://<IP_ADDRESS>:1337
Mailhog  http://<IP_ADDRESS>:8838
Console  http://<IP_ADDRESS>:9363
---      -----

I think this would also be related to #116


LiteBans "Command Manager Not Found"

Update to the latest version of the plugin.

Update to the latest version of the plugin.


Cheap Yellow Display does not use the display

The code compiles and executes but does not use the display for anything.  It is basically a compile for the ESP32

Yeah I know... updated README.md


Adding safety on number of lines and number of fields per line when WaveMode=6 (Should Fix #118)

THIS PULL REQUEST IS READY TO MERGE
Feature or improvement description
Input files are now checked for sufficient number of lines, and each line is checked to see if enough columns are provided. Explicit error messages with recommendations are provided.
The format is still rather flexible. One line of comment is required at the beginning of the file.
The file should then contain n x m values space, or comma separated, with n>=WaveTMax/WaveDt and m>=NNodes (the number of internal HydroDyn Nodes. If we want strict equality, it's a bit more work.
Related issue, if one exists
See #118
Impacted areas of the software
HydroDyn with WaveMod=6
Automated test results
None, but the safety checks can be tried by removing some data in the input file (either a line or a single field), and HydroDyn should now mention where the data is missing in the file.

@ghaymanNREL I've looked over this pull request and it looks fine to me. Could you give it a second review?


Add Configuration for Highlight Related Feature

Summary
Adds basic configuration that allows you to control when the highlight related feature is activated. You can control this for references, break points, exit points, and yield points.
Resolves #9618

bors r+


feat(AlertBanner/Avatar/CompositeAvatar): add AlertBanner, Avatar, CompositeAvatar

Description
add Angular AlertBanner
add Angular Avatar
add Angular CompositeAvatar
fix Angular Loading component display issue
fix Angular Icon component getHexFromJSON error and display issue
Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots:
Before (If applicable):

After:

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation update
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Other... Please describe:

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

Not able to pass custom class
<cui-avatar class="custom-class" title="Tom Smith" type="active"></cui-avatar>

This is an existing issue for other components #105

There is a prop named className, use this to pass custom class.

@wzpo we can't depend on className, because we also need to support ngClass

@mohammedzamakhan @pauljeter If I use @HostBinding('class.cui-avatar') instead of @HostBinding('class').  ngClass or class="custom-class" will work. But some class names are variables. e.g. cui-avatar--${this.type}, and I don't know how to use @HostBinding or host property to bind variable class names.

@wzpo would this be a viable solution? https://stackblitz.com/edit/angular-rrziff?file=src%2Fapp%2Fhello.component.ts

@mohammedzamakhan We have dynamic class names. If we only have static class names. Your code works. But we need bind class names like this, it doesn't work: https://stackblitz.com/edit/angular-ox6ddc?file=src/app/hello.component.ts

Yeah, it looks pretty messy even in how Angular Material does it https://github.com/angular/material2/blob/master/src/lib/badge/badge.ts#L43

@mohammedzamakhan Able to pass custom class and use ngClass now.
Thanks @pauljeter for the code in the Space.


docs: addition of param and result sections

Improves documentation. (addresses #22)

Signed-off-by: Leonardo Rossi<EMAIL_ADDRESS>

[ ] https://github.com/inveniosoftware/invenio-admin/pull/28/files#diff-05287a5c299bff612c62140bd2f722a5R4 update copyright year
[ ] add yourself to AUTHORS file

ping @jirikuncar


fix: gracefully ignore unlink errors

Description
I'm seeing intermittent build failures relating to deletion of a temporary file vite.config.js.mjs which is generated by loadConfigFromBundledFile:
failed to load config from vite.config.js
error during build:
Error: ENOENT: no such file or directory, unlink 'vite.config.js.mjs'
    at Object.unlinkSync (node:fs:1780:3)
    at loadConfigFromBundledFile (node_modules/vite/dist/node/chunks/dep-1513d487.js:62806:18)
    at async loadConfigFromFile (node_modules/vite/dist/node/chunks/dep-1513d487.js:62692:28)
    at async resolveConfig (node_modules/vite/dist/node/chunks/dep-1513d487.js:62344:28)
    at async doBuild (node_modules/vite/dist/node/chunks/dep-1513d487.js:43308:20)
    at async build (node_modules/vite/dist/node/chunks/dep-1513d487.js:43297:16)
    at async CAC.<anonymous> (node_modules/vite/dist/node/cli.js:747:9)

I'm not sure why it fails exactly, but I'm sure that these unlink errors can be gracefully ignored, which I did here.
What is the purpose of this pull request?

[x] Bug fix

I think this should be fixed in #9224. Are you running builds/dev in parallel?

Linked fix should work, but it could still fail if both files are written in the same millisecond, so I guess that fix here may still be valuable.

I doubt there's a chance both files are writing within the same millisecond. We're running async code above so there's a lot of variability. try..catch imposes a small perf implication too so if we can avoid it then I think it's better.

Okay, I will wait for next release then to test this again.


chore(deps): upgrade runtime dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-runtime-dependencies-2.x" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 2169

Questions ?
Please refer to the Backport tool documentation


chore(master): release 1.2.46

:robot: I have created a release beep boop
1.2.46 (2023-08-22)
Miscellaneous

deps: update dependency phpstan/phpstan to v1.10.30 (60c2e2a)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/wayofdev/php-cs-fixer-config/releases/tag/v1.2.46 :sunflower:


Backend bugfixes

Fixed issue where the "verification" field in the backwork JSON object wasn't showing up
Removed dead code from controller/authentication.go
Fixed JSON field in controller/backwork.go GetBackwork endroute (was "data"  now is "backwork")

nice


[quick_actions][android] Shortcuts do not show unless the app is exited and reopened.

quick_actions works on Android12,but not working on Android11.
which settings can solve the problem?
Steps to Reproduce

Execute flutter run on the code sample

Expected results:
App Shortcuts will show when running.
Actual results:
quick_actions works on Android12,but not working on Android11

code sample
import 'package:flutter/material.dart';
import 'package:quick_actions/quick_actions.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Quick Actions Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: const MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({Key? key}) : super(key: key);

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  String shortcut = 'no action set';

  @override
  void initState() {
    super.initState();

    const QuickActions quickActions = QuickActions();
    quickActions.initialize((String shortcutType) {
      setState(() {
        if (shortcutType != null) {
          shortcut = shortcutType;
        }
      });
    });

    quickActions.setShortcutItems(<ShortcutItem>[
      // NOTE: This first action icon will only work on iOS.
      // In a real world project keep the same file name for both platforms.
      const ShortcutItem(
        type: 'action_one',
        localizedTitle: 'Action one',
        icon: 'AppIcon',
      ),
      // NOTE: This second action icon will only work on Android.
      // In a real world project keep the same file name for both platforms.
      const ShortcutItem(
          type: 'action_two',
          localizedTitle: 'Action two',
          icon: 'ic_launcher'),
    ]).then((void _) {
      setState(() {
        if (shortcut == 'no action set') {
          shortcut = 'actions ready';
        }
      });
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(shortcut),
      ),
      body: const Center(
        child: Text('On home screen, long press the app icon to '
            'get Action one or Action two options. Tapping on that action should  '
            'set the toolbar title.'),
      ),
    );
  }
}

Hi @niumj, what version of quick_actions are you using?
Please also provide the output of flutter doctor -v.
Thank you

Hi @niumj, you appear to be on an older version of flutter, can you upgrade to the latest version of flutter to see whether or not you still experience the issue?
If you still experience the issue, please provide the updated flutter doctor -v output and share the devices that you used in your testing.

danagbemava-nc
Thank you for your advice. But I need work on [Flutter 2.10.5].

Hi @niumj, I ran your code sample on two emulators, 1 running android 11 and the other android 12 and the sample works the same on both versions of the emulator. See the recording below
What device are you experiencing this issue with? Is it debug mode or release mode?
https://user-images.githubusercontent.com/88313112/192210662-d96b1480-0492-42d6-a0fe-9ad58fd2ce3a.mp4

@danagbemava-nc
Thank you for your confirm.But on emulator ,the same your result's [0:04] :default menu's name are not displayed.
Ran on debug mode.
So I ran on real device [Xiaomi9 (Android 11)], the result is default menu ok,but quick menu isn't showed.

Hi @niumj, you can see in the video that the android 12 emulator also doesn't show the menu name either. Can you share a screenshot/ recording from your device that shows the behaviour you're expecting as opposed to what you're getting?
FWIW none of the app shortcuts seem to show a menu name, unless I'm misunderstanding what you're referring to. See the recording below
https://user-images.githubusercontent.com/88313112/192450099-d956031d-77c4-44d5-afdd-8cf6ae1507c4.mp4

@danagbemava-nc  Thank you for your attention. Reconfirm the result on the real device： I
1、after app installed ，do nothing quick_actions menu are not showed.
2、after app installed ，open the app  then back to desktop ， quick_actions menu  are showed。

Hi @niumj, I'm not sure I understand.
Can you share a recording/screenshot from your device that captures the issue you're referring to clearly?

@danagbemava-nc  Thank you for your attention.See the recording below
https://user-images.githubusercontent.com/90394057/193379294-f9f2c5a2-b321-4e2b-9134-231fcd481256.MOV
https://user-images.githubusercontent.com/90394057/193379302-3dcbff7e-a8c9-41ff-a68e-433fc37f5bca.MOV

Hi @niumj, it appears that you are using two different steps in the recordings you shared above. If you open the app on android 11 and come back to the home screen, do the shortcuts show? In your video for android 12(MIUI 13), the shortcuts do not show unless you open the app and go back to the menu.

@danagbemava-nc yes，on Android11,the result is the same too on android12.The shortcuts do not show unless you open the app and go back to the menu.

@niumj I tested with all 3 devices as below.

Pixel 3a, Android 12
Samsung Galaxy Tab A7 Lite, Android 12 OneUI 4.1, SM-T225
Realme 6, Android 11

On (1) and (2), the issue is not reproducible.
On (3) the behavior is quite different on UI but all actions are still shown on the menu. The Action one title is wrong even though its operation is still correct (the app is opened when clicking on it).

Demo (Realme 6)

And no need to open the app first. Everything is working as expected even the app is killed from Recent apps menu.
Maybe the issue is only happening on specific devices. Could you provide all device information you tested above? (include the model/Android OS version/MIUI version)

@huycozy  Thank you for your test.I tested with 2 device as above.
1.Xiaomi 9, Android 11, MIUI 12.5
2.Xiaomi 12, Android 12, MIUI 13.0.36

@niumj thanks, I would like to summarize the issue as below, please confirm it:

Plugin version: quick_actions: ^1.0.0
Actual results:
The shortcuts do not show unless you open the app and go back to the menu.
Expected results:
App Shortcuts will show up without the first opening.
The issue is happening on both Android 11 and Android 12 with these devices:

Xiaomi 9, Android 11, MIUI 12.5
Xiaomi 12, Android 12, MIUI 13.0.36

Sample code:
https://github.com/flutter/flutter/issues/112027#issue-1380200749

flutter doctor -v (stable and master)
[✓] Flutter (Channel stable, 3.3.3, on macOS 12.6 21G115 darwin-x64, locale en-VN)
    • Flutter version 3.3.3 on channel stable at /Users/huynq/Documents/GitHub/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 18a827f393 (10 hours ago), 2022-09-28 10:03:14 -0700
    • Engine revision 5c984c26eb
    • Dart version 2.18.2
    • DevTools version 2.15.0

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/huynq/Library/Android/sdk
    • Platform android-33, build-tools 31.0.0
    • ANDROID_HOME = /Users/huynq/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.0.1)
    • Xcode at /Applications/Xcode14.0.1.app/Contents/Developer
    • Build 14A400
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] IntelliJ IDEA Community Edition (version 2022.2.2)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 70.0.5
    • Dart plugin version 222.4167.21

[✓] IntelliJ IDEA Community Edition (version 2022.1.1)
    • IntelliJ at /Users/huynq/Library/Application Support/JetBrains/Toolbox/apps/IDEA-C/ch-0/221.5591.52/IntelliJ IDEA CE.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.71.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.48.0

[✓] Connected device (4 available)
    • SM T225 (mobile)           • R9JT3004VRJ                          • android-arm64  • Android 12 (API 31)
    • iPhone 14 Pro Max (mobile) • 332079AF-E895-4685-910E-7B1E18B0C6B8 • ios            • com.apple.CoreSimulator.SimRuntime.iOS-16-0 (simulator)
    • macOS (desktop)            • macos                                • darwin-x64     • macOS 12.6 21G115 darwin-x64
    • Chrome (web)               • chrome                               • web-javascript • Google Chrome 105.0.5195.125

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

[✓] Flutter (Channel master, 3.4.0-40.0.pre.15, on macOS 12.6 21G115 darwin-x64, locale en-VN)
    • Flutter version 3.4.0-40.0.pre.15 on channel master at /Users/huynq/Documents/GitHub/flutter_master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 28da87101b (37 minutes ago), 2022-10-04 20:17:24 -0700
    • Engine revision 9692f09afd
    • Dart version 2.19.0 (build 2.19.0-273.0.dev)
    • DevTools version 2.18.0

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/huynq/Library/Android/sdk
    • Platform android-33, build-tools 31.0.0
    • ANDROID_HOME = /Users/huynq/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.0.1)
    • Xcode at /Applications/Xcode14.0.1.app/Contents/Developer
    • Build 14A400
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] IntelliJ IDEA Community Edition (version 2022.2.2)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 70.0.5
    • Dart plugin version 222.4167.21

[✓] IntelliJ IDEA Community Edition (version 2022.1.1)
    • IntelliJ at /Users/huynq/Library/Application Support/JetBrains/Toolbox/apps/IDEA-C/ch-0/221.5591.52/IntelliJ IDEA CE.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.71.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.48.0

[✓] Connected device (4 available)
    • SM T225 (mobile) • R9JT3004VRJ                              • android-arm64  • Android 12 (API 31)
    • iPhone (mobile)  • d9a94afe2b649fef56ba0bfeb052f0f2a7dae95e • ios            • iOS 15.7 19H12
    • macOS (desktop)  • macos                                    • darwin-x64     • macOS 12.6 21G115 darwin-x64
    • Chrome (web)     • chrome                                   • web-javascript • Google Chrome 105.0.5195.125
    ! Error: iPhone is busy: Fetching debug symbols for iPhone. Xcode will continue when iPhone is finished. (code -10)

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

@huycozy Thanks.Your summarize is good.
But I tested by flutter 2.10.5.Just because my project's worked on flutter 2.10.5 and I don't want to upgrade othet plugins.
quick_actions: ^1.0.0
Flutter 2.10.5
dart 2.16.2

@niumj You don't need to upgrade your project, just set up the latest Flutter SDK and run quick_actions sample code on your devices.

@huycozy  Thanks. The same result on the latest Flutter SDK.
quick_actions: ^1.0.0
Flutter 3.3.3
dart 2.18.2

@niumj thanks for confirming. Labeling the issue for further investigation from the team.
Summary of issues: https://github.com/flutter/flutter/issues/112027#issuecomment-1267989708

Quick Action creates Dynamic Shortcuts and not Static Shortcuts so this is expected. To create static shortcuts , refer this page @niumj

In the terminate state when I navigate to the app screen. the screen ui shrinking in the Flutter app

@danagbemava-nc Thank you for your attention. Reconfirm the result on the real device： I 1、after app installed ，do nothing quick_actions menu are not showed. 2、after app installed ，open the app then back to desktop ， quick_actions menu are showed。

I realize this is a very late response, but I think this is a natural behavior of android. Because when I download the Microsoft Bing app from the Play store, it does the same thing.
Device : Redmi Note 10 Pro


Feature: Progressive Web App

Implement Progressive Web App for CC Search
We can make both mobile and web as a progressive app! (Add App to Home Screen)

Mobile Compatible

Desktop Compatible(If Needed)

Most of the apps nowadays are sunsetting their mobile web apps and going full PWA like:
DRIBBLE:

TWITTER:

@abhisheknaiidu new features have to go through @annatuma for triage so I added "awaiting triage" back. Please talk with her directly and get her approval before removing the label.

Thanks for the suggestion @abhisheknaiidu
While I agree that PWAs are nifty, and may be appropriate for some apps, I'm not convinced that this would be the right move for CC Search, and definitely not in the short term.
We'd need to start by doing user research to understand if there was even a need for this. Assuming there was, we'd need to look into what is feasible for us to load offline for users, while being cognizant of storage space, amongst other things. Further, it would likely require a different definition of the product entirely, since the goal of PWAs is typically to build engagement with the product, while we explicitly steer users through CC Search, treating it as a portal, and send them on to their destinations of sites containing CC licensed content.
That said, the scenario may change in time. I'm going to put this in the parking lot for now and mark it as blocked. That way, anyone with a similar idea can contribute to the discussion here.

PWAs? Seems interesting


HDDS-3855. Add upgrade smoketest

What changes were proposed in this pull request?

Introduce a new sample docker-compose environment with test script geared towards running upgrades.  Currently it only performs a smoketest: write some keys with old version, read with new one.

Add a script for performing workaround steps for HDDS-3499 during upgrade.  This is executed using ozone-runner docker image, which now comes with ldb.

https://issues.apache.org/jira/browse/HDDS-3855
How was this patch tested?
Executed upgrade acceptance test locally and on GitHub.
https://github.com/adoroszlai/hadoop-ozone/runs/815608054

Overall it looks good to me, and really impressive approach. I have a few comments -- none of them are blocker, but i like to discuss technical details...

Thanks for taking a look.  I waited with the merge exactly to have this kind of discussion. ;)

Can you please help me to understand why did you remove -f "${compose_file}"?

Each -f accepts only a single filename, so using the same command with one or more files is easier with COMPOSE_FILE approach.  Initially I used two separate files (including the one from ozone env), so I needed this fix, but then abandoned that approach.  This part of the change could be extracted to a separate issue if you prefer to simplify this one a bit.  (It allows you to run ozone/test.sh with monitoring enabled, so I'd rather not drop it completely.)

fixed ip / dedicated network in docker-compose file seems to be unnecessary in this cluster (IMHO)
you create external volume directories but /data is already a volume inside the docker containers. If you use simple docker-compose stop instead of down it can be reused. Did you consider using this approach?

Why do you prefer external volumes? (I found two arguments: easier to debug + easier to execute commands when the cluster is down. But interested if you had any other motivations...).

After stop/start this is what ozone version prints:
                  //////////////
               ////////////////////
            ////////     ////////////////
           //////      ////////////////
          /////      ////////////////  /
         /////            ////////   ///
         ////           ////////    /////
        /////         ////////////////
        /////       ////////////////   //
         ////     ///////////////   /////
         /////  ///////////////     ////
          /////       //////      /////
           //////   //////       /////
             ///////////     ////////
               //////  ////////////
               ///   //////////
              /    0.5.0-beta(Crater Lake)

Source code repository git@github.com:apache/hadoop-ozone.git -r 9b4f8fd49fa15946994bccc6c6ac50a560cfb0ea
Compiled by dchitlangia on 2020-03-16T00:54Z
Compiled with protoc 2.5.0
From source with checksum 4cde4c7a7aaa250bfbaf58220cb8e2c

Using HDDS 0.5.0-beta
Source code repository git@github.com:apache/hadoop-ozone.git -r 9b4f8fd49fa15946994bccc6c6ac50a560cfb0ea
Compiled by dchitlangia on 2020-03-16T00:53Z
Compiled with protoc 2.5.0
From source with checksum 9df32efd56424ab869a0acd0124e4bf5

So docker-compose down/up is needed because changes to the compose file (docker image, etc.) are not picked up with stop/start. And we need different images before/after upgrade.
That's the reason for both volumes and network settings.  I had started out without the network/ip settings, but the containers did not always get the same address after down/up, nor would they reuse volumes.

It seems to be a big restriction that we can't start multiple datanode on the same file system without configuring the datanode path. This is the reason why you need dn1..dn3 directories. I am wondering if we can provide a generic solution to this one. Maybe we can support ${env...} notion when we set the datanode directory?

Would be nice, I think we can explore it later.

I had started out without the network/ip settings, but the containers did not always get the same address after down/up, nor would they reuse volumes.

You mean that I can't start the upgrade cluster with different IP addresses? It seems to be a serious bug which should be fixed. But we can test it with the same approach: hard coded network stack and two different docker-compose file with different ip addresses.

This part of the change could be extracted to a separate issue if you prefer to simplify this one a bit

I am fine to include it, not a big change. It's just good to have the explanation here.

Other random thoughts:
I plan to enable acceptance tests for k8s cluster definitions, too.

I would like to be sure that those configs are up-to-date and working
Kubernetes have better tooling for more complex clusters (eg. easy SSL certificate management)
While docker-compose is easy-to-use it has some strong limitations. K8s definitions have better flexibility (especially together with the flekszible tool.

Thanks @avijayanhwx for the review, and @elek for reviewing and merging this.


Refactored create-indexes to expose a method for generating the SQL

This is in the same vein as the existing SqlForCreate method.

A revised PR will replace this one.

Coverage increased (+0.05%) to 66.683% when pulling 63dd53fc8579501b5503a6b3ca6186e75135189c on rickb777:master into 88f3ff2c0d5eb7b20af1b6b4d2051d7ddfffadb0 on go-gorp:master.


Optimize ninja -d stats

-d stats enables instrumented profiling of key functions in ninja.
However, some of those functions are invoked 6+ million times in a NOP
build of Chromium and the cost of measuring those functions dwarfs the
cost of the functions.  The total measurement
cost (per ETW sampled profiling) is 700-1200 ms, which is many times
greater than the costs of the functions.
With this change typical output looks like this:
metric                  count   avg (us)        total (ms)
.ninja parse            6580    3526.3          23203.2
.ninja_log load         1       227305.0        227.3
.ninja_deps load        1       485693.0        485.7
node stat               168997  9.6             1615.0
depfile load            327     383.1           125.3
This resolves issue #1998.

Apologies for the chaos. This is a repeat of the previous pull request but without the extraneous merge commits and other nonsense. I also removed two #includes that are no longer needed.
On the previous PR it was suggested that maybe this should be a compile-time option instead of deleting the lines entirely. In my opinion it is better to delete the lines. If somebody is working on these functions and wants to profile them then they can add back the necessary lines (and the #include). I just did that when working on EvalString::Evaluate and it worked very nicely. There is nothing magical about these three functions to justify leaving #ifdefed profiling commands in them but not elsewhere.
The other METRIC_RECORD entries in the ninja code remain and are still producing statistics. I am removing these three because have a specifically poor cost-to-benefit ratio. I hope that this makes sense.

The original PR, now closed, is here: https://github.com/ninja-build/ninja/pull/2000

Any thoughts on this? It seems to me that we either have to have an extremely efficient HighResTimer() function (__rdtsc() is the only thing fast enough) or we have to remove these three uses of METRIC_RECORD. Spending 1+ seconds to record times that are not very accurate (and have, historically, been off by a factor of 50) doesn't seem useful. I say this as somebody who always runs ninja with the -d stats flag.

I'm not sure anybody would use this flag, however. For instance, when profiling EvalString::Evaluate I just added a METRIC_RECORD macro to do profiling, then deleted it when I was done. I think the same idea makes sense for these functions. If somebody wants to work on them they can add and then remove the macros. It's just a line of code.

I agree.
Let's see if someone else has some feedback, otherwise I'll merge this for the next release.

I think "lookup node" is effectively measuring hash time, and "canonicalize path" is measuring path canonicalization, which I know @nico spent some time on, so I CC him just in case.  I agree that someone interested in profiling these can either reinstrument them locally or use a profiler instead.

I think "lookup node" is effectively measuring hash time, and "canonicalize path" is measuring path canonicalization, which I know @nico spent some time on, so I CC him just in case.  I agree that someone interested in profiling these can either reinstrument them locally or use a profiler instead.

Is this waiting on anything? I run with -d stats a lot and I'd like to skip the extra second of latency for these stats.

Can this change be done with a compile-time option, instead of removing the associated code? That would let someone who still wants to measure these two cases do so if they need to without having to dig through the code to find the right functions to instrument.
They can be turned off by default, of course.

A compile-time option could be used, but there would be as much code controlling the statistics as in the statistics, and I'm not sure it would make them much more discoverable than being in github history. The profiling of these events hasn't worked for years (it still doesn't, although I'm working on improving that) and nobody noticed, so I'm not sure that preserving them makes sense.
The last comment from @jhasse suggested that just deleting them was best, FWIW.

I agree that someone interested in profiling these can either reinstrument them locally or use a profiler instead.

this


Cannot determine timezone from "VET"

@sofiabravo103 wrote in gh-222 :

Hi, I got the cannot determine timezone from nil error and added the fixes suggested in de faqs but the error continuous in the line where I start rufus:

Here is my code:

ENV['TZ'] = 'America/Caracas'
Rufus::Scheduler.singleton.every(
<EMAIL_ADDRESS>  :times => repetitions, :first_at => @harvest_plan.start_date
) do
  CreateHarvestFromPlanJob.perform_later
end

And here is the error

ArgumentError (cannot determine timezone from "VET (etz:"America/Caracas",tnz:"VET",tzid:"constant")):

I am running Rails 4.2.4, Ruby 2.2.2 and Rufus 3.3.2. (It was also me on stackoverflow about issue #231 )

What's going on?

@sofiabravo103 Hello
What operating system are you running on?
What is the "de faq" thing you are referring to? Can you please include a link to it?
Thanks in advance

And also, may I have the complete error backtrace?
Thanks in advance!

Ubuntu 14.04.5 LTS.
I don't think it will help, but here is the complete error backtrace:

ArgumentError (cannot determine timezone from nil (etz:"Caracas",tnz:"Caracas",tzid:"constant")):
  app/controllers/harvest_plans_controller.rb:36:in `schedule_harvest'
  app/controllers/harvest_plans_controller.rb:17:in `create'

Thanks.
May I please see the code of your harvest_plans_controller.rb especially around line 36 ? (so says your backtrace).

You have it, it's this line:
Rufus::Scheduler.singleton.every<EMAIL_ADDRESS>Also "de faqs" was a typo, sorry, I meant to say "the faqs", as in frequently asked questions in the README.

OK, so it's not the complete error backtrace, Rails must be filtering it for you. Else you would see the call to Rufus::Scheduler::ZoTime#initialize where the ArgumentError is raised.
Getting inspiration from gh-230, could you please try with:
ENV['TZ'] = Time.zone.name
Rufus::Scheduler.singleton.every(
<EMAIL_ADDRESS>  :times => repetitions, :first_at => @harvest_plan.start_date
) do
  CreateHarvestFromPlanJob.perform_later
end

So that it will pick Rails's defined timezone (I've read in #230 that it's set in Rails application.rb).

Already did, almost same error:
ArgumentError (cannot determine timezone from nil (etz:"Caracas",tnz:"Caracas",tzid:"constant")):

What does
p Time.zone.class
p Time.zone
ENV['TZ'] = Time.zone.name
Rufus::Scheduler.singleton.every(
<EMAIL_ADDRESS>  :times => repetitions, :first_at => @harvest_plan.start_date
) do
  CreateHarvestFromPlanJob.perform_later
end

yield?
I am interested in the ps precise output.

2.2.2 :001 > p Time.zone.class
ActiveSupport::TimeZone
 => ActiveSupport::TimeZone

2.2.2 :002 > p Time.zone
#<ActiveSupport::TimeZone:0x0000000555b860 @name="Caracas", @utc_offset=nil, @tzinfo=#<TZInfo::TimezoneProxy: America/Caracas>, @current_period=#<TZInfo::TimezonePeriod: #<TZInfo::TimezoneTransitionDefinition: #<TZInfo::TimeOrDateTime:<PHONE_NUMBER>>,#<TZInfo::TimezoneOffset: -14400,0,VET>>,nil>>
 => #<ActiveSupport::TimeZone:0x0000000555b860 @name="Caracas", @utc_offset=nil, @tzinfo=#<TZInfo::TimezoneProxy: America/Caracas>, @current_period=#<TZInfo::TimezonePeriod: #<TZInfo::TimezoneTransitionDefinition: #<TZInfo::TimeOrDateTime:<PHONE_NUMBER>>,#<TZInfo::TimezoneOffset: -14400,0,VET>>,nil>>

But this output might be different from what would have been when I reported the issue. I did an Ubuntu upgrade and it cleared the ArgumentError. This timezone is really peculiar, it has changed twice in recent time (last one in 2016), and not as in daylight saving or anything normal like that. It was -4:30 at the start of 2016 and now it's -4. The Venezuelan president changes the timezone for political reasons whenever he feels in the mood. I though the timezone issues could be fixed with an OS upgrade, and I think they did. I don't know if you still want to fix the ArgumentError, but if you do want to feel free to ask me anything else. Just wanted lo let you know that my code is working now.

Just wanted lo let you know that my code is working now.

I am glad to learn that I spent two hours of my life on a problem that is not my fault.

Great attitude. In my opinion it is still not ok to see an ArgumentError from your code, that should not be the way to find out there is an issue with timezone. I figured it based on a guess, could have been on this problem for weeks.

Great attitude

Please elaborate.

In my opinion it is still not OK to see an ArgumentError from your code

What should it do then?

But thanks for helping out anyway.

That's the "thanks" I was hoping for.

I'm hitting a similar issue on OSX. Interestingly enough, this error started showing yesterday March 13th 2016 when daylight savings time switched.
foreman run bundle exec sidekiq -q default
2017-03-15T06:11:36.260Z 1288 TID-oxdf05wxk INFO: Running in ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin15]
2017-03-15T06:11:36.260Z 1288 TID-oxdf05wxk INFO: See LICENSE and the LGPL-3.0 for licensing details.
2017-03-15T06:11:36.260Z 1288 TID-oxdf05wxk INFO: Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org
2017-03-15T06:11:36.260Z 1288 TID-oxdf05wxk INFO: Booting Sidekiq 4.2.7 with redis options {:url=>"redis://<IP_ADDRESS>"}
2017-03-15T06:11:36.262Z 1288 TID-oxdf05wxk INFO: Loading Schedule
2017-03-15T06:11:36.262Z 1288 TID-oxdf05wxk INFO: Scheduling update_sitemap_job {"cron"=>"0 0 2 * * * America/Los_Angeles", "class"=>"UpdateSitemapJob", "queue"=>"default"}
2017-03-15T06:11:36.597Z 1288 TID-oxdf05wxk WARN: {"context":"Exception during Sidekiq lifecycle event.","event":"startup"}
2017-03-15T06:11:36.597Z 1288 TID-oxdf05wxk WARN: ArgumentError: cannot determine timezone from nil (etz:nil,tnz:"PDT",tzid:nil)
2017-03-15T06:11:36.597Z 1288 TID-oxdf05wxk WARN: /Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rufus-scheduler-3.3.2/lib/rufus/scheduler/zotime.rb:41:in `initialize'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rufus-scheduler-3.3.2/lib/rufus/scheduler/zotime.rb:198:in `new'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rufus-scheduler-3.3.2/lib/rufus/scheduler/zotime.rb:198:in `now'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rufus-scheduler-3.3.2/lib/rufus/scheduler.rb:561:in `start'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rufus-scheduler-3.3.2/lib/rufus/scheduler.rb:109:in `initialize'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-scheduler-2.1.0/lib/sidekiq/scheduler.rb:365:in `new'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-scheduler-2.1.0/lib/sidekiq/scheduler.rb:365:in `new_rufus_scheduler'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-scheduler-2.1.0/lib/sidekiq/scheduler.rb:193:in `rufus_scheduler'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-scheduler-2.1.0/lib/sidekiq/scheduler.rb:375:in `new_job'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-scheduler-2.1.0/lib/sidekiq/scheduler.rb:114:in `block in load_schedule_job'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-scheduler-2.1.0/lib/sidekiq/scheduler.rb:107:in `each'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-scheduler-2.1.0/lib/sidekiq/scheduler.rb:107:in `load_schedule_job'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-scheduler-2.1.0/lib/sidekiq/scheduler.rb:68:in `block in load_schedule!'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-scheduler-2.1.0/lib/sidekiq/scheduler.rb:66:in `each'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-scheduler-2.1.0/lib/sidekiq/scheduler.rb:66:in `load_schedule!'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-scheduler-2.1.0/lib/sidekiq-scheduler/manager.rb:29:in `start'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-scheduler-2.1.0/lib/sidekiq-scheduler.rb:33:in `block (2 levels) in <top (required)>'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-4.2.7/lib/sidekiq/util.rb:54:in `call'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-4.2.7/lib/sidekiq/util.rb:54:in `block in fire_event'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-4.2.7/lib/sidekiq/util.rb:52:in `each'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-4.2.7/lib/sidekiq/util.rb:52:in `fire_event'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-4.2.7/lib/sidekiq/cli.rb:79:in `run'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/sidekiq-4.2.7/bin/sidekiq:12:in `<top (required)>'
/Users/mosca1337/.rbenv/versions/2.2.3/bin/sidekiq:23:in `load'
/Users/mosca1337/.rbenv/versions/2.2.3/bin/sidekiq:23:in `<top (required)>'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.13.7/lib/bundler/cli/exec.rb:74:in `load'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.13.7/lib/bundler/cli/exec.rb:74:in `kernel_load'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.13.7/lib/bundler/cli/exec.rb:27:in `run'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.13.7/lib/bundler/cli.rb:332:in `exec'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.13.7/lib/bundler/cli.rb:20:in `dispatch'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.13.7/lib/bundler/cli.rb:11:in `start'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.13.7/exe/bundle:34:in `block in <top (required)>'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.13.7/lib/bundler/friendly_errors.rb:100:in `with_friendly_errors'
/Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.13.7/exe/bundle:26:in `<top (required)>'
/Users/mosca1337/.rbenv/versions/2.2.3/bin/bundle:23:in `load'
/Users/mosca1337/.rbenv/versions/2.2.3/bin/bundle:23:in `<main>'

@mosca1337 please read https://github.com/jmettraux/rufus-scheduler#i-get-zotimerb41in-initialize-cannot-determine-timezone-from-nil
Thanks!

@mosca1337 you might also be interested in testing what is currently on master, I have put on some detection code for the system's current timezone, as seen in https://github.com/jmettraux/rufus-scheduler/issues/240
Tell me if that change fixes the issue for you.
Thanks in advance.

TZ="America/Los_Angeles" rails console
Running via Spring preloader in process 1471
Loading development environment (Rails 4.2.6)
[1] pry(main)> Rufus::Scheduler::ZoTime.now
=> #<Rufus::Scheduler::ZoTime:0x007ff4301c3c58 @seconds=1489558877.130253, @time=nil, @zone=#<TZInfo::DataTimezone: America/Los_Angeles>>

rails console
Running via Spring preloader in process 1486
Loading development environment (Rails 4.2.6)
[1] pry(main)> Rufus::Scheduler::ZoTime.now
ArgumentError: cannot determine timezone from nil (etz:nil,tnz:"PDT",tzid:nil)
from /Users/mosca1337/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rufus-scheduler-3.3.2/lib/rufus/scheduler/zotime.rb:41:in `initialize'

Resolved temporarily with:
TZ="America/Los_Angeles" foreman run worker
2017-03-15T06:22:41.089Z 1504 TID-ovsbjpts0 INFO: Running in ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin15]
2017-03-15T06:22:41.089Z 1504 TID-ovsbjpts0 INFO: See LICENSE and the LGPL-3.0 for licensing details.
2017-03-15T06:22:41.089Z 1504 TID-ovsbjpts0 INFO: Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org
2017-03-15T06:22:41.089Z 1504 TID-ovsbjpts0 INFO: Booting Sidekiq 4.2.7 with redis options {:url=>"redis://<IP_ADDRESS>"}
2017-03-15T06:22:41.092Z 1504 TID-ovsbjpts0 INFO: Loading Schedule

The daylight savings time change may have been a very odd coincidence?

Upgrading from 3.3.2 to 3.3.4 solved this (should have checked for updates first). Thanks for the quick help!
Installing rufus-scheduler 3.3.4 (was 3.3.2)

foreman run worker
2017-03-15T06:25:20.877Z 1548 TID-ovnexw9ok INFO: Running in ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin15]
2017-03-15T06:25:20.877Z 1548 TID-ovnexw9ok INFO: See LICENSE and the LGPL-3.0 for licensing details.
2017-03-15T06:25:20.877Z 1548 TID-ovnexw9ok INFO: Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org
2017-03-15T06:25:20.877Z 1548 TID-ovnexw9ok INFO: Booting Sidekiq 4.2.7 with redis options {:url=>"redis://<IP_ADDRESS>"}
2017-03-15T06:25:20.880Z 1548 TID-ovnexw9ok INFO: Loading Schedule

OK, you're welcome.


feat(timtetable state indicators): timtetable state indicators, open,…

… closed or no indicator

:tada: This PR is included in version 2.3.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.3.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Incorrect Return code received while performing Count Validation- BQ vs BQ

Description: While running below YAML for Count validation, Source table wasn't existing because  of which validation failed but return code received was 0 for DVT ver 2.9.0.Whereas for version 2.6.0 for the same error return code was 0
YAML:
result_handler:
project_id: rite-aid-pocs
table_id: ra_data_validation.results
type: BigQuery
source: BQ_CONN
target: BQ_CONN
validations:

aggregates:

field_alias: count
source_column: null
target_column: null
type: count
custom_query_type: column
filter_status: null
filters: []
format: table
labels:
!!python/tuple

table
generic_launch_date

!!python/tuple

date
'{{load_dt}}'
random_row_batch_size: null
schema_name: null
source_query: select * from rite-aid-pocs.ra_landing_ext.generic_launch_date_ext_tb
where ra_load_dte = '{{load_dt}}'
table_name: null
target_query: select * from rite-aid-pocs.ra_landing.generic_launch_date where
ra_load_dte = '{{load_dt}}'
target_schema_name: null
target_table_name: null
threshold: 0.0
type: Custom-query
use_random_rows: null

Return code screenshot for version 2.9.0:
https://user-images.githubusercontent.com/129146107/228141337-a51847db-e1a6-442e-9525-fb2a80ade30b.png
Return code screenshot for version 2.6.0:
https://user-images.githubusercontent.com/129146107/228141219-d54a9261-6756-4c20-bc9a-07287931779c.png

Where in the code are you printing the return code for v2.9.0? Looks like both versions are throwing the same 404 table not found error

Hi Neha,
Here is the code chunk where we are printing the returncode:
"""
result=subprocess.Popen(["data-validation", "run-config", "-c", file_path], stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
output, error = result.communicate()
result.wait()
print("DVT execution done",output.decode('utf-8') ,flush=True)
print("returncode for DVT execution: ", result.returncode,flush=True)
"""
Code chunk:
https://user-images.githubusercontent.com/129146107/229042647-7864fa80-14d6-41fc-b79b-8fc6dc8fa6d8.png
Return code screenshot for version 2.9.0:
https://user-images.githubusercontent.com/129146107/228141337-a51847db-e1a6-442e-9525-fb2a80ade30b.png

We don't populate the return codes in DVT code, instead this is a result of the subprocess package and how it handles the calls. Please re-open if there is an issue with error handling with DVT. Thanks!


Preview v2 Enhancements

Please Select if your Request is Either something new or an Enhancement

[x] Enhancement of an existing Feature.
[ ] Request of a new feature.

Please select the area your request applies to. (Multiple selections are Possible)

[ ] Onboard. Initial vault setup and import/export pods
[ ] Retrieve. Backlinks, references, graph view
[ ] Structure. Refactoring, multi-vault and schemas
[ ] Create. Note creation, lookup, snippets and templates
[ ] Publish. Sharing your repo with the world
[x] Misc (Choose this if your not sure)

Is your feature request related to a problem? Please describe
As described in discord, preview v2 lacks some features compare to v1, just gonna address them here for the team to track them in the roadmap.
Describe the solution you'd like

Navigate with headers: in v1 we have a TOC generated when clicking the icon at the top right hand side of the preview
Export to pdf: right click and choose options to export pdf in the directory
Scroll Sync
Custon CSS

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Thanks for the report. We will be working on these items in upcoming releases. Will update this ticket when work starts

Related:

https://github.com/dendronhq/dendron/issues/1171
https://github.com/dendronhq/dendron/issues/1937


Add test-coverage; pkgdown; R-CMD-check workflows

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@8fb70a0). Click here to learn what that means.
The diff coverage is n/a.

:exclamation: Current head 705ce9d differs from pull request most recent head d4d8d94. Consider uploading reports for the commit d4d8d94 to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@           Coverage Diff            @@
##             main       #20   +/-   ##
========================================
  Coverage        ?   100.00%
========================================
  Files           ?         5
  Lines           ?       100
  Branches        ?         0
========================================
  Hits            ?       100
  Misses          ?         0
  Partials        ?         0

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Support avaje-inject aspects on generated http-client interfaces.

I got a situation like this using the new Scoped Values.
      // I have a custom logger that will log MDC from Scoped Values
      final var mdc =
          MDC_SV.isBound() ? new HashMap<>(MDC_SV.get()) : new HashMap<String, String>();
      mdc.put("Downstream-API", "api call");

      final var response =
          ScopedValue.callWhere(mdc, () -> apiInterface.call(param, "application/json", "web").toString());

I'd like to just create an aspect and have that take care of running a request in an SV context instead of repeating myself.
I think that for client interfaces annotated with @Component we could add @Component and the aspect annotations to the generated HTTP client. In this way, the inject generator can create DI/Proxy classes.

inject generator can create DI/Proxy classes

The http client instance is created by ServiceLoader and not via DI, so to me a avaje-inject proxy isn't going to be useful here ... BUT ... http client already has API for wrapping requests - RequestIntercept so to me that is the first thing to look at for adding generic before/after logic for http client requests?

Problem is that SVs don't work like threadlocals, you can't just set it and do something somewhere else, it will only be bound inside the execution of the runnable/callable/supplier provided.

inject generator can create DI/Proxy classes

The http client instance is created by ServiceLoader and not via DI, so to me a avaje-inject proxy isn't going to be useful here

I'm essentially proposing an alternative way to create an http client instance, if you add DI annotations you can instanciate via an http client in the beanscope, instead of service loader

@SentryMan  Should this issue be closed by the PR that was just merged?  I'll re-open for now as I'm not sure.

Yeah, DI and Aspect annotations are copied to the generated client, causing inject to generate proxy/DI classes


feat(HACBS-1232): create ephemeral Env/Binding

Signed-off-by: jsztuka<EMAIL_ADDRESS>Adding a logic for creation of ephemeral environments.
Order of logic:
Integration service checks integrationTestScenario for presence of env attribute.
When the attribute exists - it check if the environment exist.
In case it does - create copy of this environment with updated envVars from IntegrationTestScenario.
Envs are created by this pattern: IntegrationTestScenario x snapshot
Corresponding labels are attached to newly created environment - snapshot, integrationTestScenario
For each snapshot the environmentbinding is created also.
Example targetNamespace that was created by process: example-its-example-b81eabe8-6e53-425c-970c-0eb0baa0214b

Issue that is happening in Check sources part of checks is being fixed in: https://github.com/redhat-appstudio/integration-service/pull/73

After having a discussion with Jonathan West, it turned out that gitops is not creating TargetNamespaces at the moment.
There has been created story regarding this behavior: https://issues.redhat.com/browse/GITOPSRVCE-304


Arabic Translation

Language To Add
Arabic
I'm native Arabic speaker and I'm prepared to provide translations for it.

Same here 😊

Hi @Haboubi92 & @taha-mahmoud-ly,
Arabic should already be available as an option within the application and via crowdin: https://crowdin.com/project/bookstack
Let me know if I'm missing something though.

Closing as there has been no further clarification.


Min Max issue

clamp() won't allow MIN > MAX, so I added a min and max check to flip the values if the small breakpoint font size is larger than its large breakpoint counterpart.

I think I'll not merge the PR because of my latest comment here: https://github.com/davidhellmann/tailwindcss-fluid-type/issues/26
But thanks for investigating


Using LibreDrive for dumping discs

At https://forum.makemkv.com/forum/viewtopic.php?t=18856 one can read about the possibilities of LibreDrive, as in accessing any disc on a very basic level.
Couldn't DiscImageCreator, when using the LibDriveIo library, dump nearly everything without using special tricks? Or wouldn't it going to work with game CDs from video game consoles?

Did you see the src code? https://forum.makemkv.com/forum/viewtopic.php?f=3&t=224
At least, it's not useful for my app, I think.

dump nearly everything without using special tricks?

Special trick, I don't know what it is.

I meant like for GD discs for example. From how it read in the MakeMKV forum, it sounded like with LibreDrive one could also access such things with "unspecial" drives. Was just a thought :)


Respect system mode

What this PR does / why we need it?
If the user's system is dark, the editor theme is dark when user first visited
Any background context you want to provide?
What are the relevant tickets?

Fixes #92
Checklist

[X] Added relevant tests or not required
[X] Didn't break anything

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Phosh does not start g-s-d

Phosh does not have g-s-d available, resulting in the inability to change the screen brightness using the slider in the top bar. The following error is logged:
Failed to start org.gnome.SettingsDaemon.Power.service: Operation refused, unit org.gnome.SettingsDaemon.Power.service may be requested by dependency only (it is configured to refuse manual start/stop).

It appears that since we use systemd for gnome-session, it does not start g-s-d:
May 19 21:30:53 nixos gnome-session-binary[1184]: gnome-session-binary[1184]: DEBUG(+): app /org/gnome/SessionManager/App7 is disabled by X-GNOME-HiddenUnderSystemd
May 19 21:30:53 nixos gnome-session-binary[1184]: gnome-session-binary[1184]: DEBUG(+): GsmManager:	ID: /org/gnome/SessionManager/App7	app-id:org.gnome.SettingsDaemon.Power.desktop	is-disabled:1	is-conditionally-disabled:0

Maybe we need to start phosh using systemd too. There is a service upstream.
Though for GNOME Shell, g-s-d is started by<EMAIL_ADDRESS>and I see no such Wants in the upstream phosh.service. Perhaps we would need to add Wants that mirror the RequiredComponents from phosh.session.
cc @zhaofengli @dotlambda @samueldr

I've been using the systemd service, which in turn is inspired by that upstream service.

https://github.com/NixOS/mobile-nixos/pull/352/files#diff-d54a364f462ceeb609e5441a05525e70355cef100366f30c50b80a70e2dc4827R38-R61

I'm really not familiar with the systemd + user session stuff here, and I tried adding all the Wants from systemctl --user cat graphical-session.target to that session, but it doesn't start things.
systemd.services.phosh = {
  wantedBy = [ "graphical.target" ];
  serviceConfig = {
    ExecStart = "${pkgs.phosh}/bin/phosh";
    User = 1000;
    PAMName = "login";
    WorkingDirectory = "~";

    # XXX
    TTYPath = "/dev/tty7";
    TTYReset = "yes";
    TTYVHangup = "yes";
    TTYVTDisallocate = "yes";

    StandardInput = "tty-fail";
    StandardOutput = "journal";
    StandardError = "journal";

    # XXX
    UtmpIdentifier = "tty7";
    UtmpMode = "user";

    Restart = "always";
  };
  wants = [
    # Workaround for: https://github.com/NixOS/nixpkgs/issues/123735
    # Must be in sync with gnome.session
    # systemctl --user cat<EMAIL_ADDRESS>    "org.gnome.SettingsDaemon.A11ySettings.target"
    "org.gnome.SettingsDaemon.Color.target"
    "org.gnome.SettingsDaemon.Datetime.target"
    "org.gnome.SettingsDaemon.Housekeeping.target"
    "org.gnome.SettingsDaemon.Keyboard.target"
    "org.gnome.SettingsDaemon.MediaKeys.target"
    "org.gnome.SettingsDaemon.Power.target"
    "org.gnome.SettingsDaemon.PrintNotifications.target"
    "org.gnome.SettingsDaemon.Rfkill.target"
    "org.gnome.SettingsDaemon.ScreensaverProxy.target"
    "org.gnome.SettingsDaemon.Sharing.target"
    "org.gnome.SettingsDaemon.Smartcard.target"
    "org.gnome.SettingsDaemon.Sound.target"
    "org.gnome.SettingsDaemon.UsbProtection.target"
    "org.gnome.SettingsDaemon.Wacom.target"
    "org.gnome.SettingsDaemon.XSettings.target"
  ];
};

I assume this is a split from between user and non-user services. As phosh is a system service this way.
Is the solution to have this be a user service? Though that wouldn't help with a gdm-started session, right?

Yeah, it would have to be a user service.
And unless gnome-session is informed that phosh supports systemd, it will continue using the session file. So if you want the service to be used, you will have to start it manually using systemctl start --user phosh.
I am not sure what exactly it would take, here is a blog post describing the transition and a MR. I also asked at the Phosh channel.

Ok, I gave it a try and systemd session management seems to be working. There are two patches to be upstreamed (waiting for Purism to approve my account creation), and I'm not entirely sure what is the correct way to start phoc (currently it sits outside the session). My branch is here, and it can be tested with the following configurations:
{
  systemd.services.phosh-autostart = {
    wantedBy = [ "graphical.target" ];
    serviceConfig = {
      ExecStart = "${pkgs.phosh}/bin/phosh";
      User = 1000;
      PAMName = "login";
      WorkingDirectory = "~";

      TTYPath = "/dev/tty7";
      TTYReset = "yes";
      TTYVHangup = "yes";
      TTYVTDisallocate = "yes";

      StandardInput = "tty-fail";
      StandardOutput = "journal";
      StandardError = "journal";

      UtmpIdentifier = "tty7";
      UtmpMode = "user";

      Restart = "always";
    };

    environment = {
      ENABLE_SYSTEMD_SESSION = "1";
    };
  };
}

I believe those systemd user services should, somehow, be part of the NixOS phosh configuration.
I don't know how this could be handled, but ideally enabling posh should, by default, enable this. Or if not by default enabled, a second option like phosh.enableUserService = true.
@jtojnar anything against doing this way? Not sure if this can cause trouble.

Phosh is already in systemd.packages so I would expect it to work.


Make the status bar transparent on SessionDetailActivity

Overview (Required)
This change caused the color of the status bar to be dark_theme.
I fixed the appearance by making it transparent for post-Lollipop. 🌀
Screenshot

Before
After

:eyes:

LGTM! 💯 Thanks!!


Running Nuxt 3 development server with debugging mode not available?

Environment

Operating System: Darwin
Node Version:     v18.16.0
Nuxt Version:     3.5.0
Nitro Version:    2.4.1
Package Manager<EMAIL_ADDRESS>Builder:          vite
User Config:      srcDir, app, alias, css, modules, plugins, imports, logLevel, vite
Runtime Modules:  ~/modules/eventHandler/index.js, ~/modules/eventBus/index.js, ~/modules/router/index.js<EMAIL_ADDRESS>Build Modules:    -

Reproduction
Run below command:
node --inspect node_modules/.bin/nuxt dev

Describe the bug
I want to start Nuxt development server while enabling the debug mode. For usual case if I run a node server with debug flag, I get a prompt message something like below:
Debugger listening on ws://<IP_ADDRESS>:9229/081fbb8d-b3e0-4ef5-92e6-54242327a280

And also by that point I can find the debugging tool for the server is activated by looking at the chrome inspector tool:

But if I run that command with Nuxt 3 (3.5.0), it runs with same config as I run just nuxt dev. This did work with Nuxt 2 though, is there a change with how to run Nuxt server in debugging mode?
Additional context
No response
Logs
No response

Probably it was a temporary issue, it doesn't reproduce. I'll close this issue, sorry about that.

It looks like the nuxt(node) server does run in debug mode, but whan I open the devtool it shows only blank browser screen, and it doesn't detect any debugger point on the code.
I ran the below script to run the nuxt server, is this wrong usage?
node --inspect node_modules/.bin/nuxt dev

Also, as soon as I run the node server with the script above, the console shows the message below:
Debugger listening on ws://<IP_ADDRESS>:9229/e32af443-7ba6-4179-b8b6-3ba4c1f70ab9
For help, see: https://nodejs.org/en/docs/inspector
Starting inspector on <IP_ADDRESS>:9229 failed: address already in use

Is this saying that the inspector is not run properly? If so, what shall I check to make it run correctly?

Would you provide a reproduction?
It's also worth checking you don't already have another project running with debugger? (Does this still reproduce after restarting your computer?)

@danielroe
Unfortunately it seems that is not the case. I also checked the ports occupied, but there wasn't 9229 port on the list. I restarted my machine and ran the script again, only finding out the same result.

@danielroe I provided reproduction, if you run the command you will see the above message and outcomes.

For those who need nuxt server log which you can find on Node devtools, you can get some infomation by putting debug props to true in nuxt.config.js:
// nuxt.config.js
export default defineNuxtConfig({
    ...
    debug: true,
    ...
});

But even with this enabled, I can't get enough log message i.e. runtime error log at server-side.

Have you tried this? https://nuxt.com/docs/guide/going-further/debugging#debugging-in-your-ide
This should definitely work.

I'm running into a similar issue with Nuxt v3.5.2 when trying to attach the debugger to the node process.
I was previously able to attach the vscode debugger to the Nuxt process directly using this launch configuration:
{
      "type": "node",
      "request": "attach",
      "processId": "${command:PickProcess}",
      "name": "server: attach to nuxt"
}

I can see the debugger attaching in my terminal, but the break points will not bind in vscode.
Now I'm only able to get the debugger to work when using the fullstack: nuxt compound configuration provided by the link @danielroe provided that launches the Nuxt dev server in the vscode integrated terminal:

Have you tried this? https://nuxt.com/docs/guide/going-further/debugging#debugging-in-your-ide
This should definitely work.

I have the same problem @danielroe

Thanks for the deep dive! this should be straightforward to fix if someone wants to give it a try.


Sound and music

Add player laser sound.
Add player thrust sound.
Add explosion, levelup, music sound.
Add enemy laser sound.

https://github.com/user-attachments/assets/668f41d4-2f15-4d43-a874-26e1c262c41d