ROX-17374: Change dependency discovery method

We have a problem that downstream builds are failing because Cachito is not storing the information required for ossls to work.
We are using go list -m command to discover golang dependencies. This command requires *.info files, which are no longer persisted in Cachito.
This PR is adding change to the method of how dependencies are discovered and fixing two additional problems.
Changes
Change 1 - filter packages without directory
This is a bug discovered with the previous functionality. We have a lot of modules like this:
{
   "Path": "sigs.k8s.io/kustomize/kustomize/v4",
   "Version": "v4.5.4",
   "Time": "2022-03-28T22:46:23Z",
   "Indirect": true
}

Because of a bug in reading JSON, such records would keep Dir from the last module with Dir defined. The outcome of such an operation is that we reported a license from package_1 under the package_2 name.
Example:
Proj: "github.com/steveyen/gtreap" - Dir<EMAIL_ADDRESS>Proj: "github.com/stoewer/go-strcase" - Dir<EMAIL_ADDRESS>Proj: "github.com/streadway/amqp" - Dir<EMAIL_ADDRESS>Proj: "github.com/streadway/handy" - Dir<EMAIL_ADDRESS>
You can see that github.com/streadway/amqp will have a license from github.com/steveyen/gtreap.
Change 2 - use go mod download to get modules
Based on the investigation made here: https://github.com/chmeliik/godeps/tree/main#module-identification-methods-vs-corner-cases - there are several methods how to get dependencies. go mod download is problematic for replacements that are using local directories. I have checked stackrox and scanner, and we don't have such cases in these repositories.
We don't have information about Replace from go mod. We will simply get the package used as a replacement. And that is correct.
Change 3 - report licenses for github.com/stackrox/ repos
This is mostly relevant for replacements. We had a problem in the previous solution. We would report the license from a stackrox repository as it is a license from a replaced repository. This is incorrect. If we want to handle licenses properly - we should simply report them (also from our packages). And inside our packages, we should define the license accordingly.
Now we will have at least licenses from stackrox repos because they are actually used to build products. And not replaced libraries.
Testing
Test on stackrox/stackrox:

build ossls - go build -o ossls
use to generate report ossls audit --export ./THIRD_PARTY_NOTICES_report
check results

Also, test PR for downstream is open:
https://gitlab.cee.redhat.com/stackrox/rhacs-midstream/-/merge_requests/1399
Made comparisons
Cmp 1 - list -json -m all vs mod download -json
Get list -json -m all results.
Get mod download -json results.
Step 1:
Remove all wrongly reported packages (packages that are not available - no Dir for them) from list -json -m all results. There is still a difference between list -json -m all and mod download -json.
Step 2:
Get diff between step 1 version of list -json -m all and mod download -json. All diff packages are dependencies used in tests of our dependency packages. They are not direct test dependencies. We are not shipping these libraries, and there is no need to report their licenses.
Cmp 2 - go-licenses vs mod download -json
I used https://github.com/google/go-licenses - to get results. There have been some differences. Some libraries were not reported by go-licenses. But some additional licenses have been reported.
An example of a reported license by go-licenses: sigs.k8s.io/kustomize/kyaml/internal/forked/github.com/go-yaml/yaml - this is a case where project has multiple licenses depending on used packages.
We didn't report these with the previous solution, and it's not a regression introduced with this functionality.
An example of not reported license by go-licenses: cloud.google.com/go/compute - this is used by some tools we are using in Makefile. So it does not hurt to list it if it's used during the build of the project.

@mtodor any plans to cut a new release with these latest changes? I think the current CI images are using a version of this repo without these changes


feat(cicd): Add --forbid-only in mocha config

Closes #3818
Add the --forbid-only argument to mocha when running tests on the build server to give a more concrete warning when you've accidentally pused a it.only() test.
Before:
https://github.com/stryker-mutator/stryker-js/actions/runs/3347971096/jobs/5546534847#step:5:2780
After:
https://github.com/Giovds/stryker-js/actions/runs/3355540113/jobs/5559880282#step:5:918

Ah this actually wasn't a feature. By marking it as feat it got lumped into the changelog 😅 Fixing now.


OpenID 2.0 seems to be out of date

Trying to login with google lands on this page
https://support.google.com/accounts/answer/6206245?p=openid&rd=1

Google phased it out, yes. But there are other providers out there besides Google.

I guess this is actually a flask-open id feature request :)


LLM integration

Simple LLM implementation prototype using UIUC.chat.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 3 committers have signed the CLA.:white_check_mark: lmarini:x: yiranz:x: Kleokhovyiranz seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


nckw alternative attempt to fix dataset from alternative file

Attempt to fix dataset name option
Distinguish between --dataset and --dataMapName for combine and text2workspace respectively.
--dataset  (or -D) allows the user to specify an alternative dataset (eg a toy) to run at the command line while --dataMapName changes the default (data_obs) dataset name in the datacard mapping name.
The latter is not passed to text2workspace via command line now when running with datacard as command line input to combine (hence why this fixes the issue) so the user would need to pass via options. Not clear if this is used ever however.

Attempt to resolve issue #828

So since this PR also includes changes from #817 we should be prepared to release a revision if we merge this one, right @kcormi?

Yes, I guess we should decide what we'd like to have in a minor release and then make sure we update the documentation as nick mentioned in #817.

Sorry guys,
My fault that I let these changes slip in this PR. Let me know if you
prefer me to revert those changes to the docs here and separate out the
intended ones.
Nick
On Wed, 29 Mar 2023, 15:50 kcormi, @.***> wrote:

So since this PR also includes changes from #817
https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/pull/817 we
should be prepared to release a revision if we merge this one, right
@kcormi https://github.com/kcormi?
Yes, I guess we should decide what we'd like to have in a minor release
and then make sure we update the documentation as nick mentioned in #817
https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/pull/817.
—
Reply to this email directly, view it on GitHub
https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/pull/832#issuecomment-1488774999,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAMEVWZT6JN5GH6LX3GIKYDW6RD33ANCNFSM6AAAAAAWJOTYK4
.
You are receiving this because you authored the thread.Message ID:
@.***>

No problem Nick,
I don't know how you have historically decided when to do new releases, but I think we could just do a minor one soon. Then it is easy enough to either merge #817 first, or just close it and only merge this one. So I don't think its an issue.
Cheers,
Kyle


Fix aiohttp client

ERROR:    Exception in ASGI application
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/aiohttp/connector.py", line 986, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)  # type: ignore[return-value]  # noqa
  File "/Users/jochen/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py", line 1064, in create_connection
    raise exceptions[0]
  File "/Users/jochen/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py", line 1049, in create_connection
    sock = await self._connect_sock(
  File "/Users/jochen/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py", line 960, in _connect_sock
    await self.sock_connect(sock, address)
  File "/Users/jochen/.pyenv/versions/3.10.4/lib/python3.10/asyncio/selector_events.py", line 499, in sock_connect
    return await fut
  File "/Users/jochen/.pyenv/versions/3.10.4/lib/python3.10/asyncio/selector_events.py", line 534, in _sock_connect_cb
    raise OSError(err, f'Connect call failed {address}')
ConnectionRefusedError: [Errno 61] Connect call failed ('<IP_ADDRESS>', 5001)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/jochen/.pyenv/versions/3.10.4/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/Users/jochen/.pyenv/versions/3.10.4/lib/python3.10/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/Users/jochen/projects/will_it_saturate/will_it_saturate/clients.py", line 189, in measure_in_new_process
    elapsed, responses = asyncio.run(self.measure_server(epoch))
  File "/Users/jochen/.pyenv/versions/3.10.4/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Users/jochen/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/Users/jochen/projects/will_it_saturate/will_it_saturate/clients.py", line 184, in measure_server
    responses = await asyncio.gather(*tasks)
  File "/Users/jochen/projects/will_it_saturate/will_it_saturate/clients.py", line 168, in fetch_page
    async with session.get(url) as response:
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/aiohttp/client.py", line 535, in _request
    conn = await self._connector.connect(
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/aiohttp/connector.py", line 542, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/aiohttp/connector.py", line 907, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/aiohttp/connector.py", line 1206, in _create_direct_connection
    raise last_exc
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/aiohttp/connector.py", line 1175, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/aiohttp/connector.py", line 992, in _wrap_create_connection
    raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host localhost:5001 ssl:default [Connect call failed ('<IP_ADDRESS>', 5001)]
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 372, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/fastapi/applications.py", line 261, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/routing.py", line 656, in __call__
    await route.handle(scope, receive, send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/routing.py", line 259, in handle
    await self.app(scope, receive, send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/routing.py", line 61, in app
    response = await func(request)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/fastapi/routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/fastapi/routing.py", line 162, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/anyio/to_thread.py", line 28, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(func, *args, cancellable=cancellable,
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 818, in run_sync_in_worker_thread
    return await future
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 754, in run
    result = context.run(func, *args)
  File "/Users/jochen/projects/will_it_saturate/will_it_saturate/control/main.py", line 85, in measure
    return benchmark_client.measure(epoch)
  File "/Users/jochen/projects/will_it_saturate/will_it_saturate/clients.py", line 197, in measure
    [result, timestamps] = p.map(self.measure_in_new_process, [epoch])
  File "/Users/jochen/.pyenv/versions/3.10.4/lib/python3.10/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/Users/jochen/.pyenv/versions/3.10.4/lib/python3.10/multiprocessing/pool.py", line 771, in get
    raise self._value
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host localhost:5001 ssl:default [Connect call failed ('<IP_ADDRESS>', 5001)]

This is the correct traceback:
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 372, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/fastapi/applications.py", line 261, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/routing.py", line 656, in __call__
    await route.handle(scope, receive, send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/routing.py", line 259, in handle
    await self.app(scope, receive, send)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/routing.py", line 61, in app
    response = await func(request)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/fastapi/routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/fastapi/routing.py", line 162, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/anyio/to_thread.py", line 28, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(func, *args, cancellable=cancellable,
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 818, in run_sync_in_worker_thread
    return await future
  File "/Users/jochen/.virtualenvs/wis/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 754, in run
    result = context.run(func, *args)
  File "/Users/jochen/projects/will_it_saturate/will_it_saturate/control/main.py", line 85, in measure
    return benchmark_client.measure(epoch)
  File "/Users/jochen/projects/will_it_saturate/will_it_saturate/clients.py", line 197, in measure
    [result, timestamps] = p.map(self.measure_in_new_process, [epoch])
ValueError: not enough values to unpack (expected 2, got 1)


delete project after all property should not scale down services

ts.kubernetes.delete.project.after.all=false should not scale down k8s/ocp services
Also must only works if ts.kubernetes.ephemeral.namespaces.enabled=false
Close #342

Pinpong serverless scenario OpenShiftUsingExtensionAndDockerBuildAndServerlessPingPongResourceIT is failling by an error not related to this PR
SEVERE Caught exception while closing extension context: org.junit.jupiter.engine.descriptor.ClassExtensionContext@14823f76: javax.net.ssl.SSLException: Unsupported or unrecognized SSL message


Modify the texture of plants

I think it's great how the textures are, but I feel them somewhat weird some like the nutt and also that they have too many blocks the models making that in many quantities of a plant can cause bad optimization
If you want me to help you in what I told you my discord: Pailomot#5814

The latest update has improved textures for the Peashooter, Repeater, and Threepeater. I will try to improve the textures of other plants in the incoming updates.


[Error] Parsing error: "parserOptions.project" has been set for @typescript-eslint/parser

Hi~ I found an error descibed above, and my versions are listed below:

package
version

vscode-eslint
2.1.5

@typescript-eslint/parser
2.27.0

TypeScript
3.7.2

ESLint
6.8.0

vscode
1.44.1

Node.js
12.8.1

V8
7.8.279.23-electron.0

OS
Linux x64 4.13.0-36-generic

eslint configs:
module.exports = {
    parser: '@typescript-eslint/parser',
    parserOptions: {
        warnOnUnsupportedTypeScriptVersion: false,
        ecmaVersion: 6,
        sourceType: 'module',
        project: 'tsconfig.json',
    }
}

This error happens when I use symlink as my workspace dir, e.g.
/home/xxx/workspace/  -> /workspace
I tried to div into source code and found languageServer use symlink uri when file changed, e.g.
/home/xxx/workspace/src/index.js
but typescript-estree use extra.tsconfigRootDir = '/workspace' to createProjectProgram.
I also found a way to solve this problem:
function getFileSystemPath(uri: URI): string {
	const result = uri.fsPath;
	if (process.platform === 'win32' && result.length >= 2 && result[1] === ':') {
		// Node by default uses an upper case drive letter and ESLint uses
		// === to compare paths which results in the equal check failing
		// if the drive letter is lower case in th URI. Ensure upper case.
		return result[0].toUpperCase() + result.substr(1);
	} else {
		return fs.realpathSync(result); // I added fs.realpathSync to solve this problem
	}
}

Let me know if this solution is correct, thank you

Hi~ I found an error descibed above, and my versions are listed below:

package
version

vscode-eslint
2.1.5

@typescript-eslint/parser
2.27.0

TypeScript
3.7.2

ESLint
6.8.0

vscode
1.44.1

Node.js
12.8.1

V8
7.8.279.23-electron.0

OS
Linux x64 4.13.0-36-generic

eslint configs:
module.exports = {
    parser: '@typescript-eslint/parser',
    parserOptions: {
        warnOnUnsupportedTypeScriptVersion: false,
        ecmaVersion: 6,
        sourceType: 'module',
        project: 'tsconfig.json',
    }
}

This error happens when I use symlink as my workspace dir, e.g.
/home/xxx/workspace/  -> /workspace
I tried to div into source code and found languageServer use symlink uri when file changed, e.g.
/home/xxx/workspace/src/index.js
but typescript-estree use extra.tsconfigRootDir = '/workspace' to createProjectProgram.
I also found a way to solve this problem:
function getFileSystemPath(uri: URI): string {
	const result = uri.fsPath;
	if (process.platform === 'win32' && result.length >= 2 && result[1] === ':') {
		// Node by default uses an upper case drive letter and ESLint uses
		// === to compare paths which results in the equal check failing
		// if the drive letter is lower case in th URI. Ensure upper case.
		return result[0].toUpperCase() + result.substr(1);
	} else {
		return fs.realpathSync(result); // I added fs.realpathSync to solve this problem
	}
}

Let me know if this solution is correct, thank you

Actually I am not a fan of blindly making this change since it will affect quite some other code passes as well. Can you please elaborate what you think is the root cause of this and why the typescript-eslint/parser complain about it with a parsing error. This is absolutely not clear to me.

This error happend at @typescript-eslint/typescript-estree/src/createProjectProgram.ts line 35

extra = {
    tsconfigRootDir: '/workspace',
    filePath: '/home/xxx/workspace/src/index.ts'
   // ignore other properties
}

typescript-estree uses 'extra' opotions to init typescript program, typescript will read all files defined by our '/workspace/tsconfig.json' and stores as a Map, the key is the filename start with '/workspace'  not '/home/xxx/workspace', here is my case: ('/ssd' is mine real path and the symlink is '/home/ycma/workspace/ssd')

so currentProgram.getSourceFile(extra.filePath) returns undefined

@lordisback is this still an issue for you. If so can you please provide me with a GitHub repository I can clone that demos what you are experiencing.


Is the MAINTAINERS.md list updated?

@brian-brazil I've noticed the MAINTAINERS.md has not been updated since 2 years, so I was just wondering if it's still updated. Thanks for checking 🙏

It's correct, we haven't re-jigged all the maintainership in a while.

Thanks for checking!


Release jenkins 3.6.0-2.190.1 (automated commit)

Release jenkins 3.6.0-2.190.1 (automated commit)
Description:
Source URL: https://infinity-artifacts.s3.amazonaws.com/autodelete7d/jenkins/20191024-171945-GSln1ZSFKgX4JF3c/stub-universe-jenkins.json
Changes between revisions 30 => 40:
0 files added: []
0 files removed: []
4 files changed:
--- 30/config.json
+++ 40/config.json
@@ -1,144 +1,187 @@
 {
-    "type": "object",
-    "properties": {
-        "service": {
-            "description": "Configuration properties for the Jenkins service for DC/OS.",
-            "type": "object",
-            "properties": {
-                "name": {
-                  "description": "Unique name for the service instance consisting of a series of words separated by slashes. Each word must be at least 1 alphanumeric character and may only contain digits (`0-9`), dashes (`-`), dots (`.`), and lowercase letters (`a-z`). The word may not begin or end with a dash",
-                  "type": "string",
-                  "pattern": "^(\\/?((\\.\\.)|(([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9])\\.)*([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9]))?($|\\/))+$",
-                  "default": "jenkins"
-                },
-                "cpus": {
-                    "description": "CPU shares to allocate to each Jenkins master.",
-                    "type": "number",
-                    "default": 1.0,
-                    "minimum": 0.1
-                },
-                "mem": {
-                    "description": "Memory (in MB) to allocate to each Jenkins master.",
-                    "type": "number",
-                    "default": 2048.0,
-                    "minimum": 2048.0
-                },
-                "user": {
-                    "description": "The user that the service will run as. In 'strict' security mode, running tasks as root is disabled by default. Ensure that this user exists on each of the agents in your DC/OS cluster.",
-                    "type": "string",
-                    "default": "root"
-                },
-                "marathon-name": {
-                    "description": "The name of the Marathon framework the Jenkins master is deployed from. Change when using a MoM (Marathon on Marathon).",
-                    "type": "string",
-                    "default": "marathon"
-                }
-            },
-            "required": [
-                "name",
-                "cpus",
-                "mem"
-            ]
+  "type": "object",
+  "properties": {
+    "service": {
+      "description": "Configuration properties for the Jenkins service for DC/OS.",
+      "type": "object",
+      "properties": {
+        "name": {
+          "description": "Unique name for the service instance consisting of a series of words separated by slashes. Each word must be at least 1 alphanumeric character and may only contain digits (`0-9`), dashes (`-`), dots (`.`), and lowercase letters (`a-z`). The word may not begin or end with a dash",
+          "type": "string",
+          "pattern": "^(\\/?((\\.\\.)|(([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9])\\.)*([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9]))?($|\\/))+$",
+          "default": "jenkins"
         },
-        "storage": {
-            "description": "Storage-related configuration properties for Jenkins on DC/OS.",
-            "type": "object",
-            "properties": {
-                "host-volume": {
-                    "description": "The location of a volume on the host to be used for persisting Jenkins configuration and build data. The final location will be derived from this value plus the name set in `name` (e.g. `/mnt/host_volume/jenkins`). Note that this path must be the same on all DC/OS agents.",
-                    "type": "string",
-                    "default": "/tmp"
-                },
-                "pinned-hostname": {
-                    "description": "An optional DC/OS agent hostname to run this Jenkins instance on (e.g. <IP_ADDRESS>).",
-                    "type" : "string"
-                }
-            }
+        "cpus": {
+          "description": "CPU shares to allocate to each Jenkins master.",
+          "type": "number",
+          "default": 1.0,
+          "minimum": 0.1
         },
-        "networking": {
-            "description": "Networking-related configuration properties for Jenkins on DC/OS.",
-            "type": "object",
-            "properties": {
-                "known-hosts": {
-                    "description": "A space-separated list of hosts used to populate the SSH known hosts file on the Jenkins master.",
-                    "type": "string",
-                    "default": "github.com"
-                },
-                "virtual-host": {
-                    "description": "The virtual host address to configure for integration with Marathon-lb.",
-                    "type": "string"
-                },
-                "https-redirect": {
-                    "description": "Whether Marathon-lb should redirect HTTP traffic to HTTPS. This requires 'virtual-host' to be set. By default, this is false.",
-                    "type": "boolean",
-                    "default": false
-                },
-                "agent-port": {
-                    "description": "The port that Jenkins agents will use to connect to the master. The default port is set to 50000; a value of 0 will result in a random port being chosen.",
-                    "type": "integer",
-                    "default": 50000
-                }
-            }
+        "mem": {
+          "description": "Memory (in MB) to allocate to each Jenkins master.",
+          "type": "number",
+          "default": 4096.0,
+          "minimum": 2048.0
         },
-        "roles": {
-            "description": "Role configuration properties for Jenkins on DC/OS.",
-            "type": "object",
-            "properties": {
-                "jenkins-master-role": {
-                    "description": "The accepted resource roles that the Jenkins master itself runs as using Marathon. By default, this will deploy to any agents with the * role. For example, to deploy to a public DC/OS agent, set this to 'slave_public'.",
-                    "type": "string",
-                    "default": "*"
-                },
-                "jenkins-agent-role": {
-                    "description": "The role passed to the internal Jenkins configuration that denotes which resources to launch Jenkins agents on.",
-                    "type": "string",
-                    "default": "*"
-                  }
-              }
-          },
-        "security": {
-            "description": "Security configuration properties for Jenkins on DC/OS.",
-            "type": "object",
-            "properties": {
-                "strict-mode": {
-                    "description": "Enabled if Enterprise DC/OS is provisioned using the 'strict' security mode flag. When enabled, tasks run as the user specified below.",
-                    "type": "boolean",
-                    "default": false
-                },
-                "secret-name": {
-                    "description": "Name of the Secret Store credentials to use for DC/OS service authentication. This should be left empty unless service authentication is needed.",
-                    "type": "string",
-                    "default": ""
-                }
-            }
+        "user": {
+          "description": "The user that the service will run as. In 'strict' security mode, running tasks as root is disabled by default. Ensure that this user exists on each of the agents in your DC/OS cluster.",
+          "type": "string"
         },
-        "advanced": {
-            "description": "Advanced configuration properties for the Jenkins service. Under normal circumstances, you shouldn't need to modify these values.",
-            "type": "object",
-            "properties": {
-                "mesos-master": {
-                    "description": "URL of this cluster's Mesos master.",
-                    "type": "string",
-                    "default": "zk://zk-1.zk:2181,zk-2.zk:2181,zk-3.zk:2181,zk-4.zk:2181,zk-5.zk:2181/mesos"
-                },
-                "jvm-opts": {
-                    "description": "Optional arguments to pass to the JVM.",
-                    "type": "string",
-                    "default": "-Xms1024m -Xmx1024m"
-                },
-                "jenkins-opts": {
-                    "description": "Optional arguments to pass to Jenkins.",
-                    "type": "string"
-                },
-                "docker-image": {
-                    "description": "The Docker image to use for the Jenkins service. By default, this package will use the Jenkins image in the Mesosphere organization on Docker Hub, which you must be authenticated against. Otherwise, specify the host, image, and tag for the Jenkins image on your private Docker Registry.",
-                    "type": "string"
-                },
-                "docker-credentials-uri": {
-                    "description": "An optional URI to be fetched and extracted that contains docker credentials (e.g. file:///etc/docker/docker.tar.gz).",
-                    "type": "string"
-                }
-            }
+        "marathon-name": {
+          "description": "The name of the Marathon framework the Jenkins master is deployed from. Change when using a MoM (Marathon on Marathon).",
+          "type": "string",
+          "default": "marathon"
         }
+      },
+      "required": [
+        "name",
+        "cpus",
+        "mem"
+      ]
+    },
+    "storage": {
+      "description": "Storage-related configuration properties for Jenkins on DC/OS.",
+      "type": "object",
+      "properties": {
+        "host-volume": {
+          "description": "The location of a volume on the host to be used for persisting Jenkins configuration and build data. The final location will be derived from this value plus the name set in `name` (e.g. `/mnt/host_volume/jenkins`). Note that this path must be the same on all DC/OS agents.",
+          "type": "string"
+        },
+        "pinned-hostname": {
+          "description": "An optional DC/OS agent hostname to run this Jenkins instance on (e.g. <IP_ADDRESS>).",
+          "type": "string"
+        }
+      }
+    },
+    "networking": {
+      "description": "Networking-related configuration properties for Jenkins on DC/OS.",
+      "type": "object",
+      "properties": {
+        "known-hosts": {
+          "description": "A space-separated list of hosts used to populate the SSH known hosts file on the Jenkins master.",
+          "type": "string",
+          "default": "github.com"
+        },
+        "virtual-host": {
+          "description": "The virtual host address to configure for integration with Marathon-lb.",
+          "type": "string"
+        },
+        "https-redirect": {
+          "description": "Whether Marathon-lb should redirect HTTP traffic to HTTPS. This requires 'virtual-host' to be set. By default, this is false.",
+          "type": "boolean",
+          "default": false
+        },
+        "agent-port": {
+          "description": "The port that Jenkins agents will use to connect to the master. The default port is set to 50000; a value of 0 will result in a random port being chosen.",
+          "type": "integer",
+          "default": 50000
+        }
+      }
+    },
+    "roles": {
+      "description": "Role configuration properties for Jenkins on DC/OS. These properties are overriden when Marathon's enforceRole is enabled for top-level groups.",
+      "type": "object",
+      "properties": {
+        "jenkins-master-role": {
+          "description": "The role that the Jenkins master itself runs as using Marathon. Overriden by value of group-role when enforceRole is enabled at top-level group.",
+          "type": "string",
+          "default": "slave_public"
+        },
+        "jenkins-agent-role": {
+          "description": "The role passed to the internal Jenkins configuration that denotes which role to launch Jenkins agents as. Overriden by value of group-role when enforceRole is enabled at top-level group.",
+          "type": "string",
+          "default": "*"
+        }
+      }
+    },
+    "security": {
+      "description": "Security configuration properties for Jenkins on DC/OS.",
+      "type": "object",
+      "properties": {
+        "strict-mode": {
+          "description": "Enabled if Enterprise DC/OS is provisioned using the 'strict' security mode flag. When enabled, tasks run as the user specified below.",
+          "type": "boolean",
+          "default": false
+        },
+        "secret-name": {
+          "description": "Name of the Secret Store credentials to use for DC/OS service authentication. This should be left empty unless service authentication is needed.",
+          "type": "string",
+          "default": ""
+        }
+      }
+    },
+    "health-checks": {
+      "description": "Mesos health-check properties for Jenkins on DC/OS.",
+      "type": "object",
+      "properties": {
+        "grace-period": {
+          "description": "Health check failures are ignored within this number of seconds or until the task becomes healthy for the first time.",
+          "type": "integer",
+          "default": 30
+        },
+        "interval": {
+          "description": "Number of seconds to wait between health checks.",
+          "type": "integer",
+          "default": 60
+        },
+        "timeout": {
+          "description": "Number of seconds after which a health check is considered a failure regardless of the response.",
+          "type": "integer",
+          "default": 20
+        },
+        "max-consecutive-failures": {
+          "description": "Number of consecutive health check failures after which the unhealthy task should be killed.",
+          "type": "integer",
+          "default": 3
+        }
+      }
+    },
+    "advanced": {
+      "description": "Advanced configuration properties for the Jenkins service. Under normal circumstances, you shouldn't need to modify these values.",
+      "type": "object",
+      "properties": {
+        "mesos-master": {
+          "description": "URL of this cluster's Mesos master.",
+          "type": "string",
+          "default": "zk://zk-1.zk:2181,zk-2.zk:2181,zk-3.zk:2181,zk-4.zk:2181,zk-5.zk:2181/mesos"
+        },
+        "jvm-opts": {
+          "description": "Optional arguments to pass to the JVM.",
+          "type": "string",
+          "default": "-Xms1024m -Xmx1024m"
+        },
+        "jenkins-opts": {
+          "description": "Optional arguments to pass to Jenkins.",
+          "type": "string"
+        },
+        "jenkins-agent-user": {
+          "description": "The user that the jenkins agent task will run as.",
+          "type": "string",
+          "default": "root"
+        },
+        "docker-image": {
+          "description": "The Docker image to use for the Jenkins service. By default, this package will use the Jenkins image in the Mesosphere organization on Docker Hub, which you must be authenticated against. Otherwise, specify the host, image, and tag for the Jenkins image on your private Docker Registry.",
+          "type": "string"
+        },
+        "docker-credentials-uri": {
+          "description": "An optional URI to be fetched and extracted that contains docker credentials (e.g. file:///etc/docker/docker.tar.gz).",
+          "type": "string"
+        },
+        "containerizer": {
+          "description": "The containerizer used to deploy the Jenkins master.",
+          "type": "string",
+          "enum": [
+            "MESOS",
+            "DOCKER"
+          ],
+          "default": "DOCKER"
+        },
+        "prometheus-endpoint": {
+          "description": "An optional, relative URL path to be used by Prometheus. (e.g. 'prometheus/metrics')",
+          "type": "string",
+          "default": "v1/metrics/prometheus"
+        }
+      }
     }
+  }
 }
--- 30/marathon.json.mustache
+++ 40/marathon.json.mustache
@@ -3,6 +3,9 @@
   "cpus": {{service.cpus}},
   "mem": {{service.mem}},
   "instances": 1,
+  {{#service.user}}
+  "user":"{{service.user}}",
+  {{/service.user}}
   {{#security.secret-name}}
   "secrets": {
       "serviceCredential": {
@@ -12,46 +15,64 @@
   {{/security.secret-name}}
   "env": {
       "JENKINS_AGENT_ROLE": "{{roles.jenkins-agent-role}}",
-      "JENKINS_AGENT_USER": "{{service.user}}",
-      "JENKINS_SLAVE_AGENT_PORT": "{{networking.agent-port}}",
+      "JENKINS_AGENT_USER": "{{advanced.jenkins-agent-user}}",
       "JENKINS_FRAMEWORK_NAME": "{{service.name}}",
       "MARATHON_NAME": "{{service.marathon-name}}",
       {{#security.secret-name}}
-          "DCOS_SERVICE_ACCOUNT_CREDENTIAL": { "secret": "serviceCredential" },
-          "MESOS_MODULES": "{\"libraries\": [{\"file\": \"libdcos_security.so\", \"modules\": [{\"name\": \"com_mesosphere_dcos_ClassicRPCAuthenticatee\"}]}]}",
-          "MESOS_AUTHENTICATEE": "com_mesosphere_dcos_ClassicRPCAuthenticatee",
+        "DCOS_SERVICE_ACCOUNT_CREDENTIAL": { "secret": "serviceCredential" },
+        "MESOS_MODULES": "{\"libraries\":[{\"file\":\"libdcos_security.so\",\"modules\":[{\"name\": \"com_mesosphere_dcos_ClassicRPCAuthenticatee\"},{\"name\":\"com_mesosphere_dcos_http_Authenticatee\",\"parameters\":[{\"key\":\"jwt_exp_timeout\",\"value\":\"5mins\"},{\"key\":\"preemptive_refresh_duration\",\"value\":\"30mins\"}]}]}]}",
+        "MESOS_AUTHENTICATEE": "com_mesosphere_dcos_ClassicRPCAuthenticatee",
+        "MESOS_HTTP_AUTHENTICATEE": "com_mesosphere_dcos_http_Authenticatee",
       {{/security.secret-name}}
       "JENKINS_CONTEXT": "/service/{{service.name}}",
       "JENKINS_MESOS_MASTER": "{{advanced.mesos-master}}",
+      "JENKINS_HOME": "/var/jenkins_home",
       {{#networking.virtual-host}}
       "JENKINS_ROOT_URL": "{{#networking.https-redirect}}https://{{/networking.https-redirect}}{{^networking.https-redirect}}http://{{/networking.https-redirect}}{{networking.virtual-host}}/service/{{service.name}}",
       {{/networking.virtual-host}}
       "JVM_OPTS": "{{advanced.jvm-opts}}",
       "JENKINS_OPTS": "{{advanced.jenkins-opts}}",
+      "PROMETHEUS_ENDPOINT": "{{advanced.prometheus-endpoint}}",
       "SSH_KNOWN_HOSTS": "{{networking.known-hosts}}"
   },
   "portDefinitions": [
       {"port": 0, "protocol": "tcp", "name": "nginx"},
-      {"port": 0, "protocol": "tcp", "name": "jenkins"}
+      {"port": 0, "protocol": "tcp", "name": "jenkins"},
+      {"port": {{networking.agent-port}}, "protocol": "tcp", "name": "agent"}
   ],
   "container": {
-       "type": "DOCKER",
-       "docker": {
-       {{#advanced.docker-image}}
-           "image": "{{advanced.docker-image}}",
-       {{/advanced.docker-image}}
-       {{^advanced.docker-image}}
-           "image": "{{resource.assets.container.docker.jenkins-354-21501}}",
-       {{/advanced.docker-image}}
-           "network" : "HOST"
-       },
-       "volumes": [
-           {
-               "containerPath": "/var/jenkins_home",
-               "hostPath": "{{storage.host-volume}}/{{service.name}}",
-               "mode": "RW"
-           }
-       ]
+        "type": "{{advanced.containerizer}}",
+        "docker": {
+         {{#advanced.docker-image}}
+             "image": "{{advanced.docker-image}}"
+         {{/advanced.docker-image}}
+         {{^advanced.docker-image}}
+             "image": "{{resource.assets.container.docker.jenkins}}"
+         {{/advanced.docker-image}}
+        },
+        "volumes": [
+          {{^storage.host-volume}}
+          {
+            "containerPath": "jenkins_home",
+            "persistent": {
+            "type": "root",
+            "size": 2048,
+            "constraints": []
+            },
+            "mode": "RW"
+          },
+          {{/storage.host-volume}}
+          {
+            "containerPath": "/var/jenkins_home",
+            {{#storage.host-volume}}
+            "hostPath": "{{storage.host-volume}}/{{service.name}}",
+            {{/storage.host-volume}}
+            {{^storage.host-volume}}
+            "hostPath": "jenkins_home",
+            {{/storage.host-volume}}
+            "mode": "RW"
+          }
+      ]
    },
    {{#advanced.docker-credentials-uri}}
    "fetch": [
@@ -61,16 +82,16 @@
           "extract": true
       }
    ],{{/advanced.docker-credentials-uri}}
-   "acceptedResourceRoles": [ "{{roles.jenkins-master-role}}" ],
+   "role": "{{roles.jenkins-master-role}}",
    "healthChecks": [
     {
       "path": "/service/{{service.name}}",
       "portIndex": 0,
-      "protocol": "HTTP",
-      "gracePeriodSeconds": 30,
-      "intervalSeconds": 60,
-      "timeoutSeconds": 20,
-      "maxConsecutiveFailures": 3
+      "protocol": "MESOS_HTTP",
+      "gracePeriodSeconds": {{health-checks.grace-period}},
+      "intervalSeconds": {{health-checks.interval}},
+      "timeoutSeconds": {{health-checks.timeout}},
+      "maxConsecutiveFailures": {{health-checks.max-consecutive-failures}}
     }
   ],
   "labels": {
@@ -85,10 +106,11 @@
     "DCOS_SERVICE_PORT_INDEX": "0",
     "DCOS_SERVICE_SCHEME": "http"
   },
+  {{#storage.pinned-hostname}}
+  "constraints": [["hostname", "CLUSTER", "{{storage.pinned-hostname}}"]],
+  {{/storage.pinned-hostname}}
   "upgradeStrategy":{
     "minimumHealthCapacity": 0,
     "maximumOverCapacity": 0
-  }{{#storage.pinned-hostname}},
-  "constraints": [["hostname", "CLUSTER", "{{storage.pinned-hostname}}"]]
-  {{/storage.pinned-hostname}}
+  }
 }
--- 30/package.json
+++ 40/package.json
@@ -1,14 +1,18 @@
 {
   "packagingVersion": "3.0",
   "name": "jenkins",
-  "version": "3.5.4-2.150.1",
+  "version": "3.6.0-2.190.1",
   "minDcosReleaseVersion": "1.10",
   "scm": "https://github.com/mesosphere/dcos-jenkins-service.git",
   "maintainer"<EMAIL_ADDRESS>   "website": "https://jenkins.io",
   "framework": true,
   "description": "Jenkins is an award-winning, cross-platform, continuous integration and continuous delivery application that increases your productivity. Use Jenkins to build and test your software projects continuously making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build. It also allows you to continuously deliver your software by providing powerful ways to define your build pipelines and integrating with a large number of testing and deployment technologies.",
-  "tags": ["continuous-integration", "ci", "jenkins"],
+  "tags": [
+    "continuous-integration",
+    "ci",
+    "jenkins"
+  ],
   "preInstallNotes": "WARNING: If you didn't provide a value for `storage.host-volume` (either using the CLI or via the Advanced Install dialog),\nYOUR DATA WILL NOT BE SAVED IN ANY WAY.\n",
   "postInstallNotes": "Jenkins has been installed.",
   "postUninstallNotes": "Jenkins has been uninstalled. Note that any data persisted to a NFS share still exists and will need to be manually removed.",
@@ -18,14 +22,13 @@
       "url": "https://github.com/mesosphere/dcos-jenkins-service/blob/master/LICENSE"
     },
     {
-        "name": "Apache License Version 2.0",
-        "url": "https://github.com/jenkinsci/mesos-plugin/blob/master/LICENSE"
+      "name": "Apache License Version 2.0",
+      "url": "https://github.com/jenkinsci/mesos-plugin/blob/master/LICENSE"
     },
     {
-        "name": "MIT License",
-        "url": "https://github.com/jenkinsci/jenkins/blob/master/LICENSE.txt"
+      "name": "MIT License",
+      "url": "https://github.com/jenkinsci/jenkins/blob/master/LICENSE.txt"
     }
   ],
-  "selected": true,
-  "lastUpdated":<PHONE_NUMBER>
-}
+  "selected": true
+}--- 30/resource.json
+++ 40/resource.json
@@ -7,7 +7,7 @@
   "assets": {
     "container": {
       "docker": {
-        "jenkins-354-21501": "mesosphere/jenkins:3.5.4-2.150.1"
+        "jenkins": "mesosphere/jenkins:3.6.0-2.190.1"
       }
     }
   }

Docs Site PR Here: https://github.com/mesosphere/dcos-docs-site/pull/2599


feature: single host should allow scheduling to master

Every time I run a Single host cluster, I need to manual remove the Taint.
$ sealos run labring/kubernetes:v1.25.0 labring/helm:v3.8.2 labring/calico:v3.24.1 --single
# remove taint
$ kubectl taint nodes --all node-role.kubernetes.io/master:NoSchedule-

I add a patch to do the untaint automatic #2317.

kubeadm init .... --skip-phases=mark-control-plane

---
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
skipPhases:
- mark-control-plane

That's sounds great, I will change my implementation.

test is ok ?

test is ok ?

Test tomorrow, may contain some bugs.

Some test case need to add, need to improve.

You can refer to the test case in ci and test it with the shell to see if the final single node has stains.

https://github.com/labring/sealos/actions/runs/4259381988/jobs/7412043583#step:6:243 same not remove taint ?

this process should only invoke in initCluster, so I would rather implement it in function pkg/runtime/KubeadmRuntime.generateInitConfigs.
if sum(len(masters), len(nodes)) == 1 then append skip-phases

this process should only invoke in initCluster, so I would rather implement it in function pkg/runtime/KubeadmRuntime.generateInitConfigs.
if sum(len(masters), len(nodes)) == 1 then append skip-phases

If he just installs the join node behind master0, is this judgment a bit rough?

I think dependency injection will more appropriate

NewDefaultRuntime is a really confused function signature which provide *KubeadmConfig but will not use it.

this process should only invoke in initCluster, so I would rather implement it in function pkg/runtime/KubeadmRuntime.generateInitConfigs.
if sum(len(masters), len(nodes)) == 1 then append skip-phases

If he just installs the join node behind master0, is this judgment a bit rough?

once cluster is provisioned, the taints should remove manually.

this process should only invoke in initCluster, so I would rather implement it in function pkg/runtime/KubeadmRuntime.generateInitConfigs.
if sum(len(masters), len(nodes)) == 1 then append skip-phases

If he just installs the join node behind master0, is this judgment a bit rough?

once cluster is provisioned, the taints should remove manually.

It should only skip mark-control-plane phase when the cluster is first created and --single flag is set.
When user need join more nodes into cluster, he shouldn't provide --single flag the first time, the Taint should remain.

https://github.com/labring/sealos/actions/runs/4260038410/jobs/7413126020#step:6:250 There should be no stains

https://github.com/labring/sealos/actions/runs/4260038410/jobs/7413126020#step:6:250 There should be no stains

(CreateProcessor.PreProcess)[https://github.com/hexinatgithub/sealos/blob/b05ff82d702e9ae757f422b8ff622233a9f25c86/pkg/apply/processor/create.go#L87] use NewDefaultRuntime cause this weird bug.

It was accidentally turned off. Please reopen this issue.

你重新提交pr即可

I rewrite my code and pass all test.
Please check it.


Create quick start guideline

Create a guideline with steps to follow those, a newbie can easier start to use Elementary.

In main


Reintroduced pull_down_input, pull_up_input for GPIOs

I found this commit on @e-matteson repo that seems to take it from @burrbull repo. It's simple and working fine, and I need it to remove that useless 10k resistor that pull up my buttons.
OK @japaric @e-matteson @burrbull?

Similarly, I forked it just to save a snapshot to use as a dependency. I can say that I've successfully used it to enable pullups on a blue-pill board, but haven't thoroughly tested or reviewed it.

Neat! I had an actual use case for this a few weeks back, but then used actual pull-up-resistors instead. I would like to test it, but on the one hand I also only have the bluepill board at hand and on the other hand probably have not enough time in the near future to give it a more in-deepth test :(

I've ttested on a blue pill with a button connected to ground with mc pull up, and connected to VCC with mc pull down. Everything worked. I've reviewed the code and everything seems fine.

Looks very similar to my implementation here: https://github.com/TheZoq2/stm32f103xx-hal/commit/5c11436ed4f4e8e4a97eb2486bff9219d5d0c41d

Maybe this should be merged then ? Is japaric still the main maintainer of this crate ?

AFAIK, @japaric is the only one that can merge.

I believe so as well, and I also believe that he is pretty busy with other embedded-wg work. Perhaps someone else could step in as co-maintainer until @japaric has more time. I'd be happy to do it if required and it also seems like @TeXitoi would make a very good candidate since they are already inoffically reviewing a bunch of PRs

Check out this issue in the workgroup repository :wink:

Kinda relates #51


chore(RHTAPWATCH-566): Write service monitor to scrape the metrics

Add service monitor to scrape the metrics

Does it actually work? Are you able to query in Openshift for the metric the exporter is generating?

@yftacherzog @avi-biton Need your approval, the PR is now updated. It is working now and @hmariset can view the metrics.

Two things that we missed were updating the commit ref number and RHTAP setup.

/ok-to-test


bug: ionicon used internal in ion-reorder, ion-searchbar

Bug Report
Ionic version:

[x] 5.0.0-rc.2
Current behavior:

when disabling ion-icons usage i got error while using some components that depends on it like
ion-reorder, ion-saerchbar
GET http://localhost:8100/svg/reorder-two-sharp.svg 404 (Not Found)
Expected behavior:

the component should not be depending on ion-icons
and if not, there must be a way to override that behavior to disable the icon or change it.
Steps to reproduce:

remove ion icons import and use font-awesome instead.
use ion-reorder component or ion-searchbar
you will get this error message in the console
GET http://localhost:8100/svg/reorder-two-sharp.svg 404 (Not Found)

Related code:

insert short code snippets here

Other information:

Ionic info:

Ionic:

   Ionic CLI                     : 5.4.11 (C:\Users\sheri\AppData\Roaming\npm\node_modules\ionic)
   Ionic Framework               : @ionic/angular 5.0.0-rc.2
   @angular-devkit/build-angular : 0.803.23
   @angular-devkit/schematics    : 8.3.23
   @angular/cli                  : 9.0.0-rc.14
   @ionic/angular-toolkit        : 2.1.2

Cordova:

   Cordova CLI       : 8.1.2<EMAIL_ADDRESS>   Cordova Platforms : android 8.0.0
   Cordova Plugins   : cordova-plugin-ionic-keyboard 2.2.0, cordova-plugin-ionic-webview 4.1.3, (and 16 other plugins)

Utility:

   cordova-res : not installed
   native-run  : 0.2.8

System:

   Android SDK Tools : 26.1.1 (C:\Users\sheri\AppData\Local\Android\Sdk)
   NodeJS            : v10.16.3 (C:\Program Files\nodejs\node.exe)
   npm               : 6.9.0
   OS                : Windows 10

Thanks for the issue. I am going to close this as this is not a bug in Ionic Framework. We use certain Ionicons in internal components; however, we do not have plans to expose the ability to use different icon sets with out internal components.
Developers are able to import their own icons (such as Font Awesome) to use in their applications. You will not have to load all of Ionicons as we only load the icons you need (such as the icon for ion-searchbar or ion-reorder).

so when importing the ionicons
it is only getting the svgs of the used component only ?

Yes, that is correct. If your app is using ion-searchbar, it will only load the ionicons that ion-searchbar needs, not the entire ionicons collection.


[SDK][INFINITY-1113] Remove VIP from executor for now

Open for too long.  Reopen when ready to review and merge.


For now, test coverage reduction will not fail the build

Summary
Let us move forward and ignore test coverage

Warnings

:warning:
Please add the JIRA issue key to the PR title (e.g. MB-123)

  Generated by :no_entry_sign: dangerJS against fc1e4a83261cf00c896d4439153d0ea29fdd032d


Bug: selection.getNodes() returns wrong data.

I'm trying to change headings back to paragraphs if the heading button is clicked again but selection.getNodes() returns [TextNode] instead of [HeadingNode].
  const formatHeader = (tag: HeadingTagType) => {
    editor.update(() => {
      const selection = $getSelection()
      if ($isRangeSelection(selection)) {
        const nodes = selection.getNodes()

        // prints [TextNode] <----------------------- Here is the problem, where is HeadingNode.
        console.log(nodes)

        // prints [TextNode, HeadingNode]  <---------------- Ok, HeadingNode exists here but what's with TextNode? Also, it's not possible to use setBlocksType inside setTimeout.
        setTimeout(() => console.log(nodes), 0)

        if (selection.getNodes().some((n) => $isHeadingNode(n))) {
          $setBlocksType(selection, () => $createParagraphNode())
        } else {
          $setBlocksType(selection, () => $createHeadingNode(tag))
        }
      }
    })
  }

Lexical version: 0.12.4
Steps To Reproduce

Go to playground.
Make a heading.
Try the "Clear Formatting" button.

It doesn't work even though it should. Here's playground's source code.
https://github.com/facebook/lexical/blob/303530a5f36ea9b54e6c3ba837e7ebb4665da8f2/packages/lexical-playground/src/plugins/ToolbarPlugin/index.tsx#L786

The problem is that selection.getNodes() does not include a node if it does not have part of the selection outside the node, which seems to be a reasonable default behavior.
In setBlocksType I had to do something similar.
https://github.com/facebook/lexical/blob/0f7c78b24dec932c9e1629623bedaf4485eacb08/packages/lexical-selection/src/range-selection.ts#L61-L68
I'm also doing that same thing in my code base.
It occurs to me that getNodes could be given an optional parameter to include nodes that include the selection completely, up to an ancestor that meets a certain condition (because otherwise even root would be included).
The lines I linked above and the ones you mention could be transformed into: selection.getNodes(INTERNAL_$isBlock).

As stated above, this is expected behavior, otherwise getNodes would basically have to return all parents. Finding the block that a selection is contained in simply isn't the purpose of getNodes.


Update avalon8-pmu-test.py

please review the lastest commit, thanks.

Please review pull request.

Thinks


How can I copy content from computer and paste it on scrcpy?

Currently content on scrcpy can be pasted to computer but not the other way around. How can I copy text on computer to scrcpy?
Thank you

Which version of scrcpy do you use?

Probably #1750 #1771 d5f059c7cbef7d13889086670716d9f33f805dc4
TL;DR: if that's the problem, then your device ROM does not behave as expected on paste, so use scrcpy --legacy-paste.

I use 1.16
1.17 there is no .exe for no console but vbs and vbs shortcut cant be placed on taskbar that's why I reverted back to 1.16. could you maybe consider creating noconsole exe for 1.17?
Where will I place the --legacy-paste to shortcut?
How can I take screenshot with scrcpy?

Hello I just realied without legacy paste, copy paste from computer to phone works with shift+alt+v but it types one by one rather than the whole sentence at once. Is it possible for you to implement instant paste like it does copy pasting from phone to computer?
Also what could be a workaround for turkish characters to be typed into scrcpy with typing on turkish keyboard on my laptop. Could there be an intermediary app that could be activated upon scrcpy launch and convert my input to turkish and reflect it on scrcpy?
thank you

Actually I dont remember how it was before but now all keys work, including ü and ö, the only character that is blank when I type is "ğ". Is it possible for you to make "ğ" recognizable by scrcpy?
thank you


Pull/Issue message parsing quirks

Linking to issue/pull should only occur if # is:

Not inside a codeblock
At the beginning of line or preceded by whitespace (eg, not param#1)
Not inside a link (this should be fixed by the previous requirement)
Greater than 99

Not inside a codeblock

Agreed.

At the beginning of line or preceded by whitespace (eg, not param#1)

Agreed.

Not inside a link (this should be fixed by the previous requirement)

Indeed, previous requirement should fix this.

Greater than 99

That heuristic seems pretty random to me.


remove special characters on file names. Cannot clone on windows

For windows users up there, special characters will make windows file system cry. I found a way, but maybe it will be help someone. The following 2 files in others are causing problems. I believe it is ":"

example if you try to clone.

tagging @dustine32 @kltm

Honestly, I think it might be a bit of a stretch to consider : a special character, but it does seems to be a legit Windows-specific problem. You got me on that. Since they are "other/" old models, they could probably be chucked without too much worry, but colons in filenames is likely something that you'll have to content with at some point one way or another.

Same issue here...


Bug in wheelJoint.getLocalAxis()?

Pretty sure this is a bug though I may be missing something,  but whenever I try and get the local axis of a wheel joint I end up with what looks like a local anchor. After looking at the source it looks like getLocalAxisA is calling a native function jniGetLocalAnchorA instead of jniGetLocalAxisA. Relevant function is on line 66 in WheelJoint.java:
link to WheelJoint.java
public Vector2 getLocalAxisA(){
	jniGetLocalAnchorA(addr, tmp);    //<< Shouldn't this be jniGetLocalAxisA(addr,tmp) ?
	localAxisA.set(tmp[0], tmp[1]);
	return localAxisA;
}

Heres a code snippet to show what I'm talking about:
World world = App.engine.systems.physics.world;
BodyDef bd = new BodyDef();
bd.type = BodyType.DynamicBody;

Body body = world.createBody(bd),
	 wheel = world.createBody(bd);

PolygonShape rect = new PolygonShape();
rect.setAsBox(2, 1);
CircleShape circle = new CircleShape();
circle.setRadius(1);

body.createFixture(rect,1);
wheel.createFixture(circle,1);

Vector2 localAnchor = new Vector2(0,0),
		localAxis = new Vector2(0,1);

WheelJointDef jd = new WheelJointDef();
jd.initialize(body, wheel, localAnchor , localAxis);
WheelJoint joint = (WheelJoint)world.createJoint(jd);

System.out.println("Local Axis = " + joint.getLocalAxisA());
System.out.println("Local Anchor A = " + joint.getLocalAnchorA());

//output:
//Local Axis = [0.0:0.0]         <<< This should be outputting [0.0:1.0]
//Local Anchor A = [0.0:0.0]

Looks like a bug.


Allow removal of audit-logs container in kas when "None" policy is used

The kube-apiserver has a container called audit-logs that keeps audit records stored in the logs of the container (just prints to stdout). We would like the ability to disable this container whenever the None policy is used on the cluster. As of today, this consumes about 1gb of storage for each apiserver pod on the system. As you scale up, that 1gb per master adds up.

Closing in favor of https://issues.redhat.com/browse/OCPBUGS-31012


Remove openblas from build

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


switch base image to python:3.7-slim

blackhole requires python 3.7. ubuntu 18.04 only provides python 3.5

thanks for letting me know.  we publish all of our images on ubuntu.  i bumped to ubuntu:22.04 and we're now up to python 3.10.  Thanks for letting me know.


Vimeo bug, playing: true prop does not trigger video to play inside iframe

I am working on an app with ReactPlayer rendering inside of an iFrame. All video sources work as expected except for Vimeo. Passing playing: true to the video player does not start the video player. I forked the repo and modified the demo to illustrate the issue using two video players.
https://github.com/jmancherje/react-player/tree/vimeo-iframe-bug
Steps to reproduce:

clone this fork
npm i to get react-frame-component
Select a Vimeo video
click the Play button to send the prop to both video players
only the second video player (non-iFrame) will play correctly
(try with any other video source and it will work as expected for both video players)

Does anyone have any insight as to what is causing this discrepancy with Vimeo vs the other video sources? I'm happy to submit a PR once the issue is isolated.

Hey @jmancherje, have you tested your app using the vimeo branch of react-player? It's a branch I have been intending to merge soon that uses the newer Vimeo API, which hopefully solves the majority of existing problems with Vimeo videos.

@CookPete Thanks for pointing me to your Vimeo branch. Unfortunately the Vimeo player still does not work inside of an iFrame. I get the following error from this call on line 62 of Vimeo.js
this.player = new Vimeo.Player(this.playerId, {...DEFAULT_OPTIONS,      ...this.props.vimeoConfig.playerOptions, id, loop: this.props.loop})

The vimeo branch has now been merged and published. @jmancherje feel free to reopen this if the problem still occurs.

@CookPete Thanks for the update. I just updated the example using react-player 0.16.0 and it seems like vimeo videos are still not rendering correctly within an iframe. I made a new branch for this demo and updated the link the original issue:
https://github.com/jmancherje/react-player/tree/iframe-bug
Can you please reopen this issue?

Thanks for the updated demo. Looks like this is an issue with the Vimeo API and how it sends, receives and filters messages: https://github.com/vimeo/player.js/issues/16
Based on the discussion there, I'm not sure there is anything that we can do apart from wait for a fix on their end.


Add Tile component

Issue: https://github.com/open-cluster-management/backlog/issues/6461

Add link to issue/story in the description.

Do we have a "skeleton" tile?  I see we have a skeleton for the saved searches card.

Do we have a "skeleton" tile? I see we have a skeleton for the saved searches card.

Not as of now can add that

Minor change


Updated screenshots for @mospherePro Desktop

Lead: @ksanty

[ ] Updated screenshots needed for @mospherePro desktop reflecting color and text updates

@ksanty wrong repo for this issue

@XavierChanth well, I wanted to use the @mospherePro repo, but it wasn't accessible.  Which one should I use?  I want to keep this in the Dev Eng sprint for @sarika01 and @sachins-geekyants.  Thanks!

@XavierChanth well, I wanted to use the @mospherePro repo, but it wasn't accessible. Which one should I use? I want to keep this in the Dev Eng sprint for @sarika01 and @sachins-geekyants. Thanks!

Probably apps repo


Fix IntegerField and DateField for Excel import.

IntegerField was returning None when value = 0.0.
DateField could not handle how Excel stores dates as floats.
This addresses #274 and #152 .

@nevsan thanks for contributing. There is PR for value=0 cases for both IntegerWidget and DecimalWidgetwith test cases. Can you check?

DateField could not handle how Excel stores dates as floats.

Regarding this there is a problem that DateWidget and DateTimeWidget import/export field as string. This is ok for CSV but probably not what one want xls format. Personally in mine projects I use custom widget for dealing with dates in Excel.
One possible solution is to introduce additional keyword argument to widgets so they would know what format they should deal with.

PR is https://github.com/django-import-export/django-import-export/pull/282

Yes, that is correct. I hadn't seen that. I'll close this and open with just the excel mods (and add tests).
The solution proposed should work for Excel and CSV (it explicitly checks if it is a float, and does the normal thing otherwise). If other formats store numeric values instead of strings, then we will want a way of differentiating with format it came from.


Is the category part all gone since v4?

I have just manages to update to the new v4 gallery. Great work!
I really liked the category option from the old version. Is it all gone, or will it be reimplemented in the future?

Hello. Thanks :)
On the first versions of this lib, the header was built for specific needs on a project where category filtering was required.
With next improvements, I've decided to remove all header features because searching has too much diversity and is impossible to achieve correctly. This must be "outsourced" and won't come back in this lib that is now only focused on rendering images and do it well.
But ...
We have started this angular searchbar component https://github.com/Ecodev/natural-search. With the incoming feature Angular Elements we can hope to use it out of angular projects one day. This is only a filter set builder... we still need to build a javascript search engine to filter a list of elements.The filtered result could then be sent to natural-gallery-js for rendering...
Wait and see...


Broken for relativeElement with dynamic height

Hello!
I've been trying to use this plugin for my list of ajax-fetched results and it seems vue-affix can't properly deal with the relativeElement which height is changing during his lifetime. vue-affix saves the dimensions of relativeElement when it's mounted. If it's height changes, vue-affix is not catching the change.
I tried to extract the initialization logic to separate methods to make it able to control the flow but i'm not too good at JS and it's still broken, just in a different way. I'm gonna try to show you some demo of this later but, for now: is there anything you can say is 100% wrong in my code?: https://gist.github.com/mbajur/8273745f03846fd153e0f4b3083be9f3
Sample usage:
<template>
  <affix :delayed="true" ref="affix">...</affix>
</template>

<script>
  export default {
    // ...

    methods: {

      // First fetch
      onAjaxFetched () {
        this.$refs.affix.start()
      },

      // All the next fetches
      onAjaxFetchedMore () {
        this.$refs.affix.update()
      }
    }

    // ...
  }
</script>

Thanks in advance!

Ok, i created a sample project to illustrate my problem: https://github.com/mbajur/vue-affix-delayed

Here are few scenarios i have a problem with:

First of all - the original package. You can test the behaviour i mentioned in original post by uncommenting the regular vue-affix import in App.vue and commenting the custom one. Loading first set of results is delayed so vue-affix makes it's calculations for non-filled results div (which has no height). When results comes up - affix thinks bottom of a relative div is way higher than it really is. You can see it by just scrolling down the page and looking at the green element.
For the first, initial set of results, red affix is not working at all. After adding another set of 10 results, it starts to work but acts weird (start scrolling down - red box will start to scroll with the page way too early)
Load the page, scroll to the bottom and refresh the page. This is where things goes mad - the red box has a weird top offset instead of top: 0 and is basically permanently sticked to the top. Untill you click the "Add 10 results" - this is fixing the issue.

I'm still working really hard on #3 and will probably finish it tonight, then I'll have a look at this closely. I was sure it would have problems with dynamic content but didn't have the time to test yet, sorry for the lack of time :(
Thanks for the feedback!

Hey, sure, don't stress yourself, i really appreciate what you've done so far in here :) That's also probably the only affix/sticky plugin for vue out there and i need to use affix/sticky in my project pretty bad so i'm gonna do whatever i can to help in here with that. Let me know if you need any more feedback or testing when time will come.

Thanks for that :D
I'm changing a lot of things while implementing #3, refactoring a lot of code, I made this plugin in one day last weekend and it still have a lot of early stage code, but when I finish it I will sure ask for more tests regarding this issue :)

Hello, could you update the lib and test it again? I rewrote almost the whole code and I wanted to see if your problem still persists. I didn't work to fix it but it might be good after this refactoring. If you still have problems, describe them here again and I'll take a close look this weekend.
Thanks :)

Ha! yes, now it all works as it should. All my issues are gone :) Thanks for the awesome work!

Awesome :D
Its good to know the performance is better, as I said, i rewrote almost everything and it's much better now, I just need to update the docs with the new functionalities.


Fix typo in amqp.adoc

Fixes a typo in the reference documentation.

Thank you, @GonMMarques ; looking forward for more!


How to Clean up refresh or access token that no longer needed.

Is there a way to clean up refresh or access token that no longer needed. It will be more convenient if we have option to do this especially when user base is more than 10k .

We currently don't provide any helper in this package to do what you ask. It is up the developer how implement this.


 Only a couple of instructions were missing: ADDSUBPS and ADDSUBPD.
While finding what's missing or not tested is automated, generation of the Optab entries isn't (yet!).
$ x86db-gogen list --extension SSE3 --not-known
ADDSUBPD xmmreg,xmmrm [rm: 66 0f d0 /r] PRESCOTT,SSE3,SO
ADDSUBPS xmmreg,xmmrm [rm: f2 0f d0 /r] PRESCOTT,SSE3,SO

Is there a reason CL42990 is not merged yet?
With this one inside, I believe whole SSE will be complete [with exceptions of some MMX instructions].
CC @TocarIP


Using Template makes a git remote that uses HTTPS instead of SSH

I have used Template(interactive=true)("MyPkg") to make a package, and I then set the directory. When I then try to push to github I am asked for username and password, which can't be used anymore as github now requires the use of an SSH key.
You can get round this by going into the directory and using
git remote remove origin
git remote add origin git@github.com:HettieC/MyPkg.git
but it would be preferred to not have to perform this extra step.

You can set the remote to use ssh by customising the Git plugin (which is a plugin included in the Template by default) and setting ssh=true
e.g.
julia> tpl = Template(plugins=[Git(ssh=true)]);

julia> tpl("MyPkg")

or in interactive mode
julia> tpl = Template(interactive=true);
...
Git keywords to customize:
[press: d=done, a=all, n=none]
   [ ] branch
   [ ] email
   [ ] gpgsign
   [ ] ignore
   [ ] jl
   [ ] manifest
   [ ] name
 > [X] ssh
Enter value for 'ssh' (Bool, default=false): true
...

julia> tpl("MyPkg")

Then you should see something like
~> cd ~/.julia/dev/MyPkg/
~/.j/d/MyPkg> git remote -v
origin  git@github.com:nickrobinson251/MyPkg.jl.git (fetch)
origin  git@github.com:nickrobinson251/MyPkg.jl.git (push)

Does that resolve the issue? Do you think there's something we could do to make the ssh option easier or clearer?

That does resolve it, but seeing as Git no longer supports password authentication I'd have said it would be beneficial to make SSH the default, to avoid having to go through and change anything.
You also need to specify an ssh value when doing the above way, but just usual the defaults and not customising Git is able to produce an automatic HTTPS link.
It's not a huge issue, but takes some digging to find out what's going wrong with Git, and then starting over.

seeing as Git no longer supports password authentication I'd have said it would be beneficial to make SSH the default

That's a fair point, though as far as I can tell GitHub still recommends HTTPS (with a PAT) and that's the default for git clone, so I don't think will change the default here.
https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/
https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token


Change error domains on @objc enums to ensure they're unique

Consider this code:
class GithubAccount: NSObject {
  @objc(GithubSyncError) enum SyncError: Int, Error { ... }
}
class BitbucketAccount: NSObject {
  @objc(BitbucketSyncError) enum SyncError: Int, Error { ... }
}

Because they are @objc enums, Swift automatically derives _domains for these two error types of the form "ModuleName.SyncError", ignoring the fact that they are nested in different types and have different Objective-C names. That means an error created in Objective-C with [[NSError alloc] initWithDomain:GithubSyncError code:… userInfo:…] can be interpreted as a BitbucketAccount.SyncError in Swift and vice versa.
(Thankfully, this can't happen if you cast an error created in Swift to NSError and then back to the other type.)
This commit instead generates domains based on the Objective-C name of the enum, which already needs to be unique within the module for the generated header to be valid. It currently generates names of the form "\(objcName)Domain (\( moduleName ))", but I might still change that. The rest of this is ready to look at.
Non-@objc enums continue to use fully-qualified, pretty-printed type names, produced by String(reflecting:), as their error domains.
Resolves SR-9389 and rdar://problem/46420910.

@swift-ci please test source compatibility

After sleeping on it, I decided that people have probably hardcoded error domains in various places and we don't want to break them unnecessarily. So today's commits go back to using the Swift name of the type and simply generate it more robustly; that way only nested error types will change domains. getErrorDomainStringForObjC() doesn't handle everything that can come up in a Swift type, but it handles everything that should come up in a PrintAsObjC-able type; if the compiler detects that a given type has a construct it can't handle, it derives a call to String(reflecting: self) instead.
The version history here is now dirty enough that I might rewrite history in the morning, but I think this is the right solution.

@swift-ci please test

Build failed
Swift Test Linux Platform
Git Sha - 152158ea974df33eff8dd13c904d7c35d30668fb

Build failed
Swift Test OS X Platform
Git Sha - 152158ea974df33eff8dd13c904d7c35d30668fb

Rewrote history to remove intermediate states.

@swift-ci please test

Build failed
Swift Test Linux Platform
Git Sha - 6d681982455e58ebcbe6c4ca19df8eb4bef827b5

Build failed
Swift Test OS X Platform
Git Sha - 6d681982455e58ebcbe6c4ca19df8eb4bef827b5

Updated the commit description to reflect the current solution. @kballard, as the person who reported both SR-9389 and SR-700, does this approach look good to you?
(There's no rush; this won't merge until the new year.)

@swift-ci please smoke test and merge

@swift-ci please smoke test and merge

@swift-ci please smoke test


📃: Add myntra-clone

:red_circle: Title : Add Myntra-clone
:red_circle: Aim : To add myntra-clone
:red_circle: Tech Stack Used : HTML, CSS, ReactJS
Screenshots 📷

N/A

:white_check_mark: To be Mentioned while taking the issue :

Full name :
What is your participant role? (Mention the Open Source Program name. Eg. GSSOC, SSOC, JWOC, etc.)

Happy Contributing 🚀
All the best. Enjoy your open source journey ahead. 😎

issue already present


eclipses: Refactor eclipse plugin support to leverage p2

fixes #142191

What this PR does is rework how nixpkgs models a "plugin". (Note that there is a mismatch the language used by nixpkgs and Eclipse, specifically in what they refer to as a "plugin". But this already existed and can't really be fixed without creating confusing inconsistencies or just breaking backwards compat. And it's not really a big problem.)
Previously, the nixpkgs notion of a plugin was just a set of jars. These jars would be dumped into the dropins directory for an Eclipse instance.
The proposed nixpkgs notion of a plugin is a p2 update site, and an (optional) set of ids and versions which index into that update site. This gives Eclipse more information to help it resolve a working runtime.
This allows us to build a plugin derivation by mirroring a p2 update site locally. The problem with this is that a p2 update site is not a fixed resource, so the hash may change! An artifact in a p2 update site, for a given id and version, is specified to be a fixed resource, and so the hash shouldn't change, but artifacts can be added to an update site over time. The solution to this problem is to perform only a partial mirror, extracting a feature with a specific version and closing over its dependencies. I believe this should be reliable in practice, as the feature->plugin dependencies used are specified with exact versions.
Motivation for this change
Practical issues with the previous approach are outlined in #142191. More positively:

This approach more closely aligns with the Eclipse model as its intended to be used, for what that's worth.
I think it actually makes packaging eclipse plugins easier in most cases, as package maintainers can just use public-facing update sites as intended, rather than searching for zipped update sites (which aren't always available) or manually trawling for the transitive closure of jar dependencies which constitute a feature.
The changes should be forwards compatible with 95% existing derivations out in the wild.

Problems with this change
There are probably some users out there with custom plugin derivations which will stop working, but hopefully it shouldn't be too much work to fix them. Specifically, derivations which directly call buildEclipsePluginsBase won't work, but I think that should be rare, as it's not a user-friendly mechanism to begin with.
There will be other edge-cases too. For instance jsonedit needed changing due to the dependency on the antlr runtime. Since there is no official packaging for the antlr runtime, the jsonedit project repackages it themselves and publishes it in the jsonedit p2 repo. This is fine, but they specify the dependency in such a way that you have to use their repackaged version to make the Eclipse resolver happy. This wasn't a problem with the old dropins model as the dropins mechanism is a bit looser with the rules for wiring everything up.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Since there will be a small number of breakages downstream, does this PR warrant release notes? I have tried to keep it compatible as far as possible; most users of Eclipse shouldn't notice.
I've manually tested the changes by configuring an Eclipse instance with the following plugins:

bndtools: to make sure #142191 is actually fixed.
jsonedit, antlr-runtime_4_5, antlr-runtime_4_7: as these were the only existing plugins which needed changing.
yedit: to check that buildEclipsePlugin still works.
checkstyle: to check that buildEclipseUpdateSite still works.

They all appeared to be correctly installed and working.
I'm not an expert in Nix packaging so I expect some questionable choices will surface in review! I'll put some of my own comments inline where I wasn't sure about things.

Do you intend your author name to be a bare = like this?
$ git log -1 6ca57d323bf909fe058bc141a6f95222360933ef | grep ^Author
Author: =<EMAIL_ADDRESS>

I have a few plugins locally that I hadn't pushed to nixpkgs; I just opened https://github.com/NixOS/nixpkgs/pull/153244 to get them in.
But, one of them doesn't work with this PR.  I think it's due to a whitespace issue.
$ nix-build -I nixpkgs=$(pwd) -E 'with import <nixpkgs> {}; with pkgs.eclipses; eclipseWithPlugins { eclipse = eclipse-platform; plugins = [ plugins.mat ]; }' -K
...
Unknown option Analyzer. Use -help for a list of known options.
There were errors. See log file: /build/configuration/1641151187969.log

$ grep Analyzer /tmp/nix-build-eclipse-platform-4.22.drv-0/configuration/1641151187969.log --color
Framework arguments:  -application org.eclipse.equinox.p2.director -destination /nix/store/6zia9d3k5gxn3agl1xkssfmv46lyaxcy-eclipse-platform-4.22/eclipse -repository file:/nix/store/vspm6lm42bs9scfbh9gky03mybgpd576-eclipse-plugin-mat-1.9.1 -installIU<PHONE_NUMBER>59.Memory Analyzer IDE,201908260659.Memory Analyzer RCP,org.eclipse.mat.api,org.eclipse.mat.chart,org.eclipse.mat.chart.feature.feature.group,org.eclipse.mat.chart.feature.feature.jar,org.eclipse.mat.chart.ui,org.eclipse.mat.dtfj,org.eclipse.mat.feature.feature.group,org.eclipse.mat.feature.feature.jar,org.eclipse.mat.hprof,org.eclipse.mat.ibmdumps,org.eclipse.mat.jdt,org.eclipse.mat.jruby.resolver,org.eclipse.mat.parser,org.eclipse.mat.report,org.eclipse.mat.ui,org.eclipse.mat.ui.help,org.eclipse.mat.ui.rcp,org.eclipse.mat.ui.rcp.feature.feature.group,org.eclipse.mat.ui.rcp.feature.feature.jar,org.eclipse.mat.ui.rcp.feature_root,org.eclipse.mat.ui.rcp.feature_root.gtk.linux.ppc64,org.eclipse.mat.ui.rcp.feature_root.gtk.linux.x86,org.eclipse.mat.ui.rcp.feature_root.gtk.linux.x86_64,org.eclipse.mat.ui.rcp.feature_root.win32.win32.x86,org.eclipse.mat.ui.rcp.feature_root.win32.win32.x86_64, -tag AddIUs
Command-line arguments:  -os linux -ws gtk -arch x86_64 -application org.eclipse.equinox.p2.director -destination /nix/store/6zia9d3k5gxn3agl1xkssfmv46lyaxcy-eclipse-platform-4.22/eclipse -repository file:/nix/store/vspm6lm42bs9scfbh9gky03mybgpd576-eclipse-plugin-mat-1.9.1 -installIU<PHONE_NUMBER>59.Memory Analyzer IDE,201908260659.Memory Analyzer RCP,org.eclipse.mat.api,org.eclipse.mat.chart,org.eclipse.mat.chart.feature.feature.group,org.eclipse.mat.chart.feature.feature.jar,org.eclipse.mat.chart.ui,org.eclipse.mat.dtfj,org.eclipse.mat.feature.feature.group,org.eclipse.mat.feature.feature.jar,org.eclipse.mat.hprof,org.eclipse.mat.ibmdumps,org.eclipse.mat.jdt,org.eclipse.mat.jruby.resolver,org.eclipse.mat.parser,org.eclipse.mat.report,org.eclipse.mat.ui,org.eclipse.mat.ui.help,org.eclipse.mat.ui.rcp,org.eclipse.mat.ui.rcp.feature.feature.group,org.eclipse.mat.ui.rcp.feature.feature.jar,org.eclipse.mat.ui.rcp.feature_root,org.eclipse.mat.ui.rcp.feature_root.gtk.linux.ppc64,org.eclipse.mat.ui.rcp.feature_root.gtk.linux.x86,org.eclipse.mat.ui.rcp.feature_root.gtk.linux.x86_64,org.eclipse.mat.ui.rcp.feature_root.win32.win32.x86,org.eclipse.mat.ui.rcp.feature_root.win32.win32.x86_64, -tag AddIUs
!MESSAGE Unknown option Analyzer. Use -help for a list of known options.

Note that the comma-separated list supplied to the -installIU option starts like:
201908260659.Memory Analyzer IDE,201908260659.Memory Analyzer RCP,org.eclipse.mat.api,org.eclipse.mat.chart

and I'm guessing the spaces in Memory Analyzer IDE and Memory Analyzer RCP are confusing it.

--- a/pkgs/applications/editors/eclipse/default.nix
+++ b/pkgs/applications/editors/eclipse/default.nix
@@ -178,7 +178,7 @@ in rec {
           ${p2Director} \
             -destination $out/eclipse \
             -repository '${repository}' \
-            -installIU ${installIU} \
+            -installIU "${installIU}" \
             -tag AddIUs
           cat << EOF >> $out/eclipse/eclipse.ini
           ${jvmArgsText}

handles that, and lo and behold, I get a real error now:
Installation failed.
Cannot complete the install because one or more required items could not be found.
        Software being installed: Memory Analyzer for Eclipse IDE <IP_ADDRESS>Y3cLT2nh7E7Ag7cEU7UQg<PHONE_NUMBER>59.Memory Analyzer IDE <IP_ADDRESS>Y3cLT2nh7E7Ag7cEU7UQg)
        Missing requirement: Memory Analyzer (Charts) [optional] <IP_ADDRESS>908260659 (org.eclipse.mat.chart.feature.feature.group <IP_ADDRESS>908260659) requires 'org.eclipse.equinox.p2.iu; org.eclipse.birt.chart.engine 2.3.0' but it could not be found
        Cannot satisfy dependency:
                From: Memory Analyzer for Eclipse IDE <IP_ADDRESS>Y3cLT2nh7E7Ag7cEU7UQg<PHONE_NUMBER>59.Memory Analyzer IDE <IP_ADDRESS>Y3cLT2nh7E7Ag7cEU7UQg)
                To: org.eclipse.equinox.p2.iu; org.eclipse.mat.chart.feature.feature.group [<IP_ADDRESS>908260659,<IP_ADDRESS>908260659]

This looks like another example of the whole point of this exercise, yes?  If so, I like it!  :)

Ah, and if I instead try to install the MAT plugin on top of eclipse-jee, it satisfies that dependency but then fails on another one:
$ nix-build -I nixpkgs=$(pwd) -E 'with import <nixpkgs> {}; with pkgs.eclipses; eclipseWithPlugins { eclipse = eclipse-jee; plugins = [ plugins.mat ]; }' -K
...
Installation failed.
Cannot complete the install because one or more required items could not be found.
        Software being installed: Memory Analyzer for Eclipse IDE <IP_ADDRESS>Y3cLT2nh7E7Ag7cEU7UQg<PHONE_NUMBER>59.Memory Analyzer IDE <IP_ADDRESS>Y3cLT2nh7E7Ag7cEU7UQg)
        Missing requirement: Memory Analyzer (Charts) [optional] <IP_ADDRESS>908260659 (org.eclipse.mat.chart.feature.feature.group <IP_ADDRESS>908260659) requires 'org.eclipse.equinox.p2.iu; org.eclipse.birt.chart.engine 2.3.0' but it could not be found
        Cannot satisfy dependency:
                From: Memory Analyzer for Eclipse IDE <IP_ADDRESS>Y3cLT2nh7E7Ag7cEU7UQg<PHONE_NUMBER>59.Memory Analyzer IDE <IP_ADDRESS>Y3cLT2nh7E7Ag7cEU7UQg)
                To: org.eclipse.equinox.p2.iu; org.eclipse.mat.chart.feature.feature.group [<IP_ADDRESS>908260659,<IP_ADDRESS>908260659]

But, it says it's an optional dependency ... so should it really be failing?

I wonder if this change perhaps needs some editing of eclipse.section.md? Also, maybe it is of sufficient significance to add a brief note in the 22.05 release notes?

Yeah, I think I agree that we need some documentation updates.  The errors I'm seeing are, I think, good progress.  But, it's not at all obvious to me how to go about resolving them.  I think I need to declare the plugin's dependencies explicitly somehow, but I don't know how to do that.  And I expect other users might run into similar problems.

@rycee, yes you're right about those docs. Believe it or not I wasn't aware of them! Thank you for enlightening me, I'll push a commit with some appropriate changes. Same goes for the release notes.
@jerith666 Thank you for taking the time to look at this, and for catching the whitespace/quote mistake.
As for the issue installing the MAT plugin, I believe that's working as intended. (Don't worry the problem is solvable.)
There are a few key points here:

The [optional] part of the Memory Analyzer (Charts) [optional] feature is just a part of the name string. Meaning that it's intended to communicate optionality to the user and not to the resolver.
And in this case the user has elected to install it, and the resolver doesn't see this choice as optional. So if it can't install we get an error.

The behaviour of buildEclipseUpdateSite is to try to install all features and plugins at the update site, so naturally it tries to install the charts feature too. With the previous system it would have dumped the charts feature and all related plugin jars into the dropins dir and then just quietly failed to load them at runtime. The new system will tell you up-front that there is a problem.
So there are two options, we can either redefine the MAT plugin to only install only the main feature...
  mat = plugins.mirrorEclipsePlugins rec {
    pname = "mat";
    version = "<IP_ADDRESS>908260659";

    repository = "https://archive.eclipse.org/mat/1.9.1/update-site/";
    installableUnit = "org.eclipse.mat.feature.feature.group";
    hash = "sha256-usnlmTuLA/pAwh1706vwWtrqA4rPc4PGpODTq/aXWbw=";

    meta = with lib; {
      homepage = "https://www.eclipse.org/mat/";
      description = "The Eclipse Memory Analyzer is a fast and feature-rich Java heap analyzer that helps you find memory leaks and reduce memory consumption";
      license = licenses.epl10;
      platforms = platforms.all;
    };
  };

or we can give it enough info to find the correct dependencies...
  birt = plugins.mirrorEclipsePlugins rec {
    pname = "birt";
    version = "4.7.0.v201706222054";

    repository = "https://download.eclipse.org/releases/2021-12/";
    installableUnits = [
      "org.eclipse.birt.chart.device.extension"
      "org.eclipse.birt.chart.device.swt"
      "org.eclipse.birt.chart.engine.extension"
      "org.eclipse.birt.chart.engine"
      "org.eclipse.birt.core"
      { id = "org.mozilla.javascript"; version = "1.7.5.v201504281450"; }
    ];
    hash = "sha256-Gqcn0CjYMGbh2a4jPlEiEkr1zIWxWos7FW/Ac3QeP84=";
  };

  mat = plugins.mirrorEclipsePlugins rec {
    pname = "mat";
    version = "<IP_ADDRESS>908260659";

    repository = "https://archive.eclipse.org/mat/1.9.1/update-site/";
    installableUnits = [
      "org.eclipse.mat.feature.feature.group"
      "org.eclipse.mat.chart.feature.feature.group"
    ];
    hash = "sha256-3Xb61DruYJelI5GRQFCanaJnQu+Xzfw+5WfT+UPmch4=";

    dependencies = [ birt ];

    meta = with lib; {
      homepage = "https://www.eclipse.org/mat/";
      description = "The Eclipse Memory Analyzer is a fast and feature-rich Java heap analyzer that helps you find memory leaks and reduce memory consumption";
      license = licenses.epl10;
      platforms = platforms.all;
    };
  };

(Note that I had to push a few changes to get this to work.)

So you do have to jump through some hoops to get MAT working with the chart feature. It's awkward that you manually have to wire up the dependency on Birt.
In normal use of Eclipse when you install a plugin they can resolve dependencies from any of the update sites configured on the eclipse instance. The release repo https://download.eclipse.org/releases/2021-12/ would be configured by default, and so it'd be queried for the required BIRT plugins under the hood when you try to install MAT and everything would just work.
For now I think my way is fine, everything will work if you put a little extra energy in, you just have to be more explicit about dependencies. If you'll indulge me to speculate on some future work that could help here though...
We could achieve better ergonomics in this case if we could find a version of the release repo that had a fixed content hash and use that as a default dependency for plugins. I believe this would also have allowed you to install MAT directly on "eclipse-platform" and not just "eclipse-jee".
Now, the release repo is a composite repo, and I suspect the component repos, e.g. https://download.eclipse.org/releases/2021-12/202112081000/ have fixed content, but I can't find any documentation which guarantees this. We could probably also find a snapshot archive of it somewhere.
An alternative would be to build the release repo from source, which would be pretty cool. In turn we could define the different eclipses, "eclipse-jee" etc, in a consistent way in terms of "epp" meta-packages which depend on the release repo.
One thing at a time though.

Thanks, those examples are super helpful.  I'm trying to grok how you arrived at them ... I guess you have to install the plugin manually in order to figure out the value for the version (e.g. 4.7.0.v201706222054 for birt), right?  And it takes some iteration to find each missing dependency in turn as well, yes?

The workflow is pretty bad for manually finding dependencies to be honest. Yes one approach is just installing something manually and searching the list of installed packages to see what ends up getting installed. Something else I find myself doing is manually scanning through feature and package listings under update sites. Unfortunately not all third-party update sites provide a useful web view, and I'm not aware of any other (convenient) tools for querying them.
One reason things are so awkward is that I've smooshed together the concepts of "update site" and "plugin set to install" into a single entity. It's kind of difficult to describe the limitations and possible tradeoffs succinctly, but basically there are two steps to the process of preparing and installing a set of plugins:

Determine the subset of a p2 repository to mirror locally. This does not use the resolver, and doesn't properly chase dependencies into other repos, meaning more manual iteration is needed.
Determine the plugins from that repository to install. This does use the resolver, but can only operate over the repositories and plugins we've actually mirrored.

As I alluded to in the last comment, it would be possible to have better ergonomics for resolving dependencies (generally the same experience as manually installing plugins in Eclipse), by supporting mirroring important repositories in their entirety instead of just subsets. The costs of choosing to mirror an entire repo this way would be:

Much higher disk use during installation.

Difficulty of finding snapshot of repositories with a stable hash.

But this would be on an opt-in basis. Simple plugins and those already defined could continue to work the same way, but those with dependencies on plugins in the release repo, like MAT, would be much easier to define if the release repo were available in its entirety.
Sorry for dumping all these huge walls of text on you one after another, it might be a bit of an overload. But talking it out has helped me work through some things, I think I'll take a crack at separating out the "repository" and "plugin" concepts, see if I can land on a model that's easier to work with.

Thanks, that makes sense.  I agree it would be great to find a stable update site for each quarterly release and satisfy dependencies from there automatically somehow.
(And I for one appreciate a good wall of text -- I'm learning a lot by following this issue!  :) )

I did find a couple of small clean-ups for your patch; I've pushed them here: https://github.com/eliasvasylenko/nixpkgs/compare/eclipse-plugins-using-p2...jerith666:eclipse-plugins-using-p2

@eliasvasylenko: Do you want to pick this up again? (Currently conflicts with master.)