pydocstyle for init.py

This is for #742
I changed the format of the docstrings for cookiecutter/init.py to follow pep257 style

Codecov Report

Merging #998 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #998   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          18     18
  Lines         780    780
=====================================
  Hits          780    780

Impacted Files
Coverage Δ

cookiecutter/__init__.py
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5503196...070849e. Read the comment docs.

Thank you for your contribution @meahow! 🍪


Add Python SDK Tuner example

Good work, looks really nice!  I know it aint ready yet but here already some comments you might already consider or they can be added!

Explain why some trials my fail (this can be because the dataset for instance is inbalanced and the DSP can only work with certain amount of classes)
Make "Setting constraints" more prominent, throw in  a real life usecase (in a sentence)
Write how to get the logs of those failed trials to find the root cause
Move the dataframe solution to the top (this is so people can get wow'ed and then go for the painful json version)

Then additionally:

I'll ship a better upload functio that chooses the train/test automatically this week
I'll add a dataset download function that automately gets and extracts the data in a folder.

You can see those coming in later today


[Feature Request]  A conf option that allows to already select immediately after appending to the list of items.

[Feature Request]  A conf option that allows to already select immediately after appending to the list of items.
I request this because when working a lot with virtual machines I need to specifically select it from rofi to be able to copy/paste the item to the host machine in one go.
In other words I Copy , open rofi-greenclip select the most recent entry and select again with keyboard or mouse.
ONLY then I am able to copy it to the host.
Happens in VMware workstaion and Virtualbox.
Greenclip & rofi are also in use on the host.
Hoping this makes sense.
Thanks for this amazing application.

Yes, it would be great if the developers add this feature so that the copied can be pasted even if the window from which the copy was made is closed without resorting to greenclip.

You can select the recent item through xclip directly & paste it directly into the target. Makes even easier for you. Here's my setup btw.

This feature is called force retaking of clipboard ownership. Users have already asked
about it, but @erebe said that it is not possible now, unfortunately.
I came to greenchip from clipmenu for MIME support, but I lost force retaking of clipboard
ownership, sadly.
Duplicate of:

#123


Occasional errors occur

PythonError: Traceback (most recent call last):
  File "/lib/python3.11/site-packages/micropip/_micropip.py", line 48, in _get_pypi_json
    metadata = await fetch_string(url, fetch_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.11/site-packages/micropip/_compat_in_pyodide.py", line 34, in fetch_string
    return await (await pyfetch(url, **kwargs)).string()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python311.zip/pyodide/http.py", line 254, in pyfetch
    raise OSError(e.message) from None
OSError: fetch failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/lib/python3.11/site-packages/micropip/_micropip.py", line 576, in install
    await transaction.gather_requirements(requirements)
  File "/lib/python3.11/site-packages/micropip/_micropip.py", line 342, in gather_requirements
    await gather(*requirement_promises)
  File "/lib/python3.11/site-packages/micropip/_micropip.py", line 349, in add_requirement
    return await self.add_requirement_inner(Requirement(req))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.11/site-packages/micropip/_micropip.py", line 441, in add_requirement_inner
    metadata = await _get_pypi_json(req.name, self.fetch_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.11/site-packages/micropip/_micropip.py", line 50, in _get_pypi_json
    raise ValueError(
ValueError: Can't fetch metadata for 'pypdf' from PyPI. Please make sure you have entered a correct package name.

May be related to terminal.

The v0.0.4 version has been changed to local loading, It's not a matter.


Why

[config.py:20] - 'gbk' codec can't decode byte 0xa4 in position 245: illegal multibyte sequence

No encoding type was specified when reading the configuration file, I fixed it, please pull it again and try again.


Don't deepfreeze on each environmental action

Whenever I use hatch run with pip-deepfreeze environments, it seems like it tries to perform pip install and depfreeze, regardless of whether dependencies were changed in the pyproject.toml or not.
For example, it takes almost 10 seconds just to display current python version in such env:
No change to /Users/bobronium/.../requirements-all.txt
No change to /Users/bobronium/.../requirements.txt
Python 3.9.16
hatch run python --version  9.44s user 1.31s system 86% cpu 12.495 total

This is what I get without type = "pip-deepfreeze"
hatch run python --version  0.51s user 0.11s system 98% cpu 0.626 total

Ideally, I would expect deepfreeze to run only when hatch decides to sync environment AND finds something to be changed.

@Bobronium thanks for your interest !
Removing the forced sync raises the question of what it means for the environment to be in sync.
If I remember correctly, hatch only checks that the dependencies are installed?
But if one uses this plugs, it is is presumably because one wants precise versions to be installed.
So what we would need is a more sophisticated implementation of dependencies_in_sync(), which is also not trivial.

Right, my bad. Well, I don't have a quick solution here, unfortunately.
And maybe it's worth to move our focus on implementing this in hatch, since https://github.com/pypa/pip/pull/10748, which seemingly was a blocker for this, is resolved now.


[BUG] mergeObjects does not effect the root document

Given the collection C and the document {_id:'C/1',_key:'1',test:1,test2:2}
If running
UPDATE {_key:'1', test2:1} INTO Client OPTIONS {"mergeObjects":false} RETURN NEW

The field test will still be in the document.
Given the document {_id:'C/2',_key:'2',test:{test:1, test2:2}}
Running
UPDATE {_key:'2', test:{test:2}} INTO Client OPTIONS {"mergeObjects":false} RETURN NEW

This will return {_id:'C/2',_key:'2',test:{test:2}}
So the sub documents are not merged but the root document.
I'm on arangodb 3.12 with the java driver 7.6. But I reproduced it in the web frontend.

This is the expected behavior. With OPTIONS {"mergeObjects": false}, the attributes you specify will be set to the specified values. Attributes you don't specify remain, regardless of mergeObjects. If you want to remove attributes selectively, set them to null in an UPDATE operation with OPTIONS { keepNull: false }:
UPDATE { _key: "C", test2: null } IN Client OPTIONS { keepNull: false }

Or you can consider specifying all attributes of the document you want to keep and use a REPLACE operation, but be careful not to remove more attributes than desired - only the ones stated in the query plus immutable system attributes will remain.

Hi @konsultaner, the implemented behavior of mergeObjects is

for any attributes that are present in the overwrite value but not in the source document:

add all attributes to the document which are present in the overwrite value but not in the source document. A special case is when keepNull=false is removed, in which case the attributes which are null in the overwrite attribute value will be removed from the target document.

for any attributes that are present in the overwrite value and also in the source document:

if mergeObjects=false, overwrite the value of the attribute in the source document with the overwrite attribute value.
if mergeObjects=true, overwrite the value of the attribute in the source document with the overwrite attribute value if the attribute value in the source document is not an object (i.e. merging not possible, so we are overwriting).
if mergeObjects=true, merge the value of attribute in the source document with the overwrite attribute value if they are both objects (merging possible).

The algorithm is not performed on the very top level (i.e. the source document and the overwrite value). Instead it is performed for every top-level attribute present in the overwrite value individually. When merging two attribute values objects, the merging is also performed recursively using the same algorithm.
I understand that this behavior may not always be the desired behavior, but the merging algorithm shouldn't be changed in an incompatible way to avoid incompatibility. In case a different merging algorithm is needed, there are multiple options:

implement the custom merging in the client application (requires fetching the original document first, so may not work for large amounts and is subject to data races if there are other concurrent update operations).
implement the custom merging as a new option in the database. This would mean another hard-coded merging algorithm, which may or may not be general enough to suit everyone's needs. Also requires support from all relevant drivers in order to be able to use it properly.
implement the custom merging in the AQL query, using built-in AQL functions (example following)

FOR doc IN collection
  REPLACE doc WITH MERGE_RECURSIVE(@update) IN collection

Note that this query is not providing a different merge algorithm, but it may be adjusted to your needs. I currently can't tell what merge algorithm you have in mind.


[SPARK-14736][core] Deadlock in registering applications while the Master is in the RECOVERING mode

What changes were proposed in this pull request?
this PR fixes the issue SPARK-14736 Deadlock in registering applications while the Master is in the RECOVERING mode.
Proposed solution is to keep the registering apps in a separate list when the Master is in the RECOVERING mode and once the recovery is complete, these apps will be registered back. Pls refer the JIRA for more information
How was this patch tested?
I have tested the patch manually

Can one of the admins verify this patch?

You will need to make this thread safe - the applications are added/re-registered from separate threads, right ?

Maybe we should correct the title just like the others. Also, the title looks truncated.

@mridulm do you mean to say, the method private def registerApplication(app: ApplicationInfo): Unit = { needs to synchronized?

No.
Updates to waitingAppsWhileRecovering happens from different threads right ? Hence you will need to protect it.

@mridulm You are correct. But if we check the register application method,
`  private def registerApplication(app: ApplicationInfo): Unit = {
val appAddress = app.driver.address
if (addressToApp.contains(appAddress)) {
logInfo("Attempted to re-register application at same address: " + appAddress)
return
}
applicationMetricsSystem.registerSource(app.appSource)
apps += app
idToApp(app.id) = app
endpointToApp(app.driver) = app
addressToApp(appAddress) = app
waitingApps += app

}`
there are similar array buffers which are getting updated without being synchronized. That is why I omitted making the waitingAppsWhileRecovering synchronized.
am I doing anything wrong there?

No, you are right - this is called only from the event loop - which should ensure thread safety.
I misread where the re-registeration was happening as outside of the event loop.
Please ignore my comment.

Great! can we get this PR merged then? Is there anything else I should do in order to get this merged?

Hi @nirandaperera , I'm not too sure about the Master recovery process so I can't really comment on your code, but it would make a much stronger case for this PR if you could include a test that fails without this change.

@BryanCutler I was looking for some unit tests which could simulate this scenario in the Master.scala class, but I couldn't find any. But i think I can reproduce this in an integration test environment. can you point me to the spark integration tests?

ok to test

Test build #58162 has started for PR 12506 at commit 17e7949.

Test build #58162 has finished for PR 12506 at commit 17e7949.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58162/
Test PASSed.

@andrewor14 did you review this?

Looks reasonable to me but I don't know this code very well. Also @kayousterhout maybe or @squito

@aarondav might be the right person to look at this -- looks like he wrote most of the original code around the "RECOVERING" state way back in 2013.

unfortunately I'm not very knowledgeable here either.  I agree that this change looks reasonable, but also wish there was a test case for it.  I don't think there is any good integration test framework, and it seems there aren't any tests for recovery state now, so you'd have to build that out yourself.  One possibility -- "local-cluster" mode is closely related to a standalone cluster -- maybe that could be used to create a test?

@squito hey. Thanks for the heads up. let me check on that and try to come up with a test

Are you still working on this? @nirandaperera


introduce ip whitelist for OnlyStaffMiddleware

please merge

no need for local ip whitelist as long settings.DEBUG ist true, moved feinheit ip to fab server.create_database_and_dotenv

After team discussion we have decided not to preset any IPs.


feat: add search options used to the module list response

Fixes #165

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


The REPL prompt shows up too late

Juha: the scala> prompt shows up too late
(Olli: does it happen only once in the beginning or after each execution)

https://youtrack.jetbrains.com/issue/SCL-20177/Scala-3-REPL-prompt-"scala>"-appears-only-after-the-first-line-i

Waiting for a response from JetBrains. Check again towards the start of the semester!

Check the situation at the end of June, and make the decision what to do.

Seems to work in the "runIde" Gradle task (I don't think it worked before), investigate further


[Aviraj] Added blog how-to-create-an-nft-with-royalties-on-celo.md

Hey @avirajkhare00 Just made few changes, please look into it.
Greate tutorial btw, keep it up.

Did you add this tutorial in users.tsx file?

Hey @viral-sangani, I have accepted all the changes. Do you mean adding the tutorial inside src/data-tutorials/users.tsx file?


Can I using AD Identirymodel extensions to build a Microsoft Identity Planform?

I'm learning to build an Oauth2 and OpenID. I found that .Net doesn't provide a ready solution for it. I wonder if Microsoft used this repo to build the Microsoft identity platform?. Is there any document or guide on how to build a similar system using the open-source code provided by Microsoft?

@QuanJingHong what services are you looking to provide?

@QuanJingHong what services are you looking to provide?

I want to build a Server which to provides IdentityServer-like or Microsoft Identity platform-like features.

You can start with this class.
Closing as answered.


http.StripPrefix not properly handling wildcards with muxes

Go version
go1.22.1.linux-amd64
Output of go env in your module/workspace:
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/stan/.cache/go-build'
GOENV='/home/stan/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/stan/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/stan/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.1'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/stan/Documents/code/bluerpc/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build923806106=/tmp/go-build -gno-record-gcc-switches'

What did you do?
https://go.dev/play/p/R_kkd-iAPI0
What did you see happen?
If you try to create a nested http mux on a wildcard route using http.StripPrefix it will not work. In the earlier example you can see that the response is "body 404 page not found"
What did you expect to see?
I expected to see the mux handle function to go through and get "body Hello" printed
If I modify the earlier example and do a request on /{wildcard}/hello verbatim the request does go through :
https://go.dev/play/p/omUCziNc8Bo

Duplicate of #64909


[Bug][Github Plugin] Panic during github_graphql task, specifically when grabbing deployments

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
When obtaining github data from my private repo, the devlake process crashes with a panic.
Based on where it is in the logs, this seems to happen on the second graphql query as part of CollectAndExtractDeployments.
Based on the stacktrace, this presumably happens because we are passing nil and nil as query variables to the graphql query library. This nil seems to come from BuildQuery within github deployment parsing code
https://github.com/apache/incubator-devlake/blob/df1a1b9288f8fb29c6733aad220dd94cdb332f3d/backend/plugins/github_graphql/tasks/deployment_collector_extractor.go#L112-L115
Full stacktrace:
[signal SIGSEGV: segmentation violation code=0x1 addr=0x98 pc=0x7f6867667841]

goroutine 2277 [running]:
github.com/merico-dev/graphql.writeQuery({0x1aa4a40, 0xc00159f6b0}, {0x0, 0x0?}, 0x0, 0xbecda0?)
	/go/pkg/mod/github.com/merico-dev/graphql@v0.0.0-20221027131946-77460a1fd4cd/query.go:112 +0x61
github.com/merico-dev/graphql.query({0x0, 0x0}, 0x227?)
	/go/pkg/mod/github.com/merico-dev/graphql@v0.0.0-20221027131946-77460a1fd4cd/query.go:105 +0x7f
github.com/merico-dev/graphql.ConstructQuery({0x0?, 0x0?}, 0x0)
	/go/pkg/mod/github.com/merico-dev/graphql@v0.0.0-20221027131946-77460a1fd4cd/query.go:16 +0x46
github.com/merico-dev/graphql.(*Client).Query(0xc0014c9728?, {0x1ab4248, 0xc0015ed630}, {0x0, 0x0}, 0x0?)
	/go/pkg/mod/github.com/merico-dev/graphql@v0.0.0-20221027131946-77460a1fd4cd/graphql.go:38 +0x5f
github.com/apache/incubator-devlake/helpers/pluginhelper/api.(*GraphqlAsyncClient).Query(0xc0015aa780, {0x0, 0x0}, 0x0)
	/app/helpers/pluginhelper/api/graphql_async_client.go:167 +0x2be
github.com/apache/incubator-devlake/helpers/pluginhelper/api.(*GraphqlCollector).fetchAsync(0xc0021acc90, 0x0?, 0xc00211e6d8?, 0xc0014e85d0)
	/app/helpers/pluginhelper/api/graphql_collector.go:379 +0x252
github.com/apache/incubator-devlake/helpers/pluginhelper/api.(*GraphqlCollector).fetchOneByOne.func1.1()
	/app/helpers/pluginhelper/api/graphql_collector.go:253 +0x12d
github.com/apache/incubator-devlake/helpers/pluginhelper/api.(*GraphqlAsyncClient).NextTick.func1.1()
	/app/helpers/pluginhelper/api/graphql_async_client.go:206 +0x6b
created by github.com/apache/incubator-devlake/helpers/pluginhelper/api.(*GraphqlAsyncClient).NextTick.func1
	/app/helpers/pluginhelper/api/graphql_async_client.go:201 +0xbf

What do you expect to happen
The import task to complete successfully.
How to reproduce
I'm not 100% sure what the requirement on the state of the repo is, the enableSince is used for paging? So a repo with a sufficiently long list of deployments to exceed the size of a single page should be able to reproduce this?
But basically, setup

github repo connector with CI/CD scope enabled
create project that uses that connector
click 'collect data'
observe 'task failed, the process was terminated unexpectedly' error in the UI during the github_graphql task.
observe the above stacktrace in the logs of the devlake-lake process.

Anything else
No response
Version
v0.20.0-beta2
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Seems this has been introduced in this PR: https://github.com/apache/incubator-devlake/commit/e828cadb33ea7afc36a75b27ffb56331dafe6b46#diff-2a340770868d07c1add17eef1b9940aec95b7ba493925bf97a81a13592c6155dR113-R115
so @abeizn you may be the best person suited to propose a fix here.

Based on the graphql library, i think it may be because variables is nil and probably needs to be an empty map instead?
https://github.com/merico-dev/graphql/blob/77460a1fd4cdbf5a0716e01fc00eda4fe50871a3/query.go#L103-L107
which calls
https://github.com/merico-dev/graphql/blob/77460a1fd4cdbf5a0716e01fc00eda4fe50871a3/query.go#L111-L112
and so it's doing a nil.Kind(), I believe.

thanks for the quick fix @abeizn !


Broken link for troubleshooting installation

Hi,
the link to the troubleshooting guide in the readme appears to be broken - https://aiverify-foundation.github.io/moonshot/common_issues/

hello, thanks for bringing up this issue. we are aware and have already made a fix for this issue which we will be merging it soon.
do refer to this link for the troubleshooting guides instead. https://aiverify-foundation.github.io/moonshot/faq/


Add tags for 0.0.7 and 0.0.8

Would you mind pushing the tags for 0.0.7 and 0.0.8 so that they show up in the list of Releases?

done


[Event Modification] codeunit 5815 "Undo Sales Shipment Line" OnAfterCopyItemJnlLineFromSalesShpt

Can you please add the following to an event:
        OnAfterCopyItemJnlLineFromSalesShpt(ItemJnlLine, SalesShptHeader, SalesShptLine, TempWhseJnlLine);

to
        OnAfterCopyItemJnlLineFromSalesShpt(ItemJnlLine, SalesShptHeader, SalesShptLine, TempWhseJnlLine,WhseUndoQty);

Is it to be var, like your other requests submitted around the same time?

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


Use 64 bit unsigned integers for masks

This way we can have up to 64 components with no extra masks... when the number of components exceeds 64 we will need to consider a new way to contain them
Also use double for components, recommended here: http://wiki.luajit.org/Numerical-Computing-Performance-Guide

Don't merge... this is just a test, it requires LuaJIT 2.1, due to needing BitOps for 64 bits... I don't know if that's supported yet


There should be an option to turn off the anti-replay feature

Anti-replay is on by default. This is implemented by keeping a cache on the server of all the sessions that have been issued. When a request comes in with early data, we only accept that early data if the session is in the cache. We then remove the session from the cache to prevent re-use of the ticket.
This means that you cannot have entirely stateless session tickets in TLSv1.3 at the same time as using early_data. It also has resource implications for servers, and throws up issues around servers operating in clusters with external session caches.
For some applications this strict anti-replay approach may not be necessary. It could be that the application is designed in such a way as to manage the replay risks in another way.
There should be an option to turn off the automatic anti-replay feature. There may also need to be an API that enables an application to decide whether to accept early data or not.

Also see #6313.


[BUG] Builds fail due to SnapshotUtils.filterIndices() not being resolved

Describe the bug
Failing due to filterIndices() method being moved from SnapshotUtils to IndexUtils as part of a recent commit [#8922](https://github.com/opensearch-project/OpenSearch/pull/8922
Related component
Build
To Reproduce

Pull in latest code
Build

Expected behavior
Builds without issues
Additional Details
No response

Oops, meant to open this issue in the Security Project. You can close this one as I will raise it in there!


Update design page

Summary

Result

Awesome work @LukyVj it looks so good! 👏
I have some remarks:

It looks like the whole button should be clickable?

Did you tried with a white hover text color?

The left side menu has a scrollbar inside itself and does not take the full height space of the page

We lost the syntax color into code examples.
The get started CTA is only the third one, I expect from a documentation website to be almost the first thing visible to encourage users to start coding!
I think the get started link does not fit with the no framework needed block but more with the widgets one.

And again thank's a it's so much better <3
If you need any help for anything just ask!

The get started CTA is only the third one, I expect from a documentation website to be almost the first thing visible to encourage users to start coding!

This one is on me. We'll iterate :)

Zip for the examples are not accessible.

Hey guys, thanks for all the comments, I'll take care of everything but for today I'm switching on the meetup video edit! Thx

This looks gorgeous, small comment on:

I cannot understand the design of this button easily, so I don't get what this button does.
I like what we did on react-instantsearch, straight to the point
. Maybe we could have something similar?

Hello! 👋
@vvo @bobylito @LukyVj
I used this icon because it's something I'm used to seeing around to web for both 'copy to clipboard' and 'duplicate'
I usually go to thenounproject to make sure is not something subjective:

In my opinion, if there's an icon for a simple action like this one it's better to use that than text as it's easier to process quickly and it separates better from the written word which in this case is abundant. However, I can't speak for a user of documentation, if you guys think it's still better to go with text I don't think it's a 'design breaker' 🤷♀

Indeed as a dev it was not obvious what this button does, it looks very similar to open in a new window:
, from https://thenounproject.com/search/?q=popup window&i=539930
We could either have a nice "copy" (no need for copy to clipboard) button, much like we have "edit on github". If we want to go for an icon, that one is more often seen in documentations:
, from https://clipboardjs.com/assets/images/clippy.svg

Let's go for copy then. It can be revised later.

So, I arrive a bit after the war, but I don't see the problem of having an icon instead of text.
Plus, we do it on the dashboard for our customers, most of them are regular profiles I guess ( not so dev in some cases ) and we still show them this icon.
I'm not strongly for having an icon instead of text, but since we do it on algolia.com, I don't see why it would be a problem on a community website.

So if I sum up:

this icon can be confusing (@vvo didn't get the meaning of this icon)
icons are better, especially for such icons. But needs to be easily recognizable
if something is more universal than the other, let's choose the more universal

@auareyoualgolia do you have another proposal of icon that would be more universal?

The one on algolia.com works better to me

I agree with you @vvo, it's true that the one on the design is more for "open in a new page" but in the end, I think we can find a better one (reusing the one of the dashboard?) and it will bring design consistency across our different pages.
And as Sebastien says, good design = consistent design = trust

I think reusing the one from the dashboard would make sense too. I thought it a slightly different use but it is true it's much more clear

Heyo
So I fixed all the above listed bugs/missing stuff
Do you have a list of what's missing?


Allow integration with i18n providers

We should be able to fetch JSON translations from i18n providers as long as they support exporting. Ideally, this should be a plugin system where each provider is a standalone Nuget package that can be easily integrated.
The minimum supported features would be something along the lines of

[ ] Fetch translations from API
[ ] Do any custom transformations necessary (such as unzipping etc.)
[ ] Add them to the build process as an additional source

For reference, here is a non-exhaustive list of providers

[ ] Weblate
[ ] Crowdin
[ ] Localazy
[ ] Locize
[ ] Tolgee (@curtisy1)

They're all up for grabs, since maintaining all of these is too tedious. So when you need them, please contribute!

Since I integrated Tolgee as a PoC for #17, I'll tackle it and see how I can turn it into a plugin

Unfortunately, integrating this via separate NuGet packages is not as easy as I thought for the following reasons:
As described in the source generator docs, source generators run unordered this has a few caveats, especially the following:

we can never be sure whether our plugins execute before our main generator
we cannot access the files we downloaded from our i18n provider
Init hooks are limited in a way that we cannot print helpful debug messages to the user

As such, the main repository now contains a plugins folder that has all existing plugins inside. When implementing, you can simply inherit from the abstract base class and override the Fetch method to return your desired files. For details, see #18 for an implementation


Unbound / Blocklists: Replace JustDomains with Firebog

The JustDomains sublists haven't been updated since October 3, 2022 and appears to be abandoned.
This replaces them with the Firebog equivalents, which is still active.
Fixes #6610

@kabili207 looks like a good replacement, thanks, would you mind updating the docs as well? (https://github.com/opnsense/docs/blob/08b77849658ec6997dc924a6c662b17822843706/source/manual/unbound.rst#L3280


Problems with Manimgl

Describe the error

I'm getting weird graphics when I run the examples_scene and also many warnings. I have installed all miktex packages and deleted the \usepackage[T1]{fontenc} but I'm still not able to make it work. I'm not sure what could be wrong at this point.
Code and Error
Code:

Error:

Environment
OS System:
manim version: master
python version:

Have you solved it?I recently came across the same issue in my application.


Include search templates in package.

This should allow the library to be used outside of editable mode.

Coverage remained the same at 92.979% when pulling 0d543eab51ce2232dc0fbb9f722ad8c7d47c6897 on cmc333333:package-files into<PHONE_NUMBER>e24fce0c2fabf0985d67272ea8c084 on eregs:master.

Coverage remained the same at 92.979% when pulling 0d543eab51ce2232dc0fbb9f722ad8c7d47c6897 on cmc333333:package-files into<PHONE_NUMBER>e24fce0c2fabf0985d67272ea8c084 on eregs:master.

Coverage remained the same at 92.979% when pulling 0d543eab51ce2232dc0fbb9f722ad8c7d47c6897 on cmc333333:package-files into<PHONE_NUMBER>e24fce0c2fabf0985d67272ea8c084 on eregs:master.

Coverage remained the same at 92.979% when pulling a56e186426f2ecd78c85eb4680a89ef9f76694cf on cmc333333:package-files into<PHONE_NUMBER>e24fce0c2fabf0985d67272ea8c084 on eregs:master.

Coverage remained the same at 92.979% when pulling a56e186426f2ecd78c85eb4680a89ef9f76694cf on cmc333333:package-files into<PHONE_NUMBER>e24fce0c2fabf0985d67272ea8c084 on eregs:master.

Coverage remained the same at 92.979% when pulling a56e186426f2ecd78c85eb4680a89ef9f76694cf on cmc333333:package-files into<PHONE_NUMBER>e24fce0c2fabf0985d67272ea8c084 on eregs:master.

Bumped the app version just to remove one step from cutting a new release.

Coverage remained the same at 92.979% when pulling c7d36dd6bdafbd4bb06befdf4613e1ccbfbd5bcc on cmc333333:package-files into<PHONE_NUMBER>e24fce0c2fabf0985d67272ea8c084 on eregs:master.

Coverage remained the same at 92.979% when pulling f49297d05d39f19488e3819cf39b0cc39f562cb1 on cmc333333:package-files into<PHONE_NUMBER>e24fce0c2fabf0985d67272ea8c084 on eregs:master.

Awesome as always.


Working on early transitions

Before

After

+100

On Aug 5, 2016, at 08:23, Patricio Gonzalez Vivo<EMAIL_ADDRESS>wrote:
Before
After
―
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

I was thinking in do a fade in and out? What do you think... I don't like how suddenly disapear

I think this is the final version

:)
On Fri, Aug 5, 2016 at 8:45 AM, Patricio Gonzalez Vivo <
<EMAIL_ADDRESS>wrote:

I think this is the final version
[image: 02]
https://cloud.githubusercontent.com/assets/346914/17441999/bfd87062-5b01-11e6-9709-2f8793d2e579.gif
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tangrams/tron/issues/31#issuecomment-237886094, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA0EO5eafLnR4aXBFLECyN0-aFCPr6m6ks5qc1qCgaJpZM4Jdycq
.

Here is the commit https://github.com/tangrams/tron/commit/9942ee44916d19f07193f967f0058569cc0c88da


Fix: Editing Badge Attributes

Currently, our backend does not store attributes and is solely in the IPFS metadata hash. For editing, we need to have these attributes appear in the edit form.
Going to throw together a function on the front end that fetches the URI and parses the attributes and adds them to the edit form, but it would provide less friction if we could store these on the backend. @nftchance thoughts on this?

Yeah didn't realize that wasn't in the backend -- I can add it, ez pz

Currently functional in production by parsing the attributes from token URI. Documented the functions accomplishing this and can easily swap them out when API support arrives.

I wonder if this may also impact Manager lists.
Note from a user (Mar1na):
When changing badge attributes (I tried changing the description a few times) it clears out any managers
My suspicion:
Managers are currently stored as an approved transactor list and when editing settings of a Badge, all fields are re-established.
Can make another ticket if needed

@drakedanner Appreciate the report. Was a very small issue that I took the liberty of pushing a hot fix for. Not a huge change at all, just an issue with how I was shaping the data for the transaction was actually changing the source data instead of acting as a new shaped object.

@masonchain does that mean we should expect the manager list to remain the same when editing attributes now?
If so, I'll alert the user and let them know to re-report if they have this issue again.
Thanks sir!

@drakedanner Correct. Was just a small issue so slapped a bandaid on it before the webhooks refactor


Perfecto upload 0.0.1

What to do if the build fails?
At the moment contributors do not have access to the CI workflow triggered by StepLib PRs. In case of a failed build, we ask for your patience. Maintainers of Bitrise Steplib will sort it out for you or inform you if any further action is needed.
New Pull Request Checklist
Please mark the points which you did / accept.

[x] I will not move an already shared step version's tag to another commit
[x] I read the Step Development Guideline
[ ] I have a test for my Step, which can be run with bitrise run test (in the step's repository)
[x] I did run bitrise run audit-this-step (in the step's repository - note, if you don't have this workflow in your bitrise.yml, you can copy it from the step template.)
[x] I read and accept the Abandoned Step policy

New Step
Thank you for the new Step share! The CI check might will fail due to our extended validation engine. Nothing to worry about yet, we will get back to you shortly.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hi @sreevatsa-ramya
your step.yml seems invalid, please check our contribution guidelines.
Invalid spec /steps/perfecto-upload/info: {} "maintainer" value is required

Hi,
Please let me know where should I specify the "maintainer" value in
step.yml. I checked step guidelines and also multiple steps in the
bitrise repo but I couldn't find it anywhere. Please advise
BR,
Sreevatsa
On Fri, Jun 12, 2020 at 7:47 PM Szabados Áron<EMAIL_ADDRESS>wrote:

Hi @sreevatsa-ramya https://github.com/sreevatsa-ramya
your step.yml seems invalid, please check our contribution guidelines.
Invalid spec /steps/perfecto-upload/info: {} "maintainer" value is required
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bitrise-io/bitrise-steplib/pull/2549#issuecomment-643293851,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABLKBFDDHK5JYLSEMEQWMW3RWI2HBANCNFSM4N4JXJMA
.

Hi,
I have done the changes and have shared the step again.
Thanks,
Sree
On Sun, Jun 14, 2020 at 3:58 PM Tamas Papik<EMAIL_ADDRESS>wrote:

@trapacska requested changes on this pull request.
Hi @sreevatsa-ramya https://github.com/sreevatsa-ramya !
Thank you for the new Step!
Please see my comment, and after you've finished the changes please do a
fresh share again so your changes will be applied to the step version
you've sent in.
Also, the maintainer field:
content:
maintainer: community
path:
steps/perfecto-upload/step-info.yml
In steps/perfecto-upload/0.0.1/step.yml
https://github.com/bitrise-io/bitrise-steplib/pull/2549#discussion_r439814471
:

@@ -0,0 +1,74 @@
+title: perfecto-upload

Please use a more UX friendly name here, for example Perfecto Upload.
This is the name the users will see in the workflow editor step selector.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bitrise-io/bitrise-steplib/pull/2549#pullrequestreview-430206104,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABLKBFDSSIB2XGMQS333AYTRWSQ4RANCNFSM4N4JXJMA
.

There should be one commit for PR (except optional one for icon).


Prevent empty errors

@rwjblue
This fix should prevent adding empty or null errors to errors array.
Before:
['','','','','error',] // self._errors

After
['error'] // self._errors

It will fix an issue with inproper fomratting and counting. #273

Awesome, thank you!


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-yml-archive-non-versioning-archive/pull/514 is merged into live branch.
Auto archive content to https://github.com/opstest2/docs-yml-archive-non-versioning-archive.git

Docs Build status updates of commit ccf3925:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

docs/TOC.yml
:white_check_mark:Succeeded
View

docs/to-be-archive/level1/article3-1-1.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level1/article3-1-2.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level1/article3-1-3.yml
:white_check_mark:Succeeded
View

docs/to-be-archive/level2/article3-2-1.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level2/article3-2-2.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level2/article3-2-3.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


[Automated] Update eventing-kafka-broker-eventing-kafka nightly

/assign @knative/delivery-wg-leads
/cc @knative/delivery-wg-leads
Produced via:
for x in channel-crds.yaml source-crd.yaml; do
  curl https://storage.googleapis.com/knative-nightly/eventing-kafka/latest/$x > ${GITHUB_WORKSPACE}/./third_party/eventing-kafka-latest/$x
done

Details:

/test unit-tests_eventing-kafka-broker_main

/retest


Add a conformance Test for HTTPS listener with multiple TLS certs

What type of PR is this?
/kind feature

What this PR does / why we need it:
Add a conformance Test for  HTTPS listener with multiple TLS certs
Which issue(s) this PR fixes:

Fixes #1330
Does this PR introduce a user-facing change?:
NONE

/ok-to-test

@tokers Thanks for your comments. I have fixed them. Will you be able to review the PR?

I am a bit confused. Multi certs are Implementation-specific per the spec. How can we have a conformance tests for something Implementation-specific?

@howardjohn Not sure my understanding is correct. But in the document, It says when multiple certs are configured, GW should use the longest matching SNI. (It's under support: core). I think the longest SNI matching falls under support core but more than one secret refs fall under Implementation specific

https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1beta1.GatewayTLSConfig
But supporting multiple certs  are under support: Implementation specific

Rebased

This feels like an example of https://github.com/kubernetes-sigs/gateway-api/issues/640 - we may need to revisit how support levels are specified here, and if some subset like "more than one Kubernetes Secret" should actually be Extended support with an opt-in conformance test like this.

@robscott This seems intended to address the conformance tests requested in https://github.com/kubernetes-sigs/gateway-api/issues/851#issuecomment-1201753415, thoughts on whether that GEP already covers the expected conformance sufficiently, or if we need further discussion on what level this should be?

@ksankeerth it's been a bit since we've heard back from you, are you still able to continue with this PR?

@ksankeerth we haven't heard back from you in some time, we're going to mark this one as a draft for the time being since there's a few conflicts that need to be resolved. Let us know how we can help you if you're still interested in this one! And if you're not interested that's totally fine, just let us know and we'll unassign!

@shaneutt Apologies for the very late reply. I think there is some confusion in multi-tls support as Implementation specific.
https://github.com/kubernetes-sigs/gateway-api/pull/1526#discussion_r1019755797

I think this is going to need some more discussion. If we say that supporting multiple TLS certificates is "Extended" support, we'll need to define how multiple TLS certs should be handled. I'm not entirely sure what that looks like today, but it could be worth getting some input from anyone currently supporting that.

One concrete change here would be to limit the scope of "Extended" support to Kubernetes TLS secrets. That would result in the following:

As per the doc,https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1beta1.Listener

TLS is the TLS configuration for the Listener. This field is required if the Protocol field is “HTTPS” or “TLS”. It is invalid to set this field if the Protocol field is “HTTP”, “TCP”, or “UDP”.

The association of SNIs to Certificate defined in GatewayTLSConfig is defined based on the Hostname field for this listener.

The GatewayClass MUST use the longest matching SNI out of all available certificates for any TLS handshake.

Support: Core

If GW has to support the longest SNI matching as core support, that indirectly means GWs have multiple certificates.  However, support for multi-certificates is currently mentioned as  Implementation specific. Later, I raised this PR to mark it as extended support. I believe there is no conclusion on that https://github.com/kubernetes-sigs/gateway-api/pull/1526.  (Sorry If I missed any discussions on this)

"multiple certificates" feature is for multiple under a single Listener. I think the text you quoted is about having multiple by having N listeners with 1 certificate each


Cloud Speech sample

I'm trying to use the  speech sample to transcribe a long file. I'm using a bucket. The problem seems to be in the BackoffExponential which stops the operation after a certain number of iterations.
Using the Cloud Console on the same file terminates correctly but I would prefer to use PHP in order to concatenate the transcription.
Pete

Pete,
Is this the sample you have been working with? You should be able to increase the number of retries passed into the ExponentialBackoff instance from 10 to whatever seems reasonable for your audio length. I would recommend doubling the value up to 20 and see if you need to adjust from there. :)
Alternatively, you could poll indefinitely until the operation is complete:
$isComplete = $operation->isComplete();

while (!$isComplete) {
    sleep(3); // let's wait for a moment...
    $operation->reload();
    $isComplete = $operation->isComplete();
}

$result = $operation->results()[0];

Let me know if it helps.

Thanks David I’ll try your alternative solution since the audio file is around 30 minutes so it takes a while.
De : David Supplee<EMAIL_ADDRESS>Envoyé : vendredi 29 juin 2018 21:06
À : GoogleCloudPlatform/google-cloud-php<EMAIL_ADDRESS>Cc : PeteKirkham<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Objet : Re: [GoogleCloudPlatform/google-cloud-php] Cloud Speech sample (#1142)
Pete,
Is this https://github.com/GoogleCloudPlatform/php-docs-samples/blob/master/speech/src/transcribe_async_gcs.php  the sample you have been working with? You should be able to increase the number of retries passed into the ExponentialBackoff instance from 10 to whatever seems reasonable for your audio length. I would recommend doubling the value up to 20 and see if you need to adjust from there. :)
Alternatively, you could poll indefinitely until the operation is complete:
$isComplete = $operation->isComplete();
while (!$isComplete) {
sleep(3); // let's wait for a moment...
$operation->reload();
$isComplete = $operation->isComplete();
}
$result = $operation->results()[0];
Let me know if it helps.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/google-cloud-php/issues/1142#issuecomment-401446072 , or mute the thread https://github.com/notifications/unsubscribe-auth/AEwitHlc4t0kzRSXVsUW6JODxKezteGlks5uBnp6gaJpZM4U8uAu .  https://github.com/notifications/beacon/AEwitIQVAtmMxIQ4QyMsA-tKhiRmRoEDks5uBnp6gaJpZM4U8uAu.gif

I'm going to close this out now, please feel free to re-open if there is anything else you'd like to touch on!


container PUTs in access log delivery

currently, container PUTs are not delivered as part of access log delivery. can this be added or not?

Generally no, although in some cases they should show up. Since log delivery is set on the container level, the container creation will only be captured if log delivery is turned on in the same hour that the container is created. Normally, log delivery is turned on much after the container is created, so the original container PUT is "lost" in the old logs.


Blender: Added setting for base unit scale

Changelog Description
A setting for the base unit scale has been added for Blender.
The unit scale is automatically applied when opening a file or creating a new one.
Testing notes:

Set a default unit scale in Studio Settings: project_settings/blender/base_file_unit_scale.
Open any task with Blender.
Check if the unit scale has been applied:

I think this doesn't handle loading files from various sources - it wil set the scale for everything. Lets say you have alembics coming from Maya and alembics coming from Blender, wouldn't this break one or the other?
We don't store the information about units on representation for now so it involves a lot of guessing. But the way to mitigate it would be by setting this scale per source DCC.
When you load something with loader, you know the source of that representation and from that you can at least determine the application that was used to produce this representation. You can't know what was the unit setting there but at least you can limit it per imported data?
What I am trying to say is, that we should maybe leave the scene settings alone, and only change the scale of the loaded data (if this option is enabled). This will not influence any existing data in the scene, and it actually mimics the actual workflow of artists having to deal with stuff coming from different DCCs. What do you think?

What I am trying to say is, that we should maybe leave the scene settings alone, and only change the scale of the loaded data (if this option is enabled). This will not influence any existing data in the scene, and it actually mimics the actual workflow of artists having to deal with stuff coming from different DCCs. What do you think? @simonebarbieri

Even though that's valid suggestion, why don't we then handle it in all places and all DCCs. I'd argue that it's a bit beyond the current scope and goes completely against the fact, that loader should care at all where the data is coming from. Better place would be ensuring correct scales across the DCCs in a cross host production, which is really down to the studio and we should just provide the tools to allow it as much as possible. This is a simple setting that is reasonable to have, so I think we should merge this to allow easier unit management at least across full blender production and deal with various scales in a different PR


Code

Hello Sir,
In the README file says you can share the code and the smart trained model, I send you and email to request it but I didn't receive an answer. Is there anyway for you to share what you say on the readme file?.
Thanks.

Hello @fhuaiquimilla
I mentioned that I would be sending the pre-trained model and the code needed to run it, not the development code itself.
Therefore, I apologize for sending the code, but I can provide you with full assistance in order to develop a similar system on your own.
Thank you

Hello dear,
I am working with project similars to yours, so kindly could you share the pre-trained model with me
Thanks in advance,


Catch Seeds that are Inconsistent With Requested Settings

Fixes to prevent issues with playing seeds that don't match the requested settings like in Pink's stream.

After generating a seed, it will now throw an error popup if it doesn't match the user settings
Spazer and tunics no longer show up as potential early items as they are not technically progression items

@Vivelin A lot of what currently detects unwinnable seeds is pretty far down in the original SMZ3 generation code and use exceptions for it, so some revamp work will be needed to better trickle that data up to allow for continuing on. For now I at least made the detection of user settings not being picked up a warning screen.


fix name collision detection

property called ref in Swagger => property Ref in C# was checked against reserved keywords => no collision => constructor parameter called ref

🤖 AutoRest automatic publish job 🤖
> will publish once PR gets merged


Revert sg refactor plus additional features

what

Revert sg refactor plus additional features
https://github.com/cloudposse/terraform-aws-ecs-alb-service-task/pull/125
https://github.com/cloudposse/terraform-aws-ecs-alb-service-task/pull/129
https://github.com/cloudposse/terraform-aws-ecs-alb-service-task/pull/130

why

Undo breaking changes made prematurely
Unfreeze the code base
Merge some of the outstanding PRs in

references
N/A

/test all

/test all


Testing Issues Page

This is a test of our issues page.
Issues encountered:

We need to test issues.
Issues are untested.

@AvaAvarai Do you know about this?

What does this look like?

Can I necro?

Can this non-associated account also necro?


Telerik credentials are in source control

I was just following the readme file and noticed that there are credentials hard-coded into FrontDesk/src/FrontDesk.Blazor/nuget.config
I'm guessing this was not intentional - these should probably be deleted.

This issue is actually in the ddd-vet-sample repo, not this one. Feel free to close.


Include directives may be nil if no includes are directed

If you attempt to filter an included resource, but didn't include anything, it will crash.
If you had included a different resource this would return an error response. With this change, we safe-navigate the @include_directives, avoiding a NoMethodError on nil.

@hatch-carl Thanks!


Improvement: can you separate and isolate loading from registering?

The biggest limitation of ocLazyLoad is it's built-in loading mechanics. Even though we could use plain async loader, we can't really benefit from AMD-like loaders.
We define modules like:
// app.js:
myDefine(['services'], function () {
    return angular.module('app', ['services']);
});

// services.js:
myDefine([], function () {
    return angula.module('services', []);
});

Then dependency resolution kicks-in and does it's job. But we can't really stitch it with ocLazyLoad.
Can I suggest something like:
<script src="ocLazyLoad.js"></script>
<script src="ocLazyLoad.register.js"></script>
<script src="ocLazyLoad.load.module.js"></script>
<script src="ocLazyLoad.load.script.js"></script>
<script src="ocLazyLoad.load.css.js"></script>
<script src="ocLazyLoad.directive.js"></script>

Or even aggregating load.*.js into single load.js. The point is, module registration is quite complex on it's own, and if we could really separate it's API from everything else — the application spectre will grow exponentially.
Any thoughts?

Your request is kinda similar to this: https://github.com/ocombe/ocLazyLoad/issues/115
I will implement separate loaders as plugins so that users can use the one they want without impacting the file size, and it will also make the development of new loaders easier.
But why would you want to externalize the register / module / directive parts? I don't get what the benefit would be?

@ocombe oh, thx. No, I'm not particularly after externalizing those.. Just if we're saving filesize and organizing our dependencies, we could take out as much as possible.
Here I've implemented only register and invokeQueue functions (look at app/loader/module.js file) and it works well: http://cl.ly/0w041y3e3P0n (sorry, plunker doesn't work with my AMD loader) make code clean and without unused lines of code.

It's true that if I externalize those parts, anyone could just use this for its own plugins, that could be useful.

This is on the roadmap for 1.0 https://github.com/ocombe/ocLazyLoad/issues/139


CA certificate without Basic Constraints

Hello.
I’m not sure if this is more documentation/setup related rather than code/functional and the correct place to ask but I won’t be the only one facing this.
Recently after updating chromium some websites are rejecting my Certs. Eg Wikipedia.
Firefox does the same now too.
Sites like google accept my cert and allow sslsplit as normal.
Reading the fault in the ‘developer tools -> security issues’ it’s that the cert issuer isn’t recognised.
Obviously it’s not. It’s self cert. and I’m not that famous.
Am I missing something on setup to get round this?
Options ‘-O -P’ allow the sites to be bypassed allowing seemless access but it avoids the point of SSLSplit
Has anyone else had this problem and if so how did you overcome it?
Thanks.

iPhone safari works and allows access.
Silk Browser on Kindle (I believe Android based) works.
Seems to be PC causing the issues. Must be more rigorous key chain checking.

Good morning,
I have been looking further into this issue. While Safari works on an iDevice. Youtube app does not. This routes via the common google ip and (I'm guessing) resolves and returns the url as needs require. Odd as google works in safari. Youtube app IP from wireshark <IP_ADDRESS>
Reading the output from SSLSplit youtube app is failing due to 'Unclean SSL shutdown'.
When I visit Wikipedia.org in Chromium the site fail on security issues. Looking at the Developer Tools -> Security -> Certificate Info in Chromium it appears the 'basic constrains' of the forwarded certificate aren't listed as a Certificate Authority. Could this be causing the issue?
Does this mean crypto pairs are going to need to be generated by an established CA with Private Keys capable of deriving a certificate already recognised?
Thoughts?
Thank you,

Can you post the output of openssl x509 -in cert.pem -text on the CA certificate that you are using with SSLsplit and how you generated it? If applicable, which documentation did you follow to generate the certificate?

I made a fresh one for this post. Checked and the error continues.
Certificate:
Data:
Version: 1 (0x0)
Serial Number:
d7:f1:af:8a:80:10:d3:d0
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=UK, ST=Midlands, L=Warwickshire, O=SSLTest, CN=www.SSLTest.com
Validity
Not Before: Mar 28 11:41:00 2018 GMT
Not After : Mar 25 11:41:00 2028 GMT
Subject: C=UK, ST=Midlands, L=Warwickshire, O=SSLTest, CN=www.SSLTest.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:b9:9b:76:6b:f9:07:d1:06:3a:ca:88:2d:d4:79:
ec:ce:0d:6a:9d:53:41:99:60:e2:d0:cb:b1:84:fe:
29:f4:d9:cc:54:9a:8a:ee:9e:37:1f:32:40:d0:de:
ea:77:a5:cf:e5:51:f7:16:84:48:16:69:63:2d:55:
c7:df:df:7c:5d:9d:fb:9f:c9:1f:41:bf:4e:fc:80:
3b:8e:c6:17:a9:9b:77:c7:12:95:bd:d3:6f:f7:66:
c3:af:65:49:1b:2b:09:0d:7b:cb:75:51:a0:19:b9:
77:76:80:6f:46:2f:44:72:ae:5a:49:b3:64:0e:60:
fd:65:21:c7:37:d5:d9:b4:55:70:8f:c7:78:e7:c1:
c7:95:21:36:6c:78:10:7c:64:d4:c7:31:ad:e0:44:
ff:c7:80:0f:91:be:66:14:45:60:9d:62:06:6a:f1:
ac:c7:0b:41:ce:8c:6b:17:17:d9:db:32:a6:9e:b7:
c4:ce:5d:51:19:8d:48:31:a9:03:9f:ee:cc:17:60:
9c:68:df:d0:ea:e7:05:80:93:bc:2e:53:7b:ac:32:
9c:57:c9:39:56:42:d9:5a:4c:b5:52:c6:29:6e:85:
3a:51:77:24:e2:7d:3a:ad:45:71:0b:54:46:ae:8b:
83:77:46:45:ff:31:82:1d:ef:51:1e:69:8e:b1:96:
fe:15
Exponent: 65537 (0x10001)
Signature Algorithm: sha256WithRSAEncryption
13:42:95:5f:74:78:02:34:88:3a:28:8b:61:50:4f:36:ed:aa:
30:eb:8c:c1:4c:28:ee:74:58:44:75:c6:22:f3:29:27:aa:5b:
dd:2c:dc:7d:7a:ec:39:aa:18:3e:eb:a4:e0:c5:a7:e9:14:cc:
2f:bb:76:df:c9:00:a1:8a:7d:82:0b:5b:a3:90:85:60:8a:d0:
c9:b9:86:11:aa:f3:02:80:35:48:82:25:b7:8a:cb:25:ca:1c:
84:78:03:7d:2d:19:32:a2:71:07:37:da:2d:e2:20:b2:7a:0e:
55:19:1f:0d:8a:87:a8:d6:9b:ca:89:df:d7:2c:92:b7:e4:38:
3e:07:81:02:0d:b1:2e:0d:48:c2:ce:35:24:7f:67:68:17:33:
ad:e6:7f:b4:ee:f5:9a:20:61:46:11:1a:fe:7f:5f:f7:9d:ad:
ee:5a:0a:89:9e:2d:ed:de:16:88:0e:72:46:6a:74:81:29:cc:
b7:cf:16:c4:5a:30:9a:32:a1:1b:35:08:99:c0:ce:29:19:1c:
63:30:e8:d0:1f:3b:7a:6c:b5:05:7b:67:de:c0:4a:da:3e:be:
1e:59:5e:ad:a3:14:21:43:66:26:de:61:d8:2e:a0:4d:48:a5:
2a:4f:76:25:1d:9d:31:89:ea:5e:db:d0:24:20:f6:55:9d:1c:
eb:4a:df:b0
-----BEGIN CERTIFICATE-----
MIIDQjCCAioCCQDX8a+KgBDT0DANBgkqhkiG9w0BAQsFADBjMQswCQYDVQQGEwJV
SzERMA8GA1UECAwITWlkbGFuZHMxFTATBgNVBAcMDFdhcndpY2tzaGlyZTEQMA4G
A1UECgwHU1NMVGVzdDEYMBYGA1UEAwwPd3d3LlNTTFRlc3QuY29tMB4XDTE4MDMy
ODExNDEwMFoXDTI4MDMyNTExNDEwMFowYzELMAkGA1UEBhMCVUsxETAPBgNVBAgM
CE1pZGxhbmRzMRUwEwYDVQQHDAxXYXJ3aWNrc2hpcmUxEDAOBgNVBAoMB1NTTFRl
c3QxGDAWBgNVBAMMD3d3dy5TU0xUZXN0LmNvbTCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBALmbdmv5B9EGOsqILdR57M4Nap1TQZlg4tDLsYT+KfTZzFSa
iu6eNx8yQNDe6nelz+VR9xaESBZpYy1Vx9/ffF2d+5/JH0G/TvyAO47GF6mbd8cS
lb3Tb/dmw69lSRsrCQ17y3VRoBm5d3aAb0YvRHKuWkmzZA5g/WUhxzfV2bRVcI/H
eOfBx5UhNmx4EHxk1McxreBE/8eAD5G+ZhRFYJ1iBmrxrMcLQc6MaxcX2dsypp63
xM5dURmNSDGpA5/uzBdgnGjf0OrnBYCTvC5Te6wynFfJOVZC2VpMtVLGKW6FOlF3
JOJ9Oq1FcQtURq6Lg3dGRf8xgh3vUR5pjrGW/hUCAwEAATANBgkqhkiG9w0BAQsF
AAOCAQEAE0KVX3R4AjSIOiiLYVBPNu2qMOuMwUwo7nRYRHXGIvMpJ6pb3SzcfXrs
OaoYPuuk4MWn6RTML7t238kAoYp9ggtbo5CFYIrQybmGEarzAoA1SIIlt4rLJcoc
hHgDfS0ZMqJxBzfaLeIgsnoOVRkfDYqHqNabyonf1yySt+Q4PgeBAg2xLg1Iws41
JH9naBczreZ/tO71miBhRhEa/n9f952t7loKiZ4t7d4WiA5yRmp0gSnMt88WxFow
mjKhGzUImcDOKRkcYzDo0B87emy1BXtn3sBK2j6+HlleraMUIUNmJt5h2C6gTUil
Kk92JR2dMYnqXtvQJCD2VZ0c60rfsA==
-----END CERTIFICATE-----
#########################################################
The above was generated with the following script:
#!/bin/sh
if [[ $1 == --self ]]; then
SELF_SIGN=1
shift
fi
KEY_NAME=$1
CSR_CONFIG=$2
openssl req -config $CSR_CONFIG -new -newkey rsa:2048 -nodes -keyout /${KEY_NAME}.key -out /${KEY_NAME}.csr
#echo "Created ${KEY_NAME}.key"
#echo "Created ${KEY_NAME}.csr"
if [[ -n $SELF_SIGN ]]; then
openssl x509 -req -days 3650 -in /${KEY_NAME}.csr -signkey /${KEY_NAME}.key -out /${KEY_NAME}.crt
echo "Created ${KEY_NAME}.crt (self-signed)"
fi
##########################################################################
The template:
[ req ]
distinguished_name="SSLTest"
prompt="no"
[ SSLTest ]
C="UK"
ST="Midlands"
L="Warwickshire"
O="SSLTest"
CN="www.SSLTest.com"
#############################################################
Thank you,

I don't know if the German flag is showing on your browser but when it states ": de :" without the white space it shows a German flag. :de:

Hello,
Last piece of info. Thought it would send the output and the version I'm using:
SNI peek: [en.wikipedia.org] [complete]
Attempt reuse dst SSL session
Connecting to [<IP_ADDRESS>]:443
===> Original server certificate:
Subject DN: /C=US/ST=California/L=San Francisco/O=Wikimedia Foundation, Inc./CN=.wikipedia.org
Common Names: .wikipedia.org/.wikipedia.org/wikipedia.org/.m.wikipedia.org/.zero.wikipedia.org/wikimedia.org/.wikimedia.org/.m.wikimedia.org/.planet.wikimedia.org/mediawiki.org/.mediawiki.org/.m.mediawiki.org/wikibooks.org/.wikibooks.org/.m.wikibooks.org/wikidata.org/.wikidata.org/.m.wikidata.org/wikinews.org/.wikinews.org/.m.wikinews.org/wikiquote.org/.wikiquote.org/.m.wikiquote.org/wikisource.org/.wikisource.org/.m.wikisource.org/wikiversity.org/.wikiversity.org/.m.wikiversity.org/wikivoyage.org/.wikivoyage.org/.m.wikivoyage.org/wiktionary.org/.wiktionary.org/.m.wiktionary.org/wikimediafoundation.org/.wikimediafoundation.org/.m.wikimediafoundation.org/wmfusercontent.org/.wmfusercontent.org/w.wiki
Fingerprint: 0F:FB:95:52:F3:B1:3E:CF:AB:6E82:8C:60:88:A2:0F:D0:04:4E:4E
Certificate cache: HIT
===> Forged server certificate:
Subject DN: /C=US/ST=California/L=San Francisco/O=Wikimedia Foundation, Inc./CN=.wikipedia.org
Common Names: .wikipedia.org/.wikipedia.org/wikipedia.org/.m.wikipedia.org/.zero.wikipedia.org/wikimedia.org/.wikimedia.org/.m.wikimedia.org/.planet.wikimedia.org/mediawiki.org/.mediawiki.org/.m.mediawiki.org/wikibooks.org/.wikibooks.org/.m.wikibooks.org/wikidata.org/.wikidata.org/.m.wikidata.org/wikinews.org/.wikinews.org/.m.wikinews.org/wikiquote.org/.wikiquote.org/.m.wikiquote.org/wikisource.org/.wikisource.org/.m.wikisource.org/wikiversity.org/.wikiversity.org/.m.wikiversity.org/wikivoyage.org/.wikivoyage.org/.m.wikivoyage.org/wiktionary.org/.wiktionary.org/.m.wiktionary.org/wikimediafoundation.org/.wikimediafoundation.org/.m.wikimediafoundation.org/wmfusercontent.org/.wmfusercontent.org/w.wiki
Fingerprint: BE:3B:52:A0:EC:3D:00:55:FF:0E98:53:BE:D2:AE:11:32:93:96:85
SSL connected to [<IP_ADDRESS>]:443 TLSv1.2 ECDHE-ECDSA-AES256-GCM-SHA384
CLIENT_RANDOM 375E8FF2A2EAFAE87E32C8C25F7EAB31753755E15D734421F1C906A78C1DD92B D9DC6B8244B6A63B415DB5FAAF4E38008FF4B15BD84F5600B4855BA4E36F2CDEF3D0D3B28C25E20947E9CC3C42D499BD
Certificate cache: KEEP (SNI match or target mode)
ssl <IP_ADDRESS> 54486 <IP_ADDRESS> 443 sni:en.wikipedia.org names:.wikipedia.org/.wikipedia.org/wikipedia.org/.m.wikipedia.org/.zero.wikipedia.org/wikimedia.org/.wikimedia.org/.m.wikimedia.org/.planet.wikimedia.org/mediawiki.org/.mediawiki.org/.m.mediawiki.org/wikibooks.org/.wikibooks.org/.m.wikibooks.org/wikidata.org/.wikidata.org/.m.wikidata.org/wikinews.org/.wikinews.org/.m.wikinews.org/wikiquote.org/.wikiquote.org/.m.wikiquote.org/wikisource.org/.wikisource.org/.m.wikisource.org/wikiversity.org/.wikiversity.org/.m.wikiversity.org/wikivoyage.org/.wikivoyage.org/.m.wikivoyage.org/wiktionary.org/.wiktionary.org/.m.wiktionary.org/wikimediafoundation.org/.wikimediafoundation.org/.m.wikimediafoundation.org/wmfusercontent.org/.wmfusercontent.org/w.wiki sproto:TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256 dproto:TLSv1.2:ECDHE-ECDSA-AES256-GCM-SHA384 origcrt:0FFB9552F3B13ECFAB6E828C6088A20FD0044E4E usedcrt:BE3B52A0EC3D0055FF0E9853BED2AE1132939685
SSL connected from [<IP_ADDRESS>]:54486 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256
CLIENT_RANDOM 19368F9DBFC3C8763ED6218D896DEDFFED96BE5ADD0CD3DB1006E8CF0E44390C D1D7FE328304DBD80B55D4D85906AACC0F512087159E5CEB2D91F51C9A342132CF4837F8B64FBBA95EDFC9A5B64E7572
Unclean SSL shutdown.
SSL disconnected to [<IP_ADDRESS>]:443
SSL disconnected from [<IP_ADDRESS>]:54486
SSL_free() in state 00000003 = 0003 = SSLOK  (SSL negotiation finished successfully) [accept socket]
Failed to shutdown SSL connection cleanly: Max retries reached. Closing fd.
SSL_free() in state 00000003 = 0003 = SSLOK  (SSL negotiation finished successfully) [connect socket]
p.s. My version info:
SSLsplit 0.5.2 (built 2018-03-19)
Copyright (c) 2009-2018, Daniel Roethlisberger<EMAIL_ADDRESS>https://www.roe.ch/SSLsplit
Build info: V:FILE
Features: -DHAVE_NETFILTER
NAT engines: netfilter* tproxy
netfilter: IP_TRANSPARENT IP6T_SO_ORIGINAL_DST
Local process info support: no
compiled against OpenSSL 1.0.2l  25 May 2017 (100020cf)
rtlinked against OpenSSL 1.0.2l  25 May 2017 (100020cf)
OpenSSL has support for TLS extensions
TLS Server Name Indication (SNI) supported
OpenSSL is thread-safe with THREADID
Using SSL_MODE_RELEASE_BUFFERS
SSL/TLS protocol availability: tls10 tls11 tls12
SSL/TLS algorithm availability: !SHA0 RSA DSA ECDSA DH ECDH EC
OpenSSL option availability: SSL_OP_NO_COMPRESSION SSL_OP_NO_TICKET SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION SSL_OP_TLS_ROLLBACK_BUG
compiled against libevent 2.0.22-stable
rtlinked against libevent 2.0.22-stable
1 CPU cores detected

Okay, you want to have basic constraints in your CA certificate. Check out the openssl commands in the make file and the config file that goes with them in https://github.com/droe/sslsplit/blob/develop/extra/pki/ specifically basicConstraints = CA:TRUE
Having said that, sslsplit should warn you that you are using a CA certificate without basic constraints. I'm flagging this issue as a feature request for better sanity checking and error reporting on the provided certificate.

Evening Daniel,
I have used the makefile and made the crypto pairs. Using the RSA. cyrpto pair I am still getting the same issue.
I thought you may appriciate the below. The cert's are generated as normal with Google

But when you try wikipedia it faults.

The criticallity of the certificate being a Certification Authority seems to depend on the website in question.
I have tried the other crypto pairs generated, some don't install as they're not the right type, naturally, but the RSA pair does work, on google and github but not wikipedia.
Apologies for persisting but it's very frustrating as I use this software often.
Thank you,

Any thoughts? Ive been working on a ‘work-around’ to force passthrough on SNI matching so I can continue to use without issues. (Whitelisting in effect). ‘-P’ has stopped working too on the sites I mentioned (YouTube, Wikipedia).

I tried but I cannot reproduce this problem. I can intercept traffic to Wikipedia just fine with Chrome on macOS.

Would it be worth me issuing you a copy of the certs my OpenSSL produces? I used your GNUmakefile and used the rsa cert and key.
I have used the most up to date OpenSSL. What version are you using?

I'm using OpenSSL 1.0.2o. Perhaps a pcap of both the network traffic before and after sslsplit could help narrowing down what's happening.

Tcpdump suffice?

tcpdump -w, sure

You can mail it to me instead of posting here if you prefer

Can you retest with lastest develop after the commit referenced in #189 ?

Good afternoon Daniel.
I tried the latest develop last night.
All the raw code, only change was adding the 'CFLAG -lfts' to the GNUmakefile.
I am using a program called tinyproxy to upstream any connections to SSLSPLIT that are intended for google.com as I know this works with SSLSPLIT.
When the connection is received by SSLSPLIT it returns
"Peeking did not yield a (truncated) ClientHello message, aborting connection"
By my reckoning the connection is being forwarded too late after the handshake has commenced, as the first proxy needs to determine the destination and isn't issuing the full details to SSLSPLIT.
I presume sslsplit works on a much lower level of the network where iptables/netfilter sits and the proxy I'm using is too high level.
This leaves me to two questions.

Can SSLSPLIT try retrying a connection to reattain the connection details from those passed to it?

Can we put a string comparison near the SNI PEEK code to confirm the SNI is google who whatever. Set a ctx variable to confirm this connection will be split. Otherwise passthrough. I have tried doing the opposite of this and passing-through anything with YouTube in the SNI but it fails on the passthrough due to unsafe memory.
I set the bypass variable just after the code where the sni peek is printed to debug. And adding the bypass variable to the 3 if statements at "/* wrap client-side socket in a buffer event*/" line 1874 of pxyconn.c

Before you ask I set he bypass variable to 0 on new ctx and disposed of it with a free() at the destroy ctx.
Thank you.
Joe
Sent from my iPhone

On 12 Apr 2018, at 18:37, Daniel Roethlisberger<EMAIL_ADDRESS>wrote:
Can you retest with lastest develop after the commit referenced in #189 ?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

"Peeking did not yield a (truncated) ClientHello message, aborting connection" means that sslsplit was unable to parse the SSL/TLS ClientHello message. If this is happening, capture the respective SSL/TLS connection in a PCAP, make sure that it is actually a ClientHello, and then open a separate issue on that.
Until I have a reproducible test case I cannot do much, I'm afraid.
Why are you adding libfts if you did not change any code?

As an afterthought, please provide details on the platform and build environment. Is this some weird libc without fts(3), then I suspect your platform may be part of the problem.

Hello Daniel,
Apologies for the long delay.
I have been trying to figure out what has been going on.
My platform is OpenWRT a POSIX Linux OS for small embedded systems.
I have found something interesting.
Google.co.uk works find,
Google.com does not.
Google.com reports the same error as youtube.
I have attached two pcap files, one for google.com the other for
google.co.uk
From what I can see the browser is listing the connection as insecure due
to the transmitions being received out of order? or being retransmitted?
Please let me know when you have chance to look into this?
Thank you,
Joe
On Sun, Apr 15, 2018 at 6:49 PM, Daniel Roethlisberger <
<EMAIL_ADDRESS>wrote:

As an afterthought, please provide details on the platform and build
environment. Is this some weird libc without fts(3), then I suspect your
platform may be part of the problem.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/droe/sslsplit/issues/188#issuecomment-381424677, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABRexftM4moByntBiiZuxQGdjeQwad1wks5to4hFgaJpZM4Suxr5
.

Joe, can you test drive branch issue/195 for me to see if the change fixes the issue you are having? It fixes the keyUsage field to be in line with the actual leaf key type instead of copying it from the upstream certificate.

Morning Daniel,
I will have a look this evening.
Cheers,
Joe
On Sun, 15 Jul 2018 at 01:02, Daniel Roethlisberger <
<EMAIL_ADDRESS>wrote:

Joe, can you test drive branch issue/195
https://github.com/droe/sslsplit/tree/issue/195 for me to see if the
change fixes the issue you are having? It fixes the keyUsage field to be in
line with the actual leaf key type instead of copying it from the upstream
certificate.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/droe/sslsplit/issues/188#issuecomment-405057468, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABRexXdk03vy58Nk74ubi0iVi2RT2jm9ks5uGoawgaJpZM4Suxr5
.

If you start chromium-browser from a terminal, you will see the numerical NSS error code in error output that more specifically narrows down why Chromium raises NET::ERR_CERT_INVALID. If it's -8102 (SEC_ERROR_INADEQUATE_KEY_USAGE), that would be a strong indication that the root cause is the same as for isse #195. If it is something else, that number helps further investigation.

Good evening Daniel,
I compiled and tested and its still reporting the same issue on the YouTube
App, output attached.
Google.com doesn't work either,
I'm not in a position to close chrome down to start via CLI.
Thank you for your efforts,
Joe
On Sun, 15 Jul 2018 at 15:59, Daniel Roethlisberger <
<EMAIL_ADDRESS>wrote:

If you start chromium-browser from a terminal, you will see the numerical
NSS error code in error output that more specifically narrows down why
Chromium raises NET::ERR_CERT_INVALID. If it's -8102 (
SEC_ERROR_INADEQUATE_KEY_USAGE), that would be a strong indication that
the root cause is the same as for isse #195
https://github.com/droe/sslsplit/issues/195. If it is something else,
that number helps further investigation.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/droe/sslsplit/issues/188#issuecomment-405097595, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABRexd35Az5_GeGUUtsRzZ4SlSXEdiDeks5uG1jkgaJpZM4Suxr5
.

Okay, please let me know the NSS error code when/if you have the opportunity to run Chrome or Chromium from a terminal.

Hi Daniel,
I am trying to debug a bunch of Chrome cert errors as I am using sslsplit; running either Chrome or Chromium from osx terminal does not show the NSS error code. Even dev tools do not show any error codes.
Regards!
PS: thank u very much for an awesome tool, so much to learn ...

@aboulfad I have just pushed an LD_PRELOAD overlay for NSS certificate validation failure debugging into extra/nssdebug/. You can use that to get an output into your terminal indicating the exact NSS error number that causes certificate validation to fail. But please open a new issue for your problem instead of hijacking this one.

Hey, could you try a certificate with an expiration date of less than 39 months?
Google with Android 7.0 has been throwing certificate errors in it's applications when the root certificate has an validity of more than 39 months.
I haven't checked in the desktop browser however, but know for sure this was one of the issues I ran into with Android.

I would expect the 39 months limit to only be effective for issued certificates, not CA certificates. We are not generating any leaf certificates with a lifetime longer than a year, so we should not run into this issue. Nevertheless, I've reduced the validity period of the default CA certificate in extra/pki to 3 years.
Reference: https://cabforum.org/2017/03/17/ballot-193-825-day-certificate-lifetimes/

Hey @droe I managed to find my source, I should have looked for this when I made my comment.
Original article I read:
https://blog.nviso.be/2018/01/31/using-a-custom-root-ca-with-burp-for-inspecting-android-n-traffic/
Links to source:
https://bugs.chromium.org/p/chromium/issues/detail?id=475745
It does indeed seem to be the leaf certificates that are mentioned, not the CA, apologies.


Allow for complete model reloading

Background
At present, there are some variables (e.g. ml_f_EM) that are not returned by run_sustain_algorithm but are saved in the pickle files. It doesn't make sense to me that running the model gives you different outputs from loading the pickle file that you get from running it. The run_sustain_algorithm should just output what gets saved (and should probably output it as a dict to avoid having to refer to the order in which things are output).
Further to this, when trying to load previous results from a pickle file, a lot of the same setup is still required as when running the model in the first place, which can result in a lot of unnecessary boilerplate. There is some code that indicates there was once a desire for this. If anybody knows why this wasn't pursued or if there is some obstacle I haven't noticed please let me know.
Solutions
1. Expand what's pickled and reconstruct model instance
A @classmethod to recreate the model instance from a pickle file, resulting in the same thing as if you were to run the method (with Z_vals etc. bundled in), would simplify the average workflow significantly, and would be a fairly simple change. The main issue with this is that it would probably break people's existing code, and would require the notebook(s) to be updated. It would, however, keep to the current concept (pickle the results/arrays).
2. Modify how pySuStaIn pickles and save/load model in its entirety
We could also just pickle/unpickle the model instance itself, following a few changes to enable this. The process should also fix #27 and #41, on top of addressing the above (and simplifying things a lot). I reckon it should be doable without too much bother. Further details/considerations can be found here. This would also be best-served by turning the arrays that are usually pickled into attributes, and so would lead to a lot of small changes through the code.
Practicalities

Either approach will very likely break previous code (if people update, that is!). So, this may want to form part of a proper, versioned release.
I am happy to work on this, but it will be at weekends etc. (though shouldn't take long).

If core maintainers agree, I'll go ahead with it, but if not then I will leave it.

It's a great idea, particularly the second part. Thanks a lot for offering to code it up, think it would be worth discussing before commiting too much time to it. Agree that it would be best if it forms part of a versioned update, together with new notebooks (we should also start archiving those so people can find examples for the version they're using). Let's try to set up the quarterly pySuStaIn developers meeting we talked about ages ago and discuss it there - I'll send round an email.

Sounds like a plan, I'll keep an eye out for your email!


Correct environment spelling in bug report template

Description
Environment spelling is wrong in bug report template so I corrected it.
Notes for Reviewers
I don't consider creating an issue for this required, so I made PR directly.
Signed commits

[X] Yes, I signed my commits.

@amitamrutiya Thanks for your contribution, let's discuss this on the website's call. Please add this as an agenda item to the meeting minutes. The call is scheduled for Monday at 5:30 PM IST.


Projected Views

Description
A new kind of resource that abstracts indices from searches.  A view define a projection of targeted indices.  I'm trying to start out with as barebones of an implementation to get design feedback and find a good matching vertical scenario that could be added to dashboards.
Related Issues

Resolves https://github.com/opensearch-project/security/issues/3888
RFC https://github.com/opensearch-project/OpenSearch/issues/6181
Based off https://github.com/peternied/OpenSearch-1/pull/139

High Level Design
Views are stored in cluster metadata, standard CRUD operations.  A view can be used for _search which validates and transforms the request onto the 'physical' structure of the data like indices.  See more in the views-design.md [link] included in this PR.
sequenceDiagram
    participant Client
    participant HTTP_Request as ActionHandler
    participant Cluster_Metadata as Cluster Metadata Store
    participant Data_Store as Indices

    Client->>HTTP_Request: View List/Get/Update/Create/Delete
/views or /views/{view_id}
    HTTP_Request->>Cluster_Metadata: Query Views
    alt Update/Create/Delete
        Cluster_Metadata->>Cluster_Metadata: Refresh Cluster
    end
    Cluster_Metadata-->>HTTP_Request: Return
    HTTP_Request-->>Client: Return

    Client->>HTTP_Request: Search View
/views/{view_id}/search
    HTTP_Request->>Cluster_Metadata: Query Views
    Cluster_Metadata-->>HTTP_Request: Return
    HTTP_Request->>HTTP_Request: Rewrite Search Request
    HTTP_Request->>HTTP_Request: Validate Search Request
    HTTP_Request->>Data_Store: Search indices
    Data_Store-->>HTTP_Request: Return
    HTTP_Request-->>Client: Return

Check List

[x] New functionality includes testing.

[x] All tests pass

[X] New functionality has been documented.

[X] New functionality has javadoc added

[X] Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
[X] Commits are signed per the DCO using --signoff
[X] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
[X] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@cwperks @DarshitChanpura could you take another look?  I've snapped to the patterns associated with the OpenSearch repo and created integration tests.  While I still need to add some additional transport actions and associated tests (Get/Deleted/Update/Enumerate) they will follow the same conventions established with CreateViewAction and the ViewService.

Docs PR:

https://github.com/opensearch-project/documentation-website/issues/6363

@reta @msfroh @mch2 Mind taking a look?

@reta @msfroh @mch2 Mind taking a look?

@peternied I should be able to look tomorrow, if that works for you, thanks!

Thanks for this feature @peternied , I think codewise it looks good (to me) but there are few concerns that I would like to share:

conceptually, at least at this stage, I see not much benefits of using views vs doing search with index pattern like  GET /sales*/_search , but probably view would evolve?
more importantly, storing views in a cluster state could potentially significantly increase the size of the cluster state: we probably have to think about:
a) should we impose any limits on the number of views cluster may have?
b) should we impose any limits on view's name length?
c) the index pattern is basically is the dynamic view, do we want to support the static views, where the index names are preset?

@reta Thanks - great questions as always, here are some responses:

probably view would evolve?

I've created https://github.com/opensearch-project/OpenSearch/issues/12322 as an issue to track what direction the feature could go in.  I'm trying to start out with as barebones of an implementation to get design feedback and find a good matching vertical scenario that could be added to dashboards.

more importantly, storing views in a cluster state could potentially significantly increase the size

I agree - cluster stability is really important and I do not think that views drastically impacts this problem. I couldn't find an issue about how this is managed or needs to be managed other than [1] - do you know of one?

[1] https://github.com/opensearch-project/OpenSearch/issues/11527

Think that Views should not be backported to any 2.x release until it is no longer experimental - for this basic implementation this seems like feature creep.  If you give me parameters like <100 views, each view <25 targets I could adding that but I'd rather spend cycles on interesting use cases so that I have a problem that views are going to be used for everything.  Does that seem like a sensible to move forward?

Think that Views should not be backported to any 2.x release until it is no longer experimental - for this basic implementation this seems like feature creep.

Thanks @peternied , I believe this comment largely alleviates my concerns (since we'll have sufficient time to iterate over views and make the appropriate decision later on).

@peternied LGTM a few minor comments but looks good!

@peternied - Can you share more details on how we are planning to use the views differently than aliases / index-pattern today before we make View first class citizen of Opensearch?

@jainankitk Thanks for looking into this PR and the related RFC

before we make View first class citizen

I'm not sure views will become a first class citizen.  This feature is experimental until we can get more solid scenarios behind it.  I'm trying to organically build up use cases and clear value before shipping this as part of the product.  Even this 'basic' implementation is 3k lines of code, I like to start here to make incremental process.
In this pull request https://github.com/opensearch-project/OpenSearch/pull/11957#issuecomment-1944538324 also has more context around some of these thoughts

how we are [...] views different than aliases / index-pattern

Index patterns don't exist in the backend, they are a dashboards only concept.  I think that views can become the representation of the Dashboards index pattern object which includes metadata for the field to sort time series data on and derived fields and transformations.
Alias are directly attached to concrete indexes.  Views don't have those limitations.  While there is some existing filtering logic into alias, I'd rather not muddle what an index is with a view - as both of these can live side by side associated with different use cases.

This feature is experimental until we can get more solid scenarios behind it. I'm trying to organically build up use cases and clear value before shipping this as part of the product.

IMO, we should work backwards from the use cases instead of shipping experimental APIs and establish use cases and clear value later

Alias are directly attached to concrete indexes. Views don't have those limitations.

I don't understand this. Alias can point to multiple indices and index patterns. How are views different than that?

While there is some existing filtering logic into alias, I'd rather not muddle what an index is with a view - as both of these can live side by side associated with different use cases.

And, this is why we need clarity on strong use case Views are serving to not extend alias and introduce Views

Even this 'basic' implementation is 3k lines of code, I like to start here to make incremental process.

Please correct me if I am wrong, but most of the 'basic' implementation is standard template for CRUD APIs. So, I will not worry about the number of lines in the code! :)
I am big proponent of making incremental progress, but would really to see the value proposition clearly before investing resources in it.

@andrross @reta @msfroh any other items that you'd like to see resolved before merging? I think I've covered all outstanding feedback

@peternied from implementation perspective, LGTM. As we discussed offline (and in comments here), there are clearly concerns regarding the compelling use cases of this feature and how the end result looks like. I would love to hear others' opinions on the subject, personally I am feel like I am missing something to say definitive "yes" or "no" with respect to getting it in.

Thanks for your thoughts @reta @jainankitk.  I'm trying to keep this PR and the View construct as lean as possible - as it is expensive to get a feature built up in OpenSearch.  If we can get a minimal version in core, it makes the next set of improvements faster.
I like the notion of iterative development - its fair if you think that this feature has too wide of a possibility space.  I am stuck with business realities that I do not yet know what is going to justify the next phase of 'funding' for this project.  Maybe it will be perf improvements, document level security, sandboxing request resources, search pipeline improvements, or something completely new to me.
I believe with this foundation unblocks further work that is largely outside of core [1].  I think keeping this work in main (v3.0.0) gives lots of headroom to improve or revert if it doesn't go anywhere.

[1] https://github.com/opensearch-project/OpenSearch/issues/12322

Thanks for your thoughts @reta @jainankitk. I'm trying to keep this PR and the View construct as lean as possible - as it is expensive to get a feature built up in OpenSearch. If we can get a minimal version in core, it makes the next set of improvements faster.
I like the notion of iterative development - its fair if you think that this feature has too wide of a possibility space. I am stuck with business realities that I do not yet know what is going to justify the next phase of 'funding' for this project. Maybe it will be perf improvements, document level security, sandboxing request resources, search pipeline improvements, or something completely new to me.

While I am fine with getting clarity on all the use cases for new feature over time, but it is good to have atleast 1-2 accompanying use cases.
I am also okay to wait for funding on those 1-2 use cases before we start development (since feature is marked experimental until then), but there should be more clarity on how that ties or is different compared to other existing constructs in the system

I believe with this foundation unblocks further work that is largely outside of core [1]. I think keeping this work in main (v3.0.0) gives lots of headroom to improve or revert if it doesn't go anywhere.
* [1] [[Feature Request] Views features wishlist #12322](https://github.com/opensearch-project/OpenSearch/issues/12322)

Probably other maintainers can chime in here, but adding logic to core with intent of potentially reverting does not make much sense to me. Maybe we can keep the PR open and merge as and when we get clarity on the use cases. Given most of the logic is in new files, I don't foresee many conflicts as well.

In my opinion, we have created a moving target that will prevent us from ever seeing delivery.
There are frequent changes in OpenSearch core which lack immediate use cases or which are contingent on further work to be valuable. I am happy to provide examples privately but will avoid doing so publicly since it is not considerate of those who have put effort into features I would list. I don't think there is anything wrong with these types of changes, but would ask we stick to a consistent standard when reviewing changes.
I would also like to address a couple of the main blockers that have come up here:
@reta you had previously voiced support of the change https://github.com/opensearch-project/OpenSearch/pull/11957#issuecomment-1946242978 but changed your mind and are now pushing @peternied to add additional code to see this move forward. Of course it is fine to change your mind--everyone does--but I think we should try to be mindful of the feedback we are providing when reviewing PRs. Reviewing the discussion, I have not seen much actionable input provided to Peter. We are setting a moving target when we ask him to add something to convince us the change is worthwhile. Instead we should be providing clear feedback saying what we would like to see for this to move forward. What issues have been raised that you feel need to be addressed before this can go forward? In my opinion, Peter has provided a basic use case and a framework for expansion. But if you feel we need a concrete difference between Views and index patterns or aliases can you please put that down so we can make sure that we give Peter direction in our feedback?
@jainankitk I understand your concerns about Views and agree that we should not be making changes which we expect to be reverted. That being said, I think Peter has provided some resources (for example his documentation of Views: https://github.com/opensearch-project/documentation-website/pull/6394) which can help address some of your questions. You also mentioned concerns about shipping experimental APIs. I believe the intent is for Views to only be a part of 3.0 for now so that this can be expanded upon. This means it will not be shipped. 3.0 is full of similar changes which are not completely supported at this time. I don't see any reason why we should hold a higher bar for Views versus other changes which have already gone in--to this point we have released entire repositories which we do not actively support. I do not think it will be realistic for Peter to make any further changes in places other than core without having the foundation in place. Adding this code in core offers very little cost (worse case scenario is we do have to revert the change a year from now) while providing a base for development which Peter has explored. As with Reta, if you are still unconvinced could you please list the actionable feedback which you would like to see for this to move forward? Is the issue the overlap with existing features that you want clarified or something else? I think this would greatly help this move forward.
Thanks everyone for reading; I appreciate how you are raising the bar and expecting the best but I want to keep us grounded.

@reta you had previously voiced support of the change #11957 (comment) but changed your mind and are now pushing @peternied to add additional code to see this move forward. Of course it is fine to change your mind--everyone does--but I think we should try to be mindful of the feedback we are providing when reviewing PRs.

Thanks @scrawfor99 , I didn't change my mind to be fair - the code looks good, as I stated, but I do not see the compelling use case for this feature (and so does @peternied in offline conversations), aka "killer app" that clears out all doubts and confirms that we need this feature. What I have seen though - the people (other reviewers) asking the same questions I've been asking and that makes me think that we may be pushing the feature we don't have a clear vision for yet. Also, again this came out from comments thread, we are doomed to provide multiple ways to do the same things - this will cause confusion for users and additional work for maintainers. As per Peter's comment, we won't be targeting 2.x (no direct user exposure), this is on the plus side to get it merged, I just fail to see what we could do with the views now (as per this change set), that we cannot do with core features (aliases / index patterns) or sql plugin.

I would really like "something" that is not an index that we can attach search pipelines to, so that they can be used to enforce rules on requests/responses (like what's proposed in https://github.com/opensearch-project/OpenSearch/issues/10938).
I really like the the idea of a construct where we can say "All search access through this window will follow these rules". I can see how aliases could be that construct. From an enforcement angle, if we can manage permissions on aliases (rather than the underlying indices), it would also solve my problem.

On the other hand, I see how aliases have a lot of preexisting baggage. Creating something new that doesn't have the baggage is also appealing (which is why I've been an eager cheerleader for views).

The views address only the read side (hence views) but aliases do both (write and read), I would be inclined to deprecate aliases in favour of views but that would not work.

@scrawfor99  - Thank you for providing your inputs. Please see my response below.

That being said, I think Peter has provided some resources (for example his documentation of Views: https://github.com/opensearch-project/documentation-website/pull/6394) which can help address some of your questions.

I went through the mentioned documentation and could see big overlap between the definition for alias and view.
Opensearch documentation for alias mentions:
An alias is a virtual index name that can point to one or more indexes.
If your data is spread across multiple indexes, rather than keeping track of which indexes to query, you can create an alias and query it instead.
Above documentation of view mentions:
A View in OpenSearch is a virtual index that allows you to define a projection over one or more physical indexes. It enables you to perform search operations across multiple indexes as if they were a single index. Views can be used hide index schema, dynamically reshape from its indexed format, and limit data exposure.

I believe the intent is for Views to only be a part of 3.0 for now so that this can be expanded upon. This means it will not be shipped. 3.0 is full of similar changes which are not completely supported at this time. I don't see any reason why we should hold a higher bar for Views versus other changes which have already gone in--to this point we have released entire repositories which we do not actively support.

I will not comment on the other features or repositories that have already gone in. But as principal, we have to start somewhere and keep raising the bar. Mistakes made in past should not be a reason to keep repeating them, unless ofcourse we agree it was not a mistake.

I do not think it will be realistic for Peter to make any further changes in places other than core without having the foundation in place. Adding this code in core offers very little cost (worse case scenario is we do have to revert the change a year from now) while providing a base for development which Peter has explored. As with Reta, if you are still unconvinced could you please list the actionable feedback which you would like to see for this to move forward? Is the issue the overlap with existing features that you want clarified or something else? I think this would greatly help this move forward.

It is primarily the significant overlap with existing alias feature due to which I am not completely sure if we need Views. And thats where I am wondering if there is some documentation on clarifying how Views are different than alias and why can't aliases evolve to what we are envisioning for Views. I am generally in favor of improving existing feature over introducing new features.

On the other hand, I see how aliases have a lot of preexisting baggage. Creating something new that doesn't have the baggage is also appealing (which is why I've been an eager cheerleader for views).

Well put @msfroh
I appreciate the back and forth, merging this change isn't a one-way door for OpenSearch, lets keep iterating on these ideas.