Initial commits for the v3 programmatic interface

PR for #2193: Interfaces and a simple implementation for the OpenAPIConfig interfaces a new module called swagger-web.

Thanks a lot for the feedback @frantuma
Here are my thoughts:

That's an interesting point - can you provide an example scenario where we would use the OpenAPIController and OpenAPIConfig in a non-web environment?   Given that these interfaces are controlling the building of documentation for REST APIs, I am having a hard time imagining a non-web environment.  By the way, the choice of "web" was on purpose to keep it general from jaxrs or servlet, so to mean any web module that is exposing REST APIs.

Good point.  I like OpenApiConfigBuilder + build method.

We definitely don't want more than 1 Filter version in the repo, so if you could please share the current work that would be great.  We could always pass that in through the general user-defined map too.

That's true.  We'll revisit that.  Perhaps we could have both the classes and the package variations, so the user can have choices (they are optional)

That's a tricky area.  I can see a point in providing access to the Reader to control the reading behaviour - but then again, if users have a nice Filter that controls what goes in, would someone really want to implement a full Reader, since transforming a set of Classes into OpenAPI models is what most people would use the library for.     For the Scanner, I really think that taken care of from OpenAPIConfig (ie: using the getClasses method, and user defined Map<String,Object>).

We were hoping to push these in as the "spec", so we can work in parallel in the plumbing from different implementations (ie: OSS, WebSphere, or vendors, etc).  We will definitely contribute a JAXRS sample for OpenAPI v3, but using the OpenAPIConfig to configure the Reader (beyond its current default) is something that could be done in a separate issue/PR.

Committed changes to address comments 2 and 4 above.

@arthurdm some comments about points 1 and 5 above.

you're probably right, I was thinking (not much..) about e.g. some scenario where user would want to expose an existing app service layer, with the spec used to build a dynamic javafx or swing UI based on the schema or the like; Don't have a strong opinion here though.

here I still think it would be valuable to keep the scanning logic separate from config building, as it could e.g. potentially be time consuming, and also be "dynamic" and so on. I understand we can pass whatever in a Map<String, Object>, but this sounds too generic imo. Also for the reader I would think that adding it to the config would add value, e.g. to allow a user to use a custom (or library) jaxrs-free reader, or a scala reader, keeping an easy entry point.
I understand we have diverging opinions here, not sure if @webron or @fehguy have any comment to add, we can give it maybe a little more thought and then we decide.

Thanks @frantuma
I can see your points about Reader and Scanner.  How about this revised set of interfaces:

public interface OASConfigBuilder {
OASConfig build(Map<String, Object> environment);
}
public interface OASConfig {
OpenAPI getOpenAPI();
 boolean isScanDisabled();

 Class<?> getFilterClass();

 Set<String> getResourcePackages();

 Set<String> getIgnoredClasses();

 Map<String, Object> getOptions();

}
public interface OASScanner {
Set<Class<?>> classes();
Map<String, Object> resources()
}
public interface OASReader {
OpenAPI read(OASConfig configuration, OASScanner scanner)
}
So basically, the application developer would choose which services they want to provide to the framework (OSS Swagger, WebSphere, etc):
custom config:   META-INF/services/io.swagger.oas.web.OASConfigBuilder
custom scanner:  META-INF/services/io.swagger.oas.web.OASScanner
custom reader:  META-INF/services/io.swagger.oas.web.OASReader
For each of those, the framework would have default services impl if they aren't specified by the app dev.
Thoughts?

@arthurdm Thanks for comments and revised PR! getting there I'd say.. I add here a couple of  comments.
We have a working "integration implementation" draft, which makes use of swagger-web module for a part of the functionality. Possibly it can be further refactored to "integrate more" with swagger-web design concepts, however at the moment it references and implements scanner and reader, and uses config and builder classes which implement swagger-web related interfaces.
This effort is visible in branches for core and for samples (jersey2 and java-dropwizard modules).
At the moment it uses a slightly modified version of OpenApiConfig interface, which basically includes also reader and scanner and adds couple of props/methods which were previously scattered around 2.0 config classes.
As you know, my idea is that scanners and readers are part of the game, and their implementation classes would be part of the config; if you have a strong opinion against it however, we can keep them out from the interface (we'll keep them in the specific oss "integration implementation"), in this case this would be the interface more similar to what you had in mind (with additional props added).
Another possible point is about returning Class from e.g. getFilterClass, in my idea I'd rather have a simple and serializeable  bean so returning strings, this would facilitate loading of config from files, or any declarative config, with no custom deserialization mechanism, and would also delegate class loading to the caller. This alternative is implemented here (for "full config), or here (for "minimal config").
Basically my preferences would be in this order:

Full / Strings
Full / Classes
Basic / Strings
Basic / Classes

Please let me know what you think, and what's your preference, and we can use that (merging with your latest effort) in the delivered version.
An additional point is about naming: I would feel that sticking to OpenApi instead of oas would be possibly be more consistent with the rest of the code and the spec, let me know what you think.

@arthurdm forgot a further point about builder: in my version I removed the parameters (as you coded in the first place..), as that would be passed to the builder via its implementation

I had a chat  with @frantuma today and the latest proposal that we're discussing is:
Introduce these 4 interfaces:
`public interface OpenAPIConfigBuilder {
OpenAPIConfig build(Map<String, Object> environment);
}
public interface OpenAPIConfig {
OpenAPI getOpenAPI();
String getFilterClass();

String getReaderClass();

String getScannerClass();

Set<String> getIgnoredClasses();

Map<String, Object> getOptions();

Set<String> getResourceClasses();

Set<String> getResourcePackages();

boolean isScanDisabled();

}
public interface OpenAPIScanner {
Set<Class<?>> classes();
Map<String, Object> resources()
}
public interface OpenAPIReader {
OpenAPI read(OpenAPIConfig configuration, OpenAPIScanner scanner)
}
`
So the flow is that an application developer provides (optionally) an implementation of OpenAPIConfigBuilder, which would then allow then to give the framework a copy of OpenAPIConfig that may contain things like a custom Reader, Scanner, options, etc.
Not part of this PR, but for completeness:  the idea is that the "specification" part of MicroProfile's OpenAPI Java proposal will say something like this:
The application developer can provide an OpenAPIConfig to implementations of the OpenAPI Java spec in 2 ways:
(1) By specifying an openapi-config.yaml file in the application's classpath, which contains the configuration corresponding to an OpenAPIConfig object (link to schema)
(2) By specifying the qualified name of their OpenAPIConfigBuilder service inside META-INF/services, following JDK practice (https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html)

@arthurdm @webron @fehguy
looks quite ok to me, just one note:
Would add to config interface isScanAllResources and isPrettyPrint, and remove (if there is no particular reason to keep it) isScanDisabled, so like:
public interface OpenAPIConfig {

    Set<String> getResourcePackages();

    Set<String> getResourceClasses();

    String getReaderClass();

    String getScannerClass();

    String getFilterClass();

    Collection<String> getIgnoredRoutes();

    OpenAPI getOpenAPI();

    Map<String, Object> getUserDefinedOptions();

    boolean isScanAllResources();

    boolean isPrettyPrint();
}

@arthurdm an additional note about loading via service loader; this doesn't cover the case where uses needs different configs in same JVM (e.g. a config per servlet); a possible solution could be adding a getName() to the interface to allow the framework to e.g. match it to servlet name, even if I don't like it so much.

Hey @frantuma - thanks for the comments.
We chose isScanDisabled instead of isScanAllResources to be more descriptive of the purpose of this boolean:  is it used to disable scanning.     We felt that isScanAllResources was not being descriptive enough (for example: I am turning off the scan for ALL resources, but maybe SOME resources will still be scanned?)
Regarding the multiple configs per JVM, I see this as an implementation add-on, rather than a specification item.  Therefore, this is where the Map<String,Object> that is passed into OpenAPIConfigBuilder#build can be used, where the implementation (ie: Swagger OSS) could pass in an option such {key = SERVLET_NAME, value = MyServlet} and the proper OpenAPIConfig would be return.

with regard to isScanDisabled and isScanAllResources, isScanAllResources is bad naming actually (better isReadAllResources), and is meant (used like that in 2.0) to allow to read/resolve all classes passed to reader independently of the presence of specific annotation.
I am not sure what is the usage/meaning isScanDisabled, what does it mean to disable scanning?
About the multiple configs per JVM, in this way if I am not mistaken, in order to have different configs e.g. per servlet (a quite common scenario), a final user would need to implement a builder without knowing what this Map<String,Object> contains, for example this SERVLET_NAME, in the best case it would be specified by the implementation and therefore different depending on implementation.

hey @frantuma - I see now what you mean with isReadAllResources.   The optionisScanDisabled is for scenarios where the user will provide a full OpenAPI model via the Config, so it turns off annotation scanning. (no need to scan the app).
About the multiple JVMs, I still believe that is implementation specific - for example, in Liberty our JVM (the app server) supports multiple applications (WAR module), and each application will have their own OpenAPIConfig inside their WAR module.  So even without extra Map options we can already achieve multiple apps in a JVM scenario.
Perhaps you are referring to multiple APIs in a single WAR module (single app), which then that can be handled differently depending on the implementation (some implementations may not support it).
Lastly, I think isPrettyPrint is outside of the spec - it deals with the serialization format of the vendor/implementation as it handles the OpenAPI model that it got from the OAS3 Java programming model (ie: from OpenAPIConfig or from OpenAPIReader).  How that model is handled will vary per implementation.

hey @arthurdm isScanDisabled is now clear, I feel that this also could be an implementation specific thing, but no strong opinion; the same goes for prettyPrint and isReadAllResources at this point.
With regard to the multiple config per JVM, it's indeed per classloader (e.g. per web application), a common scenario would be e.g. exposing v1 and v2 of APIs or the like; I still feel this would fit in scope of the "spec", but maybe it can be part of a later version.

hey @frantuma - I think we can then agree to include isReadAllResources and isScanDisabled as spec, and other things like isPrettyPrint as implementation-dependent.  So we will add isReadAllResources into the current PR.
Regarding the multiple API per application, my thought was that the javadoc of the OpenAPIConfigBuilder#build method would specify that if implementations/vendors want to support different configurations for different APIs in the same application, then they must define an appropriate key  in the Map that gets passed into build - the reason I am trying to keep this away from spec is because "servlet name" wouldn't be application to all implementations, such as Spring.

Using https://github.com/swagger-api/swagger-core/pull/2398 instead


Improve return messages for verify-image-repository API

When image repository doesn't have auto_rebuild_tags set, the return
message of verify-image-repository API tells user the repository doesn't
exist, like this:
{
    "error": "Bad Request",
    "message": "Cannot get repository foo/bar from Lightblue.",
    "status": 400
}

Instead of this message, freshmaker should tells users that this
repository doesn't have auto_rebuild_tags set. After change, the return
message will be something like:
{
    "error": "Bad Request",
    "message": "The \"auto_rebuild_tags\" in COMET is not set.",
    "status": 400
}

In this change, the return message is changed to include more info about
the repository and the rebuildable images, including the 'auto_rebuilds_tags'
for repository and tags of image. Something like this:
{
    "auto_rebuild_tags": [
        "latest"
    ],
    "images": {
        "foobar-1-123": {
            "content_sets": [
                "content-set-1",
                "content-set-2"
            ],
            "tags": [
                "latest",
                "1-123"
            ]
        }
    },
    "msg": "Found 1 images which are handled by Freshmaker for defined content_sets."
}

JIRA: CLOUDWF-3237

+1


Theads and asio-grpc

Thank you for implementing this excellent project to provide a consolidated way of executing async grpc command and send/receive tcp packages asynchronously with boost asio library. I just begin to use boost asio recently and have a couple of quesions when using this library.
According to this link: https://www.boost.org/doc/libs/1_78_0/doc/html/boost_asio/overview/core/threads.html, multiple threads may call io_context::run() to set up a threads pool and the io_context may distribute work across them. Dose asio-grpc's execution_context also guarantee thread safety if threads pool is enabled on it? I am using C++20 coroutines and assuming that each co_spawn will locate a thread from the threads pool and run the composed asynchronous operations. Correct me if my understanding is wrong. What if the composed asynchronous operations contains a blocking operation, it may block the running thread and how can I prevent the other co_spawn call to use the blocked thread for execution? In additional, co_spawn could spawn from both execution_context and excutor. I am guessing that if spawn from execution_context it will locate a new thread and run while from excutor, it will just run on the thread that the excutor is running. Is my guessing correct?
Meanwhile #8 mentions that if co_spawn non-grpc async operation like steady_timer from grpc_context, it will automatically spawns a 2nd io_context thread. So it seems that asio-grpc internally maintain two threads for both grpc execution_context and io_context to run async grpc operations and other async non-grpc operations. And the last comments says version 1.4 would also support ask io_context for a agrpc::GrpcContext. Considering my application would serve many clients and for each client's requst issue one single composed asynchronous operations containing one async grpc call and several async tcp read&write to the server call and response back to the client, will asio-grpc guarantee there won't have interleave between the grpc operation and the tcp operations when the single composed asynchronous operation is co_spawned from either grpc_context or io_context since they are from two context on two thread? Also does asio-grpc support the mode of having threads pool for io_context and single thread for grpc_context or both have threads pool enabled?
                      one single composed asynchronous operations
                     /                                            \
client1 --> { co_wait async grpc operation, co_wait async tcp operations } --> server
client2 --> { co_wait async grpc operation, co_wait async tcp operations } --> server
clientN ...

Hope to get some guidence from you. Thanks.

Hi, thank you for your positive feedback. I count five questions :)

Does asio-grpc's execution_context also guarantee thread safety if threads pool is enabled on it?

At the moment the GrpcContext may only be run on one thread at a time, see documentation of run(). This is because gRPC used to recommend using one grpc::CompletionQueue per thread. (They now recommend two threads but I have not implemented support for that yet. I also assume that the extra synchronization needed in both library and user code will outweigh the potential performance benefit.)
You are correct that asio::io_context choses one of the threads that call run() when co_spawning a coroutine.

What if the composed asynchronous operations contains a blocking operation, it may block the running thread and how can I prevent the other co_spawn call to use the blocked thread for execution?

True blocking is generally bad for both asio::io_context and GrpcContext. By true I mean things like waiting for a std::future or calling std::this_thread::sleep_for. Other "blocking" operations like async_waiting for a asio::steady_timer are fine. If you have true blocking functions then you could post them onto a thread_pool:
    asio::thread_pool thread_pool;
    asio::co_spawn(grpc_context,
                   [&]() -> asio::awaitable<void>
                   {
                       co_await asio::post(asio::bind_executor(thread_pool, asio::use_awaitable));
                       // Now executing on a thread of the thread_pool, perform blocking tasks here

                       // Optionally switch back to the GrpcContext explicitly.
                       co_await asio::post(asio::use_awaitable);
                       // Or switch back implicitly by using a rpc function like
                       co_await agrpc::finish(writer, grpc::Status::OK, asio::use_awaitable);
                   });

I am guessing that if spawn from execution_context it will locate a new thread and run while from excutor, it will just run on the thread that the excutor is running. Is my guessing correct?

No, the behavior of co_spawn with executor and co_spawn with execution_context is identical. This applies to both asio::io_context and GrpcContext.

Considering my application would serve many clients and for each client's requst issue one single composed asynchronous operations containing one async grpc call and several async tcp read&write to the server call and response back to the client, will asio-grpc guarantee there won't have interleave between the grpc operation and the tcp operations when the single composed asynchronous operation is co_spawned from either grpc_context or io_context since they are from two context on two thread?

I am not sure what you mean by 'interleave', there won't be any race conditions. I personally do not like how asio::execution_context creates a background thread when using GrpcContext for asio io objects like steady_timer. I prefer to manage the io_context and GrpcContext explicitly.
If your application handles RPCs requests but performs most of its work on an io_context then you could do:
    agrpc::repeatedly_request(
        &test::v1::Test::AsyncService::RequestUnary, service,
        asio::bind_executor(grpc_context,
                            [&](grpc::ServerContext& server_context, test::msg::Request& request,
                                grpc::ServerAsyncResponseWriter<test::msg::Response>& writer) -> asio::awaitable<void>
                            {
                                // Just an example, could be tcp_socket etc.
                                asio::steady_timer timer{io_context, std::chrono::milliseconds(10)};
                                co_await timer.async_wait(asio::bind_executor(io_context, asio::use_awaitable));
                                // By using bind_executor execution will not switch back to the GrpcContext
                                // when the timer expires which might provide better performance if you perform
                                // more io_context related operations afterwards

                                // Eventually finish the RPC
                                test::msg::Response response;
                                co_await agrpc::finish(writer, response, grpc::Status::OK, asio::use_awaitable);
                            }));

or co_spawn onto the io_context directly. Note that you must use asio::bind_executor for all functions of this library in that case:
    asio::co_spawn(
        io_context,
        [&]() -> asio::awaitable<void>
        {
            test::msg::Request request;
            grpc::ServerAsyncResponseWriter<test::msg::Response> writer{&server_context};
            if (!co_await agrpc::request(&test::v1::Test::AsyncService::RequestUnary, service, server_context, request,
                                         writer, asio::bind_executor(grpc_context, asio::use_awaitable)))
            {
                co_return;  // gRPC server is shutting down
            }
            // Now executing on a thread of the GrpcContext

            // co_spawn the next agrpc::request here.
            // agrpc::repeatdely_request does that automatically but it cannot yet be customized to co_spawn onto an
            // io_context directly.

            // Perform your io_context related tasks. The first awaited async operation will automatically switch
            // execution to the io_context.
            asio::steady_timer timer{io_context, std::chrono::milliseconds(10)};
            co_await timer.async_wait(asio::use_awaitable);
            // Now executing on a thread of the io_context

            // Eventually finish the RPC
            test::msg::Response response;
            co_await agrpc::finish(writer, response, grpc::Status::OK,
                                   asio::bind_executor(grpc_context, asio::use_awaitable));
        },
        asio::detached);

Also does asio-grpc support the mode of having threads pool for io_context and single thread for grpc_context or both have threads pool enabled?

Like mentioned earlier, GrpcContext currently does not support thread pools and can only use a single thread for RPCs. Even if it did, you would have full control over how many threads are being used by calling run() on as many threads as you want.

Another variation of the last example that I gave that takes care co_spawning the next agrpc::request in a timely fashion:
    asio::io_context io_context;
    auto request_handler = [&](grpc::ServerContext& server_context, test::msg::Request& request,
                               grpc::ServerAsyncResponseWriter<test::msg::Response>& writer) -> asio::awaitable<void>
    {
        // Now executing on a thread of the io_context.
        // Perform your io_context related tasks.

        // Eventually finish the RPC.
        test::msg::Response response;
        co_await agrpc::finish(writer, response, grpc::Status::OK,
                               asio::bind_executor(grpc_context, asio::use_awaitable));
    };
    agrpc::repeatedly_request(&test::v1::Test::AsyncService::RequestUnary, service,
                              asio::bind_executor(grpc_context,
                                                  [&]<class T>(agrpc::RepeatedlyRequestContext<T>&& context)
                                                  {
                                                      asio::co_spawn(
                                                          io_context,
                                                          [&, context = std::move(context)]()
                                                          {
                                                              return std::apply(request_handler, context.args());
                                                          },
                                                          asio::detached);
                                                  }));

Thank you for the very detailed and quick response. I got the point that GrpcContext use a single thread which works like a strand and asio::bind_executor will dispatch the async grpc calls to the GrpcContext one by one.
Another question regarding the grpc calls.  To increase the concurrency of grpc calls, is it able to enable multiple connections  and load balance policy in a grpc channel by grpc::CreateCustomChannel? And will it have better performance?

I cannot find an option to enable multiple connections among the channel arguments, where did you read about it? I cannot see how it would be benefit for HTTP2 connections which multiplex within one connection already anyways.
I wrote a simple benchmark for a helloworld client here. On my machine it seems that:

Creating multiple channels does not improve throughput, e.g. one channel per thread. (Even with the GRPC_ARG_USE_LOCAL_SUBCHANNEL_POOL option, in fact that option reduces performance in this case.)
Creating multiple stubs does not improve throughput, e.g. one stub per thread.
Creating multiple agrpc::GrpcContexts (per thread) improves throughput as expected.

I have not used load balance policies yet, so I cannot help here. You could try to get an answer in the mailing group or by creating an issue on grpc/grpc.

Hi Tradias, I tested the aysnc grpc call performance in multiple coroutine and created https://github.com/vangork/test-async-grpc/blob/main/test-grpc-cpp/test.cpp. It can archieve higher request/s rate than doing serially in one single post https://github.com/vangork/test-async-grpc/blob/main/test-grpc-cpp-benchmark/test-benchmark.cpp(customized based on your benchmark). But in this case, is std::unique_ptr<pravega_grpc::ControllerService::Stub> stub_) thead safe or it is necessary to put a Rwlock over it as it is sharing between the coroutines?
Regarding to the multiple connections, I also can't find a method or property to set it. But just came across the GRPC Performance Best Practices doc metioned:

Each gRPC channel uses 0 or more HTTP/2.

It also mentioed Create a separate channel for each area of high load, not sure if one stub can connect to a channel pool?
The commonly used grpc library tonic for rust language supports to create a channel with balance_list. https://github.com/vangork/test-async-grpc/blob/e558d84592a6d90aec8b67389e5a302aabb3e013/test-grpc-rust/src/main.rs#L26-L30. FYI

Better performance is expected since your example uses a higher concurrency. I must say I do not know how to properly benchmark a gRPC client, I have only ever benchmarked servers using ghz. ghz differentiates between number of CPUs, maximum number of parallel in-flight requests (they call this 'concurrency') and number of connections (I assume this refers to the number of Channels). If we want to benchmark a gRPC client then we probably want to make all of these things configurable, so that we can play around with different values and see what works best.
The Stub is coming from gRPC itself and from what I can see in its implementation it seems thread-safe. In your example there is only one thread that uses the Stub, so no sharing takes place anyways.
Creating a pool of Channels and distributing work among them sounds easy enough. I also found this recommendation in the .NET gRPC library:

Use a pool of gRPC channels, for example, create a list of gRPC channels. Random is used to pick a channel from the list each time a gRPC channel is needed. Using Random randomly distributes calls over multiple connections.
https://docs.microsoft.com/en-us/aspnet/core/grpc/performance?view=aspnetcore-6.0#connection-concurrency

I would personally use a round-robin strategy instead of Random.
I would create one Stub per Channel in such a pool (similar to here), instead of worrying about attaching one Stub to multiple Channels. A Stub is actually rather lightweight, so that shouldn't be an issue.
I think such a channel pool should not be part of asio-grpc seeing that it can be implemented entirely without asio-grpc and seeing that the gRPC team is working on making it unnecessary (see 'side note').

I have adjusted the example here to create a configurable amount of GrpcContexts and Channels and round-robin distribute work among them. I hope that provides an idea on how to possibly implement it on your side.

Thanks very much!

I will close the issue here. If you have more questions or requests then do not hesitate to open another issue :).


RFE: --no-network

Short description
When trying to run
dephell deps convert --from pyproject.toml --to setup.py

inside of the building scripts for openSUSE packaging (because sometimes pyproject.toml-based tools are just not stable yet enough).
Unfortunately, there is absolutely no network access inside of the build systems (for obvious security reasons), so every run of the scripts fails.
What I would like to get was the option --no-network (or something like that), which would cause dephell whenever it really needs some resource which is not installed in the system, just print the error message with the missing resources and failed.
Output
[    8s] + dephell deps convert --traceback --level=DEBUG --from pyproject.toml --to setup.py
[    9s] DEBUG load dependencies... (format=flit, path=pyproject.toml)
[    9s] ERROR ConnectionError: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /pypi/requests/json (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f15bb39f0d0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
[    9s] Traceback (most recent call last):
[    9s]   File "/usr/lib/python3.8/site-packages/urllib3/connection.py", line 156, in _new_conn
[    9s]     conn = connection.create_connection(
[    9s]   File "/usr/lib/python3.8/site-packages/urllib3/util/connection.py", line 61, in create_connection
[    9s]     for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
[    9s]   File "/usr/lib64/python3.8/socket.py", line 918, in getaddrinfo
[    9s]     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
[    9s] socket.gaierror: [Errno -3] Temporary failure in name resolution
[    9s]
[    9s] During handling of the above exception, another exception occurred:
[    9s]
[    9s] Traceback (most recent call last):
[    9s]   File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 665, in urlopen
[    9s]     httplib_response = self._make_request(
[    9s]   File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 376, in _make_request
[    9s]     self._validate_conn(conn)
[    9s]   File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
[    9s]     conn.connect()
[    9s]   File "/usr/lib/python3.8/site-packages/urllib3/connection.py", line 300, in connect
[    9s]     conn = self._new_conn()
[    9s]   File "/usr/lib/python3.8/site-packages/urllib3/connection.py", line 168, in _new_conn
[    9s]     raise NewConnectionError(
[    9s] urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f15bb39f0d0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution
[    9s]
[    9s] During handling of the above exception, another exception occurred:
[    9s]
[    9s] Traceback (most recent call last):
[    9s]   File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
[    9s]     resp = conn.urlopen(
[    9s]   File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 719, in urlopen
[    9s]     retries = retries.increment(
[    9s]   File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line 436, in increment
[    9s]     raise MaxRetryError(_pool, url, error or ResponseError(cause))
[    9s] urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /pypi/requests/json (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f15bb39f0d0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
[    9s]
[    9s] During handling of the above exception, another exception occurred:
[    9s]
[    9s] Traceback (most recent call last):
[    9s]   File "/usr/lib/python3.8/site-packages/dephell/cli.py", line 64, in main
[    9s]     result = handler()
[    9s]   File "/usr/lib/python3.8/site-packages/dephell/commands/deps_convert.py", line 76, in __call__
[    9s]     resolver.apply_envs(set(self.config['envs']))
[    9s]   File "/usr/lib/python3.8/site-packages/dephell/controllers/_resolver.py", line 166, in apply_envs
[    9s]     self.apply(dep)
[    9s]   File "/usr/lib/python3.8/site-packages/dephell/controllers/_resolver.py", line 29, in apply
[    9s]     for new_dep in parent.dependencies:
[    9s]   File "/usr/lib/python3.8/site-packages/dephell/models/dependency.py", line 86, in dependencies
[    9s]     for dep in self.group.dependencies:
[    9s]   File "/usr/lib/python3.8/site-packages/dephell/cached_property.py", line 18, in __get__
[    9s]     value = obj.__dict__[self.func.__name__] = self.func(obj)
[    9s]   File "/usr/lib/python3.8/site-packages/dephell/models/dependency.py", line 66, in group
[    9s]     for group in self.groups:
[    9s]   File "/usr/lib/python3.8/site-packages/dephell/models/groups.py", line 169, in __iter__
[    9s]     release = self.releases[0]
[    9s]   File "/usr/lib/python3.8/site-packages/dephell/cached_property.py", line 18, in __get__
[    9s]     value = obj.__dict__[self.func.__name__] = self.func(obj)
[    9s]   File "/usr/lib/python3.8/site-packages/dephell/models/groups.py", line 34, in releases
[    9s]     releases = self.dep.repo.get_releases(self.dep)
[    9s]   File "/usr/lib/python3.8/site-packages/dephell/controllers/_repos.py", line 138, in get_releases
[    9s]     return repo.get_releases(dep=dep)
[    9s]   File "/usr/lib/python3.8/site-packages/dephell/repositories/_warehouse/_api.py", line 75, in get_releases
[    9s]     response = session.get(url, auth=self.auth)
[    9s]   File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 543, in get
[    9s]     return self.request('GET', url, **kwargs)
[    9s]   File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
[    9s]     resp = self.send(prep, **send_kwargs)
[    9s]   File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
[    9s]     r = adapter.send(request, **kwargs)
[    9s]   File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
[    9s]     raise ConnectionError(e, request=request)
[    9s] requests.exceptions.ConnectionError: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /pypi/requests/json (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f15bb39f0d0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
[    9s] error: Bad exit status from /var/tmp/rpm-tmp.Z2jJpN (%prep)
[    9s]
[    9s]
[    9s] RPM build errors:
[    9s]     Bad exit status from /var/tmp/rpm-tmp.Z2jJpN (%prep)
[    9s]
[    9s] stitny failed "build python-dephell_changelogs.spec" at Mon Mar 23 23:15:33 UTC 2020.
[    9s]

Steps to reproduce
See above: run dephell deps convert in the openSUSE Build System.
Config
This was pyproject.toml from dephell_changelog package.
Versions
{
  "cache": "0b",
  "path": "/usr/lib/python3.8/site-packages/dephell",
  "python": "/usr/bin/python3",
  "version": "0.8.1",
  "versions": {
    "dephell-archive": "0.1.6",
    "dephell-argparse": "0.1.2",
    "dephell-changelogs": "0.0.1",
    "dephell-discover": "0.2.10",
    "dephell-licenses": "0.1.6",
    "dephell-links": "0.1.5",
    "dephell-markers": "1.0.3",
    "dephell-pythons": "0.1.14",
    "dephell-setuptools": "0.2.3",
    "dephell-shells": "0.1.4",
    "dephell-specifier": "0.2.1",
    "dephell-venvs": "0.1.17",
    "dephell-versioning": "0.1.1"
  }
}

@mcepl Why do you think this would help any? dephell converts projects across tools, those tools may or may not have the ability to do something like that (most don't).

I am sorry, but I really don't understand what you are saying. My RFE was to make dephell deps convert working in the environment where network is not available. What’s so difficult to understand?

Thank you for your feedback. I think the problem isn't a missed flag to forbid network requests but that it tries to do so when it's possible to avoid it. Let me see what we can do to not make network requests here.


Support specifying virtual network name for Azure agents

Please squash your commits into single one.

Done.

I see 3 problems in this pull request:

A virtual network name parameter is optional, must not be marked with the star
Since this option is advanced and not demanded by most users, it should be hidden under internal property. (See sample usage of ENABLE_IMAGE_NICKNAME in https://github.com/JetBrains/teamcity-vmware-plugin/search?utf8=✓&q=ENABLE_IMAGE_NICKNAME)
Since the option requires a name of existing network, the values in the field should be prepopulated by the plugin instead of having user to type the value in.

Unfortunately, I can't accept this pull request for now.
If there's a demand for this option, I will implement points above by myself later.


Converting mistral7b finetuned model produces garbage output

Hi,
I've finetuned a Mistral-7B-v0.1 model from hugging face using QLoRa as per the typical SFT scripts released by the hugging face team. However, after converting it to gguf format using the docker method, I found that my model produces garbage.
For reference, this is the format of the prompts I trained on:
### User:
{{user prompt}}

### Assistant:
{{assistant response}}
</s>

The inference results from hugging face look ok, I struggle with getting the model to stop generating but setting max_tokens generally solves this (although an unideal workaround).
But when converting to gguf and running via Ollama, the model often goes off-task and does its own thing. Furthermore, unless I put '### Assistant:' and '' as stop tokens, it often produces input interleaving with lots of '### Assistant''s e.g.:
We've noticed that some people love to go sightseeing in Paris and we have a special offer for this year!

### Assistant:
I love Paris

### Assistant:
I love Paris.

So I have a few questions/paths to investigate:

Has anyone been able to successfully fine-tune a model and convert it to gguf and get it to work well with Ollama?
Could it be that I'm doing something wrong in converting the mistral finetuned model? I save the merged lora model and run the docker converting to gguf via llama.cpp on that merged model as per normal.
Could it be that I need to produce a better 'base' fine-tuned model before converting to gguf for it to work well in Ollama?

Note, I also tried converting a base mistral7b model downloaded from hugging face as as sanity check and that worked great.

I've used fine-tuned models converted from safetensors with ollama successfully, but I usually check out the latest ggerganov/llama.cpp and run the convert.py and quantize.py from the repo, because I was burned by some gguf bugs in the past. You might want to try this, but your description of needing to use stop tokens sounds normal to me, most models will continue for some time without explicit stop tokens.

@65a @amritap-ef  I'm having a hard time understanding what I'm doing wrong to upload fine-tuned models to ollama. I had recently created fine-tuned layers on google-collab here. I run:
curl http://server.local:11434/api/create -d '{
  "name": "test_mistral",
  "modelfile": "FROM mistral\nADAPTER /home/robot/adapter_model.bin"
}'

which creates a fine-tuned model that doesn't work. I get an error about the fine-tuned layers. I don't know what to do and scowering the internet hasn't gotten me closer to figuring that out. Please any help would be greatly appreciated.

Just found this issue again. There are now docs which explain how to do this here and a lot better tutorials (on the web/youtube) on how to do this properly. I'm going to go ahead and close out the issue since it's now a year old 😅


[9.x] Update the format for database stored dates to include microseconds

This is an attempt to fix this issue: https://github.com/laravel/framework/issues/44157

This is a breaking change unfortunately. It's also intentionally that microseconds are not saved here.


Create streaming interface for client streaming.

What this PR does / why we need it:
Extracts the current websocket streaming in to an interface that can be replaced.
This work was done by @mstergianis in #1905 this PR is just addressing the merge conflict.

@wwitzel3 happy to resolve the above two requests. I'll send you a patch

@wwitzel3 in going through the exercise of resolving the requests; I realized that StreamError should also be part of pkg as a custom implementation should have the option to return a fatal stream error. Wanted to get your thoughts on moving this much implementation out of internal before I spend more time on it.

Making StreamError public is fine - realistically this is only going to be consumed internally by folks at VMware.
Eventually, internal/errors should be public as well and can have a similar API to pkg/store so third party usage or tools can add warnings/errors that will be visible by UI.

Apologies, I made the changes requested above and forgot to update this thread.
I've made a PR against @wwitzel3's branch. The diffs ended up being quite large so I thought that the easiest way to make the change.

replaced by #2581


Vendor.js chunk changes everytime on build

http://stackoverflow.com/questions/38530831/webpack-vendor-chunk-changing-every-time
Webpack version:
"^1.13.1"
Please tell us about your environment:
OSX El Capitan
Current behavior:
Expected/desired behavior:

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem along with a gist/jsbin of your webpack configuration.

What is the expected behavior?

What is the motivation / use case for changing the behavior?

Browser: [all | Chrome XX | Firefox XX | IE XX | Safari XX | Mobile Chrome XX | Android X.X Web Browser | iOS XX Safari | iOS XX UIWebView | iOS XX WKWebView ]

Language: [all | TypeScript X.X | ES6/7 | ES5 | Dart | ...]

There is a huge discussion in this issue about what can cause that: https://github.com/webpack/webpack/issues/1315
I hope that you can find the solution there.
And the relevant example: https://github.com/webpack/webpack/tree/master/examples/chunkhash

Its not the same as #1315 as #1315 says that the chunk hash changes while the file contents are the same. My file content changes @somebody32

@paramsingh88 see last comment here, could be relevant too: https://github.com/webpack/webpack/issues/959

duplicate of #1315

A records file or the NamedModulesPlugins/HashedModulesPlugin would help here. the problem is ids changing.


[PLAT-2319] Allowing Axis on the Transform Widget to be disabled

Summary

These changes introduce an ability for the developer to disable specific axis on the transform widget. For example:
          <vertex-viewer-transform-widget
            id="transform-widget"
            x-rotation-disabled
            y-rotation-disabled
            z-rotation-disabled
          ></vertex-viewer-transform-widget>

including the properties above will result in the following:

Test Plan

Verify that the following properties disable the action, and show a visual indicator that the triangle meshes are unable to be interacted with for each of the following triangles:
x-rotation-disabled,   y-rotation-disabled,   z-rotation-disabled,   x-translation-disabled, y-translation-disabled, z-translation-disabled

Release Notes

Possible Regressions

Dependencies

The sonar issue appears to be unrelated to my changes.

Can the outline of the disabled handles also be greyed out?

Can the outline of the disabled handles also be greyed out?

Discussed at our sync - this should be a dedicated effort. Created this ticket. https://vertexvis.atlassian.net/browse/PLAT-2910


Bitwise operators and comparison operators

This will pretty much finish the compiler / bytecode MVP! :tada:
Neither is perfect, there are still some missing improvements to both even to bring it into full parity with PUC-Rio Lua 5.3, but after this all of it exists in some form.
I think the only things left for perfect parity are supporting more than 64k constants in a function (Lua supports 256k), constructing tables in a more efficient way with something like OP_SETLIST, and table pre-sizing in OP_NEWTABLE.  There might be something else I'm forgetting...


Canvas-Portal: Need to run canvas-portal inside the Canvas namespace

Description

As Canvas-portal is a part of Canvas services, so we are planning to run Canvas-portal within the Canvas namespace. Therefore, we deployed Canvas-portal inside the Canvas namespace, but it seems it hasn't established the connection with the running ODA components deployed inside the components namespace on the same cluster and is showing a blank page over Canvas-portal. Please find the screenshot below.
So, we need to modify the Canvas-portal code to run within the Canvas namespace. It would also be nice to have the connection string configurable via application.properties or environment variables.
https://github.com/tmforum-oda/oda-canvas/tree/master/canvas-portal
Canvas Portal deployed inside canvas namespace over Innovation Hub GKE cluster --> Portal running, but running ODA component don't show over UI.
http://<IP_ADDRESS>:8080/canvas-portal/index.html

Canvas Portal deployed inside components namespace over Innovation Hub GKE cluster --> Working as expected.
http://<IP_ADDRESS>:8080/canvas-portal/index.html

The Canvas-Portal currently defaults to using the components namespace, but given that it's a service within the Canvas platform, it would indeed be more appropriate to use the Canvas namespace. I agree with your suggestion. Let's proceed with creating a Pull Request to transition to the Canvas namespace.


Import from previous git

This pull request imports the project from the repository that was
used for the prototype:
https://github.com/jhernand/openshift-monitoring

I don't mind importing this as-is just to get started, but I think the very next task will be to get through all the boilerplate stuff to integrate with the standard build/release tooling.

Opened https://github.com/openshift/autoheal/issues/2 to track the next step.

Thanks @ironcladlou. I have merged the pull request so that other in the team can start sending pull requests to this repo instead to the previous one. I am now starting to work on renaming the packages and putting in place the standard build scripts.
@moolitayer, @zgalor, @elad661 please send your pull requests to this repo.


Question/Suggestion: Creation of CREATE TABLE statements

Hi,
First I just want to say I really like this library, I think you've done a great job here and hacking on the code helped me improve my understanding of Cpp11 a great deal. So thanks for that! :)
One of the things I've wrote is a little helper function that creates a CREATE TABLE statement from a given table. For example if we use the following PPGEN code
SQLPP_DECLARE_TABLE(
(tab_test),
(id     , int         , SQLPP_PRIMARY_KEY)
(name   , varchar(255), SQLPP_NOT_NULL   )
(feature, int         , SQLPP_NOT_NULL   )
)
My function outputs:
CREATE TABLE tab_test(id INTEGER AUTOINCREMENT NOT NULL,name TEXT NOT NULL,feature INTEGER NOT NULL);
Currently this only supports AUTOINCREMENT, NOT NULL and basic types (Everything is INTEGER or TEXT). But obviously this is just a quick&dirty proof of concept!
I was wondering if this is something you'd like to merge into the library. If so I'll work on making it more robust.

Hi,
Some time ago I started to work on the same functionality for PPGEN, but still not finished.
The idea is that SQLPP_DECLARE_TABLE() contains a 'static const char *' member that can be used by any databases front-ends.
At the moment, the generation of the request of creating tables for sqlite3 is implemented (not completely).
Using this PPGEN code:
SQLPP_DECLARE_TABLE(
	(tab_person
		,SQLPP_ENGINE("InnoDB")
		,SQLPP_CHARACTER_SET("utf-8")
		,SQLPP_COMMENT("'tab_person' table comments")
	)
	,
	(id, int
		,SQLPP_PRIMARY_KEY
		,SQLPP_AUTO_INCREMENT
	)
	(name, varchar(255)
		,SQLPP_NOT_NULL
		,SQLPP_DEFAULT('some name')
		,SQLPP_INDEX() // auto-generated index name
	)
	(feature, int
		,SQLPP_NOT_NULL
		,SQLPP_INDEX("col_feature_index") // hardcoded index name
	)
	(age, int
	)
)

this request can be generated:
    static constexpr const char* _create_table = "BEGIN;\n"
                                                 "CREATE TABLE "
                                                 ""
                                                 "tab_person"
                                                 "("
                                                 "\n\t"
                                                 "id"
                                                 " "
                                                 "int"
                                                 " "
                                                 "PRIMARY KEY "
                                                 "AUTOINCREMENT "
                                                 "\n\t"
                                                 ","
                                                 "name"
                                                 " "
                                                 "varchar(255)"
                                                 " "
                                                 "NOT NULL "
                                                 "DEFAULT "
                                                 "'some name'"
                                                 "\n\t"
                                                 ","
                                                 "feature"
                                                 " "
                                                 "int"
                                                 " "
                                                 "NOT NULL "
                                                 "\n\t"
                                                 ","
                                                 "age"
                                                 " "
                                                 "int"
                                                 " "
                                                 "\n);\n"
                                                 "COMMIT;";

Hi,
Glad you like the library and that it helped you with your understanding of C++11 :-)
I like the idea of generating CREATE TABLE statements using sqlpp11. It is on my wish list :-)
As of now, my goal is to

add a few more optional fields to the C++ column and table specification types, e.g. primary_key or constexpr x default_value = 3 etc.
add a create_table function to sqlpp11
provide a default interpretation in sqlpp11
provide specialized interpretations in the connector libraries

Should not be that hard, actually.
Of course, it would make sense to extend and use @niXman 's PPGEN to specify the C++ types for this, see examples/ppgen.cpp

I am very interested to this feature too... can I ask what is its status, and if there is anything that we can do to speed up development?

@niXman Maybe you could comment?
In any case, the best way to speed things up is to create a pull request :-)

I have some ideas in mind for a pull request, indeed! :)
However, I am a very new user of the library and I would therefore appreciate an headstart:

is there any way to query the fields of a table, i.e. to retrieve a list of fields without knowing them in advance? Is this an acceptable way to go, in case this feature is just "not ready yet"?
in the same way, is there a way to obtain a list of field tags (e.g. "not null", "default", etc.)
how do I translate the tags or field types to SQL commands? is there a sqlpp11 function or should I look in the connectors?
is boost.hana an acceptable dependency?

Thank you again for the useful library!

Hi,

table types contain a _column_tuple_t. This would give you the data you need. In both, table and column, _alias_t::_name_t should be used to obtain the name.
can_be_null is available already, however there is data missing like AUTOINCREMENT or a default value. These could be added to the column specifications.
I would try to go for a default implementation in sqlpp11 which could be (partially) overridden in the connector libraries
boost.hana is C++14, while sqlpp11 is C++11. Also, I try to keep dependencies to a minimum.

Cheers,
Roland

Thank you for the information. That already got me started.
@niXman Would you mind share your code for this feature, so I can take over where you left it?
Maybe @blapid can do the same?

I've had a semi-functioning proof of concept back when I posted this issue, but unfortunately did not have the time to keep working on it.
I'll post the snippets below, I hope you'll find them useful but I'll understand if you wont like them ;)
When I've worked on this, I've detached from v0.35, I hope this will still work for newer versions.
Alright, let's get into it:
The first thing I did is add type names to some data types. For example for the boolean data type, I changed the file /include/sqlpp11/data_types/boolean/data_type.h like this:
     template <typename T>
     using _is_valid_operand = is_boolean_t<T>;
+
+    static constexpr const char _literal[] =  "BOOLEAN";
+    using _name_t = make_char_sequence<sizeof(_literal), _literal>;

I've done this for bool, integer and text. Remember that these might change depending on the actual DB that we're using. So this might actually belong in the connector api? Maybe with default values in the library itself? You'd probably have to do some more template structs that takes an sqlpp11 data type as typename and hold it's own relevant name under _name_t or uses the default from the library. But I may be missing an easier solution :).
Then I had to use variadic template magic to go through all of the columns and create the CREATE query. I've used  include/sqlpp11/interpret_tuple.h as a reference, mainly copying what interpret_tuple does with slight alterations.
The full snippet:

#include <tuple>
#include <sqlpp11/type_traits.h>
#include <sqlpp11/serialize.h>
#include <sqlpp11/detail/index_sequence.h>
#include <sqlpp11/container/connection.h>
#include <stdio.h>
namespace sqlpp_test
{
  template <typename Element, typename Separator, typename Context, typename UseBraces>
  static void interpret_tuple_element(
      const Element& element, const Separator& separator, Context& context, const UseBraces&, size_t index)
  {
    if (index)
      context << separator;
     /*
    if (UseBraces::value)
      serialize_operand(element, context);
    else
      serialize(element, context);
     */
     context <<  Element::_alias_t::_name_t::char_ptr() << " " << Element::_traits::_value_type::_name_t::char_ptr();
     if (sqlpp::detail::is_superset_of<typename Element::_traits::_tags, sqlpp::make_traits<sqlpp::tag::must_not_insert,
         typename sqlpp::tag::must_not_update>::_tags>::value)
     {
         context << " AUTOINCREMENT";
     }
     if (!sqlpp::detail::is_element_of<typename sqlpp::tag::can_be_null, typename Element::_traits::_tags>::value)
     {
         context << " NOT NULL";
     }

  }

  template <typename Tuple, typename Separator, typename Context, typename UseBraces, size_t... Is>
  auto interpret_tuple_impl(const Tuple& t,
                            const Separator& separator,
                            Context& context,
                            const UseBraces& useBraces,
                            const sqlpp::detail::index_sequence<Is...>&) -> Context &
  {
    // Note: A braced-init-list does guarantee the order of evaluation according to 12.6.1 [class.explicit.init]
    // paragraph 2 and 8.5.4 [dcl.init.list] paragraph 4.
    // See for example: "http://en.cppreference.com/w/cpp/utility/integer_sequence"
    // See also: "http://stackoverflow.com/questions/6245735/pretty-print-stdtuple/6245777#6245777"
    // Beware of gcc-bug: "http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51253", otherwise an empty swallow struct could
    // be used
    using swallow = int[];
    (void)swallow{0,  // workaround against -Wpedantic GCC warning "zero-size array 'int [0]'"
                  (sqlpp_test::interpret_tuple_element(std::get<Is>(t), separator, context, useBraces, Is), 0)...};
    return context;
  }

  template <typename Tuple, typename Separator, typename Context>
  auto interpret_tuple(const Tuple& t, const Separator& separator, Context& context) -> Context &
  {
    return sqlpp_test::interpret_tuple_impl(t, separator, context, std::true_type{},
                                sqlpp::detail::make_index_sequence<std::tuple_size<Tuple>::value>{});
  }

  template <typename Tuple, typename Separator, typename Context>
  auto interpret_tuple_without_braces(const Tuple& t, const Separator& separator, Context& context) -> Context &
  {
    return sqlpp_test::interpret_tuple_impl(t, separator, context, std::false_type{},
                                sqlpp::detail::make_index_sequence<std::tuple_size<Tuple>::value>{});
  }
}

The main changes being:
     ...
     context <<  Element::_alias_t::_name_t::char_ptr() << " " << Element::_traits::_value_type::_name_t::char_ptr();
     if (sqlpp::detail::is_superset_of<typename Element::_traits::_tags, sqlpp::make_traits<sqlpp::tag::must_not_insert,
         typename sqlpp::tag::must_not_update>::_tags>::value)
     {
         context << " AUTOINCREMENT";
     }
     if (!sqlpp::detail::is_element_of<typename sqlpp::tag::can_be_null, typename Element::_traits::_tags>::value)
     {
         context << " NOT NULL";
     }
     ...

The first line adds the name of the column, followed by its type name (that we've added above).
The second part checks that the columns traits have must_not_insert and must_not_update (iirc), and if so, adds "AUTOINCREMENT".
The third part checks can_be_null as @rbock mentioned.
This may be used as following:
#include <sqlpp11/sqlpp11.h>
#include <sqlpp11/ppgen.h>

SQLPP_DECLARE_TABLE(
	(tab_test)
	,
	(id     , int         , SQLPP_PRIMARY_KEY)
	(name   , varchar(255), SQLPP_NOT_NULL   )
	(feature, int         , SQLPP_NOT_NULL   )
)

int foo(int, char* [])
{
  sqlpp::container::context_t context;

  context << "CREATE TABLE " << tab_test::tab_test::_alias_t::_name_t::char_ptr() << "(";
  sqlpp_test::interpret_tuple(tab_test::tab_test::_column_tuple_t{}, ',', context);
  context << ");";
  printf("%s\n",context.str().c_str());
}

First, we declare the table using PPGEN, however, any table would work assuming it has the right traits.
Then, in our function, we start the CREATE query with the table's name, then use interpret_tuple to add the rest of the information.
Finally, we close the query and printf.
FIN
As I've said, proof of concept, but I think it's possible to integrate it in if you have the time. As @rbock said, default value (and probably other stuff that we're missing) need to be added to column specification or to the library.
Hope this helps!

@blapid, @rbock, @matpen, I'm sorry, but I will not be able to reply about subj until the end of the week.

Thank you @blapid for your snippets: my plan is to get them to work, and fork a branch where we can share progress. I would like to ask @rbock about his opinion on the type names: storing them in the data types (data_types.h) seems logical, but maybe it is harder to override in the connectors? Should we centralize all types in a separate function?
@niXman Thank you for posting: do not worry, I have limited free time at the moment, too, so I won't be doing much before you find the type to reply.

@matpen I would use the serialize function for serializing data types. Then you can use specialization in the connectors to override where necessary (just as with everything else).

Currently sqlpp11 supports the data type date (via sqlpp::day_point), datetime (via sqlpp::time_point). I would like to ask if there is any corresponding update on the files under sqlpp11/ppgen/colops ?

Unfortunately I did not have time to work on those files. So no, there is no change, at least from my side.

@choiip done: https://github.com/rbock/sqlpp11/pull/185

I assume since it has been a year since the last comment this feature never made it to the light of day. How do you recommend initializing a database system at the moment? Running the queries from the cmd line manually via a file input?
Is there a way in the C++ lib to detect if the tables do not already exist to trigger this cmd line functionality?

I am adding this to the wish list I am currently compiling.

Any update on this feature?

No update for sqlpp11. Admittedly, I am hoping for someone who actually has a use case (I don't) :-)
I wrote a prototype in sqlpp17. I won't finish that library, but you could try to migrate that code to sqlpp11.

@rbock I'm working at a different level, using LLVM (clang, libclang, libtooling) to synchronise [bidirectionally] OpenAPI with the code. I've done the same in Python, and WiP for Rust, Swift, Java, and Kotlin.
I am currently weighing up ODB, Oat++, SOCI, cppdb, and sqlpp11 for the ORM layer, and will decide what to use for the REST API layer once that first decision is made (as some of these were developed as part of a REST API ++ framework).
With my compiler driven approach, I expect people to be able to rapidly produce REST API + db backends with full test and doc coverage, and be able to immediately translate the interfaces across language boundaries (e.g., to frontends, or to rewrite a slow-performing Python backend in C++ or Rust).
Currently focussing on the non-Turing-complete aspect of this, translating just the interfaces, tests, docs, defaults, and types.
Happy to beta test, but I've got this more immediate problem and I don't want to get distracting (unless I get into the game of actually maintaining ORMs… but doubt I have the time)

I am sharing the same predicament: Time is too scarce.
Providing interfaces to create tables via sqlpp11 is possible. But it is difficult to offer one interface across all database vendors as they all have their own specific nuts and bolts. That is why I left the table creation logic outside of the actual C++ code. Or in other words: My C++ code usually expects the database to exist before the program gets started.
I am not sure I fully understand your goal, but I think I am aiming for something similar (long term): Being able to translate fluently between objects in C++ and their representation in databases using reflection (in C++26 or so).

@rbock Happy to assist. What I need are examples inputs and outputs, say with guards or similar, for each database. Then I'll using LLVM libclang libtooling to translate a POC++C (Plain Old C++ Class) to a "database table" and back again.
But I don't know what it's meant to look like, which is why I'm seeking a ready-made framework for this…

Let me point out that you can create tables in sqlpp11 using the connection's execute method. It obviously does not have the same type safety guarantees as the rest of the library offers, but it is an option.
Personally, I won't write a create table method for sqlpp11 any time soon. There are just too many different flavors of how to create tables between different databases. IMO, the cost of creating such functions for each connector is not justified by the benefits.


Add icon

@VelocityDesign Does it work for you?

This better work or im gonna stop accepting your PRs.

your lucky, it worked

This better work or im gonna stop accepting your PRs.

Of course it works!!!

@hello-smile6 have your PRs haven't...

Hey, @hello-smile6 did you SECRETLY change npm run start?

Hey, @hello-smile6 did you SECRETLY change npm run start?

@JaydenDev I'll make a workflow that'll fail if electron-forge start is found anywhere in package.json, okay? Sorry. Making another PR to fix the start command.

@JaydenDev I'll make a workflow that'll fail if electron-forge start is found anywhere in package.json, okay? Sorry. Making another PR to fix the start command.

ok now your adding repo bloat again.
I already fixed the command
refrain from commenting on closed PRs.

@JaydenDev I'll make a workflow that'll fail if electron-forge start is found anywhere in package.json, okay? Sorry. Making another PR to fix the start command.

ok now your adding repo bloat again. I already fixed the command refrain from commenting on closed PRs.

I was just trying to help!


Implement io.edgehog.devicemanager.StorageUsage Interface in Device Runtime

See also: https://github.com/edgehog-device-manager/edgehog-astarte-interfaces

Already merged here #47 .


RPi4 Bookworm - PiShrink v0.1.5  ERROR occurred in line 261: parted is not installed.

Title says it all -
Installed newest script on fresh bookworm install on raspberry pi4. If script is run interactively using SSH or at console there is no error and it succeeds whereas using it under a script running as root, throws below error.
PiShrink v0.1.5 - https://github.com/Drewsif/PiShrink
pishrink.sh: ERROR occurred in line 261: parted is not installed.
exact command passed in script as well as on ssh/console
sudo /usr/local/bin/pishrink.sh -a -d -v -z /mnt/backup/pathtodestination/$HOSTNAME.$(date +%Y%m%d).img
OS version -
Linux masteraio 6.6.51+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.51-1+rpt3 (2024-10-08) aarch64 GNU/Linux
parted version as reported is -
parted (GNU parted) 3.5
Thank you

What is the output of sudo which parted? Its possible parted is not in root's $PATH

Here’s the output
17:27:01-pi@masteraio: ~$sudo which parted;echo "";echo $PATH
/usr/sbin/parted
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
———————
I checked 2 bullseye installs and 1 bookworm install that has same output as above for $PATH & all 3 are running on a daily schedule.
Hope this helps narrow down the issue.


Web - App works with delay, when reinvite the member to WS

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.14-0
Reproducible in staging?: y
Reproducible in production?: n
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
Action Performed:

Launch the app
Log in with any account
Go to WS and invite phone number
Reinvite the same phone number

Expected Result:
App should work without delay, when reinvite the member to WS
Actual Result:
App works with delay, when reinvite the member to WS
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence
https://github.com/Expensify/App/assets/93399543/5e67d575-8533-4a95-b41a-7735af0b2304
View all open jobs on GitHub

Issue is not reproducible in production
https://github.com/Expensify/App/assets/93399543/f34edcf5-1c40-4162-b98b-673cd1593ca9

I believe this is a regression https://github.com/Expensify/App/pull/32962, trying to verify locally now. Although it looks like it's already being reverted so I can also retest on staging after that

Retested on staging, I think we're good to close:
https://github.com/Expensify/App/assets/7277067/c128ebbd-db08-4375-a7fb-36720b123b01

This still happens for me on main with High Traffic account. Looks like it was not a regression from #32962.
https://github.com/Expensify/App/assets/13985840/12307183-97ef-4762-9aea-248dfc080c91

Lets reopen while we can confirm this was fixed on staging, seems like this is not necessarily linked to the PR which got reverted

This happens on prod as well, so I don't think this needs to block deploy

Coming back from OOO and I can't reproduce on staging or prod. Going to close again but if someone is still able to reproduce lmk!


[stable/mysql] Option to create a custom service account

What this PR does / why we need it:
Add option to create a custom service account.
Special notes for your reviewer:
It does not change chart's default behavior.
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[ ] DCO signed
[ ] Chart Version bumped
[ ] Variables are documented in the README.md
[ ] Title of the PR starts with chart name (e.g. [stable/mychartname])

/lgtm
/ok-to-test


Add DocFx metadata element

e.g. something to replace this:
var docfxMetadata =
    "---" + Environment.NewLine +
    "uid: this_is_a_tester" + Environment.NewLine +
    "---";

new MdDocFxMetadata(key, value)

This maybe needs a yaml generator class too...
e.g.
---
documentType: tester
seeAlso:
  - uid: this_is_a_uid
    name: New page
  - uid: this_is_another_uid
    name: Second page
---

Consider whether this is a new class within the md maker, or a separate project for yml..?


Forge 1.18.2 Crash with REI

Hello,
My game crashed twice today when I clicked on the recipe viewer in the Charger.  Looks like there is something conflicting with REI.  I'm attaching both logs for your review.
Using Energized Power 1.3.6
REI 8.3.588
crash-2023-02-12_17.31.39-client.txt
crash-2023-02-12_17.23.21-client.txt

I can't reproduce it. Can you please send me a list of all installed mods.

I can't reproduce it. Can you give me more details.
Does the game crash in other inventories too (e.g. Energizer)?

it seems to be caused by the REI Plugin Compatibilities mod. I still couldn't reproduce the issue, but I have found a way to disable the compatibility layer for my mod. It might be fixed in version 1.18.2-1.3.8. Please try it out and close issue if fixed.
You can download this version here: CurseForge

That fixed it. Thank you!


Document lack of support for Pod traffic shaping on Photon OS 3

Egress traffic shaping is not supported because of a missing Kernel
module (ifb), which is required by the bandwitdh CNI plugin.
This change also updates TestPodTrafficShaping to skip the test if some
of the required Kernel modules for traffic shaping are missing.
Fixes #1516

Codecov Report

Merging #1548 (24e0537) into master (9d3d10b) will decrease coverage by 22.65%.
The diff coverage is 12.56%.

@@             Coverage Diff             @@
##           master    #1548       +/-   ##
===========================================
- Coverage   63.31%   40.66%   -22.66%
===========================================
  Files         170      106       -64
  Lines       14250    12751     -1499
===========================================
- Hits         9023     5185     -3838
- Misses       4292     7098     +2806
+ Partials      935      468      -467

Flag
Coverage Δ

kind-e2e-tests
?

unit-tests
40.66% <12.56%> (-0.62%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

cmd/antrea-agent/agent.go
0.00% <0.00%> (ø)

pkg/agent/agent.go
12.08% <0.00%> (-36.63%)
:arrow_down:

.../agent/flowexporter/connections/conntrack_linux.go
28.37% <0.00%> (-0.79%)
:arrow_down:

pkg/agent/openflow/client.go
16.61% <0.00%> (-43.26%)
:arrow_down:

pkg/agent/openflow/pipeline.go
28.17% <0.00%> (-41.61%)
:arrow_down:

pkg/agent/stats/collector.go
51.13% <ø> (-46.60%)
:arrow_down:

pkg/apiserver/certificate/cacert_controller.go
10.81% <0.00%> (-45.58%)
:arrow_down:

pkg/controller/networkpolicy/mutate.go
0.00% <0.00%> (ø)

pkg/controller/networkpolicy/validate.go
0.00% <0.00%> (-44.83%)
:arrow_down:

pkg/ovs/openflow/ofctrl_action.go
15.96% <0.00%> (-45.39%)
:arrow_down:

... and 151 more

/test-e2e

/test-conformance
/test-networkpolicy


Implement script_fields for Search

Hi,
I don't see any obvious way to use script_fields and it's the only feature I am missing.
Thanks

Hello, thanks for your report. I'll look into it this week.

I've implemented both this and https://github.com/vinted/elasticsearch-dsl-rs/issues/218 and released with version 0.4.14. Please report if something is not as expected.

Awesome, thank you! I'll probably test this next week.
Cheers


Prisma Find Unique on Binary Columns

Bug description
Prisma findUnique on a unique binary column returns different result than Prisma findFirst on that column
How to reproduce

Execute a findUnique query on a Binary Column from multiple threads that will get auto-batched by Prisma.
Changing the findUnique -> findFirst fixes the problem, but both should return the same result
Expected behavior
I expect user1 === user2 in the below code snippet
Prisma information
// prisma
model User {
   id Bytes @db.Binary(16) @id

  firstName String @db.VarChar(255)
 // .. other fields
}

// ts
const someParallelyCalledFunction = (id: Buffer[]) => {
    const user1 = user.findUnique({where: { id: id }, select: {id: true} })
    const user2 = user.findFirst({where: {id: id }, select: {id: true} })
}

In this case, user1 is null but user2 is not null. Both user1 and user2 should be equal
Environment & setup

OS: MacOS
Database: MySQL
Node.js version: v18.13.0

Prisma Version
4.16.2

I can reproduce this with Prisma 4.16.2:
prisma:query SELECT `repro-current`.`User`.`id` FROM `repro-current`.`User` WHERE `repro-current`.`User`.`id` IN (?,?)
{ id: <Buffer 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61> }
{ id: <Buffer 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61> }
prisma:query SELECT `repro-current`.`User`.`id` FROM `repro-current`.`User` WHERE `repro-current`.`User`.`id` = ? LIMIT ? OFFSET ?
{ id: <Buffer 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61> }
prisma:query SELECT `repro-current`.`User`.`id` FROM `repro-current`.`User` WHERE `repro-current`.`User`.`id` = ? LIMIT ? OFFSET ?
{ id: <Buffer 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61> }

However this is not reproducible anymore with recent Prisma versions:
prisma:query SELECT `repro-current`.`User`.`id` FROM `repro-current`.`User` WHERE `repro-current`.`User`.`id` IN (?,?)
findUnique { id: <Buffer 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61> }
findUnique { id: <Buffer 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61> }
prisma:query SELECT `repro-current`.`User`.`id` FROM `repro-current`.`User` WHERE `repro-current`.`User`.`id` = ? LIMIT ? OFFSET ?
findFirst { id: <Buffer 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61> }
prisma:query SELECT `repro-current`.`User`.`id` FROM `repro-current`.`User` WHERE `repro-current`.`User`.`id` = ? LIMIT ? OFFSET ?
findFirst { id: <Buffer 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61> }

git bisect points at 3a95e6ce0974a5df4ff5976eb11e79cef189e078, so this is a bug in the old protocol.
Please update to the latest Prisma version as this is already fixed.
If for whatever reason you can't update to Prisma 5 yet, alternatively you can enable the jsonProtocol preview feature in your current version instead, but updating to the latest version is highly recommended.

Thanks Alexey! We're blocked on updating to Prisma v5 (or enabling the jsonProtocol) due to another issue with Binary columns getting auto-encoded as Base64 strings (on write but not on read), but i think that has to do with our own usage of Prisma JSON columns.
I'll keep this in mind, thanks for the quick look into this!


Gatsby v2

Will be merged when v2 is officially out.

Deploy preview for portfolio-emma ready!
Built with commit aeadd1ba96780295eb354ebbe6448c6da7b19ae4
https://deploy-preview-9--portfolio-emma.netlify.com


As a repo contributor, I want to make sure my frontend code passes checks before/after it is merged, so that we maintain a high standard of quality and consistency

Description
We should implement the best practices of continuous integration of source code, meaning performing the following checks for PRs:

Linting/formatting (eg eslnt, prettier)
Security auditing (eg npm audit)
License check (eg license-checker)
Dependency updating (eg dependabot etc)
Container-based unit testing / reporting (eg jest, junit)
Integration testing (e.g. puppeteer)
Snapshot testing (jest-snapshot, etc)

Solution
Set up minimal pipelines with a chosen CI tool such as Github Actions/Jenkins/TravisCI that implement the above checks
Describe alternatives you've considered
The alternative is to not perform these checks, which can lead to ruin.
Links to user research or other resources
N/A
Tasks

[ ] Standup basic CI/CD for commits
[ ] Add the above checks
[ ] Determine the minimal set of necessary checks out of the gate and potentially create followup issues for the rest

Definition of "Done"

[ ] Checks run on every PR
[ ] We have identified a basic set of tests that run on every PR / Commit and these are running regularly and producing results

Status update: In progress, some of these may be higher priority than others.


Use GitHub actions

The Travis CI builds aren't working, and GitHub actions are working well anyway. We gain macOS 11, but lose macOS 10.14. Seems like an okay trade-off for now as we don't specify any minimum supported version.
Closes #22

That's great! I think this is perfect.

That's great! I think this is perfect.


The pictures in XDoc posts are sometimes broken

When I go down in any debugger, images are correctly created:

Same happens for local files:

and images can be opened correctly when read again:

I also tried to revert GtFileUtility class>>#imageElementFromFileReference: and the effect is same with the following older method definition:
GtFileUtility class>>#imageElementFromFileReference: aFileReference
	"It returns a text editor if an unsupported image"
	<return: #BlElement>
	aFileReference exists ifFalse: [^ BlElement new asPannableElement].
	^ (ImageReadWriter gtReaderClassFromFileReference: aFileReference)
		ifNil: [ BrEditorElement new
			editor: (BrTextEditor new
				text: ('Unsupported format for ', aFileReference fullName asString) asRopedText);
			constraintsDo: [ :c | c vertical matchParent. c horizontal matchParent ] ]
		ifNotNil: [ :aReaderClass |
			(aReaderClass gtFormFromFileReference: aFileReference) asElement asPannableElement ]

It looks like if I scroll very slowly and always wait for a figure to be displayed, it works. If I scroll a bit faster, it starts to produce errors.
I tried it with and without Bloc Composer and I do not observer differences.

Executing the first two snippets works well:

Executing the third snippet indicates issues:

aFile := FileLocator imageDirectory asFileReference / 'image.jpg'.

aText := 'computer' asRopedText attribute: (GtResizablePreviewAttribute new
	result: aFile;
	height: 300;
	showSelector: #gtFigureFor:).

aMemory := FileSystem memory.
aMemoryFile := aMemory workingDirectory / 'image.jpg'.
aFile copyTo: aMemoryFile.
aMemoryFile.

aText := 'computer' asRopedText attribute: (GtResizablePreviewAttribute new
	result: aMemoryFile;
	height: 300;
	showSelector: #gtFigureFor:).

Trying
GtFileUtility imageElementFromFileReference: aMemoryFile.

Shows the same issue:

The image data are fine:

There is an issue with SkiaImage:

I tried the same with PNG format and it works:

On other hand, executing:
anElement := GtFileUtility imageElementFromFileReference: aMemoryFile.
aSpace := BlSpace new
	extent: 800@800;
	addChild: anElement;
	show

produces:

with:

One conclusion is that SkiaImage does not handle in-memory JPG files.

The last code snippet works with a PNG file:

This still does not explain the issue with ZIP file system. The error appears before using SkiaImage:

Image is properly displayed in a ZIP file:

Which is not surprising because re-evaluating the same debugger line does not indicate any issue:

Since the errors reported https://github.com/feenkcom/gtoolkit/issues/1332#issue-659452673 likely happen when a document is scrolled fast, I think that there there is some raise condition or memory corruption. Maybe it is not possible to read two files simultaneously from the same ZIP file?

here we have it, it is not possible to access same ZIP file simultaneously from different processes:

Complete Playground:
aFile := FileLocator imageDirectory asFileReference / 'image.png'.

aText := 'computer' asRopedText attribute: (GtResizablePreviewAttribute new
	result: aFile;
	height: 300;
	showSelector: #gtFigureFor:).

aMemory := FileSystem memory.
aMemoryFile := aMemory workingDirectory / 'image.png'.
aFile copyTo: aMemoryFile.
aMemoryFile.

GtFileUtility imageElementFromFileReference: aMemoryFile.

anElement := GtFileUtility imageElementFromFileReference: aMemoryFile.
aSpace := BlSpace new
	extent: 800@800;
	addChild: anElement;
	show.

aText := 'computer' asRopedText attribute: (GtResizablePreviewAttribute new
	result: aMemoryFile;
	height: 300;
	showSelector: #gtFigureFor:).

FileLocator localDirectory asFileReference.

aZip := FileLocator imageDirectory / 'dir.zip'.
aZipFile := (FileSystem zip: aZip) open workingDirectory / 'dir' / 'image.png'.

aText := 'computer' asRopedText attribute: (GtResizablePreviewAttribute new
	result: aZipFile;
	height: 300;
	showSelector: #gtFigureFor:).

aText := ('computer' asRopedText attribute: (GtResizablePreviewAttribute new
	result: aZipFile;
	height: 300;
	showSelector: #gtFigureFor:))
		append: (String cr, String cr) asRopedText;
		append: ('computer two' asRopedText attribute: (GtResizablePreviewAttribute new
	result: aZipFile;
	height: 300;
	showSelector: #gtFigureFor:)).

Wow, this is a beautiful debugging session!

Extracting to a temporary directory is also broken:


Change Option.None to be a static field instead of a method

Precedent

System.Collections.Immutable.ImmutableList<T>.Empty
System.Collections.Immutable.ImmutableQueue<T>.Empty

Advantages
Open for discussion
Disadvantages

This is a breaking change
Others?

We will do this for 3.0.
Open Question, what about MethodGroup-Style invocations?

We could add a None method to the non-generic, static Option class for this.


Pipeline fingerprint generation

The fingerprinting now runs sequentially with the rest of the import process.
It could be pipelined to make better use of the CPU. Multiple threads should
even be used to take advantage of multicores.

Original issue reported on code.google.com by adrian.sampson on 14 Jan 2011 at 2:51

Do not comment here, but on sampsyo/beets#126


feat: IconSwitch

Codecov Report

Merging #97 (1a00f7d) into master (f216d95) will decrease coverage by 0.42%.
The diff coverage is 6.25%.

@@            Coverage Diff             @@
##           master      #97      +/-   ##
==========================================
- Coverage   64.41%   63.99%   -0.43%
==========================================
  Files         262      264       +2
  Lines        1734     1747      +13
  Branches      323      326       +3
==========================================
+ Hits         1117     1118       +1
- Misses        616      628      +12
  Partials        1        1

Impacted Files
Coverage Δ

src/icons/Light.js
0.00% <0.00%> (ø)

src/token-panel-pro/ColorTokenContent.tsx
10.98% <0.00%> (ø)

src/IconSwitch.tsx
9.09% <9.09%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Add PSP to the deprecation guide

See also https://github.com/kubernetes/website/pull/26581
/cc @deads2k

@reylejano missed your comment before pushing this - how do you want to handle it? This information seems relevant prior to v1.21 being released, so I'd vote to keep it on the main branch.

/assign
@cc @sftim @kbhawkey @jimangel @irvifa
@tallclair I've cc'd sig-docs leads to get their input

/hold until consensus on adding this to main vs dev-1.21 branch
feel free to remove hold when a decision has been reached
cc @palnabarun

SInce PSP deprecation is targeted 1.21, please consider submit this to the dev-1.21 branch.

/milestone 1.21
/assign @tengqm
/sig auth

@annajung I was suggesting this going into dev-1.21 branch because PSP is not deprecated in 1.20 yet. Considering that the target page already contains some future predicting contents, maybe we can lift the hold on this?

Considering that the target page already contains some future predicting contents, maybe we can lift the hold on this?

I intentionally limited this page to items already announced as deprecated in a released version, which had an unequivocal "switch to use ____" call to action. Neither of those is yet true for PSP, so I wouldn't merge this to master yet.

I intentionally limited this page to items already announced as deprecated in a released version, which had an unequivocal "switch to use ____" call to action. Neither of those is yet true for PSP, so I wouldn't merge this to master yet.

@liggitt @tallclair How about submitting this PR to the dev-1.21 branch so that it merges with master on the 1.21 release date (April 8). March 31 is the "ready to merge" deadline for the dev-1.21 and the PR can be modified until then.

We published https://blog.k8s.io/2021/04/06/podsecuritypolicy-deprecation-past-present-and-future/

/lgtm

Not unholding, but I recommend that whoever approves this does also unhold it.

Hi @tallclair , what do you think of the suggestion of replacing controller with plugin . The PSP blog post is live now, what do you think of linking to the blog post. This PR can stay targeted to the master branch

I'm definitely happy to see this merge in even if there are nits. The commit in the PR right now could be polished, but is factually accurate enough to merge.


HAProxy can't start

Windows build number: 10.0.17134.285
Distro: Ubuntu 18.04.1 LTS
Installed HAProxy 1.8.13 from vbernat's ppa and here's what happened when I tried to start it:
[WARNING]: Failed to get the default value of TCP_MAXSEG
[WARNING]: Failed to get the default value of TCP_MAXSEG for IPv6
[ALERT]: Starting frontend http: cannot bind socket [<IP_ADDRESS>:80]
[ALERT]: Starting frontend https: cannot bind socket [<IP_ADDRESS>:443]

netstat -anp shows no processes listening on port 80 and 443.

You deleted the template. Note second sentence.

cannot bind socket [<IP_ADDRESS>:443]

That port is open in in Windows and thus can't be bound. I assume port 80 is too, but there is no (straightforward) way to not bind 443 in windows.

[WARNING]: Failed to get the default value of TCP_MAXSEG

An strace would show the setsockopt(2) failing, but it it looks like a soft fail. Hasn't been reported before AFAIK. Suppose this issue can be the landing zone for that.

netstat -anp shows no processes listening on port 80 and 443.

#2249. From the CMD prompt do netstat.exe -an | find.exe "443"

Hi, @therealkenc!
Somehow it works after I did a reset from apps & features.

The TCP_MAXSEG warning can probably be ignored.

Yes, you're correct. Now it works even with that warning.
Thanks for your help!


Bug application/json;charsetxxxxx  is not considered as json

https://github.com/cedric05/dothttp/blob/1293b83b291b89ffc8c4503182221c64d5527040/dothttp/request_base.py#L288

https://github.com/cedric05/dothttp/commit/68ee020e2472678d5e92890765ffe45bc29cc582


RuntimeError: Working outside of application context

Hi
I was checking web app FLASK https://github.com/patrickloeber/python-fun/tree/master/webapps/flask
But I got this error:
RuntimeError: Working outside of application context.

This typically means that you attempted to use functionality that needed
the current application. To solve this, set up an application context
with app.app_context(). See the documentation for more information.

Hi,
In this app used wrong name for SQLAlchemy object "db" which conflicts with library. Just rename it.

Hi,
Just change
db.create_all()
on
with app.app_context():     db.create_all()


Support for unregistering individual SQL drivers

In order to get access to a a particular underlying SQL driver connection object (to perform driver-specific operations) on that connection, the sqlite driver requires that you register a unique driver, such as in this example:
https://github.com/mattn/go-sqlite3/blob/4b0af852c17164dce48e6754e3094c55192e4934/backup_test.go#L27
	// This function will be called multiple times.
	// It uses sql.Register(), which requires the name parameter value to be unique.
	// There does not currently appear to be a way to unregister a registered driver, however.
	// So generate a database driver name that will likely be unique.
	var driverName = fmt.Sprintf("sqlite3_testBackup_%v_%v_%v", testRowCount, usePerPageSteps, time.Now().UnixNano())

	// The driver's connection will be needed in order to perform the backup.
	driverConns := []*SQLiteConn{}
	sql.Register(driverName, &SQLiteDriver{
		ConnectHook: func(conn *SQLiteConn) error {
			driverConns = append(driverConns, conn)
			return nil
		},
	})

Because the sql package doesn't have a function for unregistering drivers, this basically is just leaking memory.
The fakedb_test.go test in the sql package itself actually just calls a private unregisterAllDrivers() function to work around this, although this wouldn't be sufficient if you wanted to only unregister some drivers which were no longer being used.
Perhaps another solution would be to expose the driver connection(s) via some other strategy, so that this workaround wouldn't be needed.

Database drivers should begin offering and using (in tests) https://golang.org/pkg/database/sql/#OpenDB to open a driver.Connector.


updated listUUIDs() for ead reload.

listUUIDs() is expected to extract all UUIDs from an EAD ingest file for comparison with existing EAD works within a collection.  Currently, it's only extracting the first level of children from the file, and not traversing to all levels within the file.
This request provides the fix to extract all UUIDs from an EAD ingest file.

@ninhnguyen could you please review this change?  Thanks.


Add support for node 5.1.x

Add support for node 5.1.x by updating references. Set node versions between 5.1.0 and 6.0.0 as verified.

:+1: