luci-mod-notification: initial checkin

This pullrequest adds the LuCI module to display/show the notification from the system.
The required backend for the Ubus is covered in this pull request.
@hnyman  before this is accepted, and could be merge you/we have to update/add this module to weblate.
@aparcar @karlp what do you think of this?

I'd like to have this as a service showing on whatever tab is open rather in some sub menu. Specifically after login a list of things to take care of would be nice.

I'd like to have this as a service showing on whatever tab is open rather in some sub menu. Specifically after login a list of things to take care of would be nice.

Unfortunately, I'm not quite sure what you mean by service?
That's how it looks on my systems.

I see great, i didn't fully compile this in my brain. Wrongly i assumed it'd be its own page in the menu.
The implementation right now looks good!
What do you think about adding the notifications to the side rather than blocking the full view? Just like the "please set a root password" notification which is present but still allows to do other things.

@feckert I'm unsure if the message IDs are a clean implementation. It introduces "corner cases" to a single file rather than keeping it modular. Image you have a Adblock Update notification and the developer wants to add a "Update now" button. Adblock would be an optional package but to support this kind of notification it requires to be glued into notification.js.
How about having a "default" template containing title, description and close button for generic notifications, for other notifications the specific app returns a render-able object. The "Adblock update check daemon" adds a notification with the ID adblock.notification.update_available, which is a function defined luci-app-adblock/.../adblock.js. The function receives title, description and data which is then rendered and shown.
This approach allows i18n usage and even other interfaces, like a Android App or CLI to handle notifications.
I'd really like to get a comment from @jow- since he surely has more knowledge on system-design.

I think first we should define a schema of possible notification properties and ensure that these properties are output-agnostic.
Something along these lines:
{
    "title": "An adblock update is available",
    "description": "A longer description of an event, can be multiple lines and become very long, a potential UI renderer would put it behind a 'Read more...' button or similar.",
    "actions": [
      { "action": "dismiss" },
      { "action": "link", "url": "http://example.org/description/of/something" }   /* would be rendered by ui as "Read more..." button opening a new tab */
      { "action": "upgrade", "package": "adblock" }  /* would be rendered by ui as "Upgrade..." button triggering the corresponding opkg actions */
    ]
}

The same JSON could also be rendered on the command line, but instead of buttons it would render the actions as ASCII descriptions:
An adblock update is available

Longer description of an event, can be multiple lines and become very long, a potential UI
renderer would put it behind a 'Read more...' button or similar.

 - Check http://example.org/description/of/something for more information
 - To upgrade, execute "opkg upgrade adblock"

I think first we should define a schema of possible notification properties and ensure that these properties are output-agnostic.

The schema looks good to me. As a remark which I failed to empathize before, non generic actions should be possible too... or not? It's not about upgrading a regular package but performing a package specific action.
Staying with the example, the (pseudo) "upgrade" action would be adblock-updater fetch http://adblock-update.org/list.txt. This could be represented as { "action": "shell", "command": "adblock-update [...]" } (dangerous?) or calling a LuCI function adblock.update-list which follows ACLs.
For now we could also define a minimal subset and extend "external" functions later on.

Ideally the description property content would support some minimal markdown style formatting, at least to allow things like bullet lists, bold, italic, underline and maybe hyperlinks.

Sounds good, let's please use Markdown and just a minimal subset (e.g. no tables). It looks fine in pure ASCII and should be easy enough to render.

@jow @aparcar Sorry I'm not convinced if that's the right way to go.

That is then no longer generic.
The problem with the translation is then also still open. Where should we add that.
It could also be that there is no LuCI module for the generated notification.
If I can create an action array with many action, how does that looks in the LuCI?

I have imagined the following generic notification.

Interface goes up/down
Mwan3 interface is online/offline
Hard disk is full
.....

In other software areas, IDs are also used. Why not here? The IDs handling can also be realized via plugins not all IDs must be handled in the switch state. If we use IDs this would make the handling clearer.
The advantage from my point of view is that the notification creation remains simple and I don't have to think in advance about what might be needed in the LuCI. Also all LuCI things for the notification are encapsulated in the LuCI module if I set an ID. For this to work, the creators of a notification must of course use a valid ID if the notification should be displayed correctly in the LuCI.
If no ID is set in the notification, then no special handling is done for this message to get displayed in the LuCI.

{
    "title": "An adblock update is available",
    "description": "A longer description of an event, can be multiple lines and become very long, a potential UI renderer would put it behind a 'Read more...' button or similar.",
    "actions": [
      { "action": "dismiss" },
      { "action": "link", "url": "http://example.org/description/of/something" }   /* would be rendered by ui as "Read more..." button opening a new tab */
      { "action": "upgrade", "package": "adblock" }  /* would be rendered by ui as "Upgrade..." button triggering the corresponding opkg actions */
    ]
}

I'd like to also see a field priority there so a bunch of relatively unimportant adblock-update-notifications can not take away the attention from something rather critical (think: disk full). Very high priority notifications should always go on top of all other notifications and may also be visually highlighted (bold red frame or something like that).

1. That is then no longer generic.

Why not?

2. The problem with the translation is then also still open. Where should we add that.

Why? It is trivial to extract the translation strings from structured JSON files. In fact we do that already for menu JSON entries in LuCI. What kind of problem do you see here?

3. It could also be that there is no LuCI module for the generated notification.

I proposed a generic definition with a fixed set of potential properties (title, descriptions) and actions (dismiss, link, upgrade). Why would there be notification specific LuCI modules needed?

4. If I can create an action array with many action, how does that looks in the LuCI?

Row of buttons for example, like in a normal modal dialog.

I have imagined the following generic notification.
* Interface goes up/down

* Mwan3 interface is online/offline

* Hard disk is full

* .....

These would be all notifcations with a sole action dismiss:
{
  "title": "Interface \"wan\" is up",
  "actions": [ { "action": "dismiss" } ]
},
{
  "title": "MWAN3 tracked interface \"wan_b\" has been marked offline",
  "actions": [ { "action": "dismiss" } ]
},
{
  "title": "Hard disk is full",
  "description": "There's less than 5MB of space available on /. You should remove unused files and directories."
  "actions": [ { "action": "dismiss" } ]
}

In other software areas, IDs are also used. Why not here? The IDs handling can also be realized via plugins not all IDs must be handled in the switch state. If we use IDs this would make the handling clearer.

I have no problem with IDs, but I really don't want to carry dozens of "notification type plugins" for notification banners users will likely never see. You'd also need to duplicate these plugins then for each potential output channel (rendering in ui, rendering in terminal, rendering for mail sending, rendering for slack notification, ...).

The advantage from my point of view is that the notification creation remains simple and I don't have to think in advance about what might be needed in the LuCI. Also all LuCI things for the notification are encapsulated in the LuCI module if I set an ID. For this to work, the creators of a notification must of course use a valid ID if the notification should be displayed correctly in the LuCI.
If no ID is set in the notification, then no special handling is done for this message to get displayed in the LuCI.

And if LuCI is too old and has no handler for a particular ID (or notification type) then the notification is silently not shown? Or is it then handled by generic fallback code? If we need this generic fallback code then why not draft the system in a way that no type specific code is needed in the first place?

I proposed a generic definition with a fixed set of potential properties (title, descriptions) and actions (dismiss, link, upgrade). Why would there be notification specific LuCI modules needed?

Think of the example of adblock lists (ie. list data, not the package itself being updated): We will have to have actions to link to specific luci-apps to let the user perform an action there (think: click on "update lists" in luci-app-adblock) and/or have an action for generic RPC)

We will have to have actions to link to specific luci-apps to let the user perform an action there (think: click on "update lists" in luci-app-adblock) and/or have an action for generic RPC)

No. I was under the impression that a generic notification system is discussed here, not a LuCI notification system. A generic notification system has no notion of "trigger arbitrary action in arbitrary view of random ui".

Ok, cool :) Never mind the adblock use-case then.
Maybe some more ideas for things to be potentially shown as notifications:

! running in initramfs (no actions)
! rootfs mounted read-only (no actions)
! SELinux running in permissive mode (no actions)
ramoops/pstore crashlog of a previous boot available (maybe with number of records, actions: view pstore logs, dismiss/clear, could be implemented via /usr/libexec/rpcd offering ubus methods as well as luci-app-pstore offering a view using those methods)
system time unset (actions: read more about RTCs, NTP and GPS -- or redirect to LuCI system/time setting)
country not set (actions: read more... -- or redirect to LuCI wireless/device/country setting)

So to me it looks like there are system-level notifications (marked by !) without actions to be selected from, so that's easy (and that's the part I'm mostly interested in myself).
Now to the more juicy ones:
In order to be able to redirect to the system/time settings in case date/time is unset we could have the date/time settings view define that it allows reacting to the system-time-unset notification. Ie. instead of linking from the notification to specific view in LuCI, let a view declare which types of notifications it can handle.

@feckert @jow- @aparcar @dangowrt Uber mega spam... Did we found a way to implement this? Also the ubus proposal got stuck. Why? I can see many usercase where having a notification system is useful. Also this can be recycled to force a first init of the system by giving the user multiple notification to correctly setup the device.

It is not only a LuCI notification but from my point of view a system notification. This means that other systems that do not use LuCI can also genrate a notification here. The problem as far as I know was the I18n. Where should this be managed? Who is now taking care of the translation of the ubus or the LuCI?

@feckert IMHO we should not care about it... Every command and everything in shell in general is not translated and honestly if someone use console he probably know english as it's the defacto standard for IT configuration...
So we should really care about translation in LuCI. (a solution would be adding a small module for weblate for the package that will handle the description of all these notifications?)

ping on this. See linked PR in packages.


Support deconvolution.

@qingqing01 How about add this issue to a milestone, so we can group related issues and mark them with a deadline?

@wangkuiyi I will add milestones for this issue and other supports for images today.


Modified Response Body is Truncated

I have noticed a couple of related issues filed which are closed (referenced below), but there is no standing issue for a feature request/ bug for modification of a response body.
This could be helpful for a couple of reasons that I can think of.  Currently, status code modification is allowed, but if a user is unauthorized, it would be nice to provide body content as to why.  If an exception is thrown, it could also be helpful to be able to modify the body as needed.  Or if we are wrapping a request that is SOAP, for example.  We'd like to be able to make the request to the SOAP service get the response and then modify it.  More general case would be if the service responds only in xml and we want to convert it to json.
I've created a GatewayFilter that decorates the response, but the buffer is always fixed to the size of the originating response.  So if the response that I intend to send is larger, it gets truncated.
public class ModifyBodyGatewayFilterImpl implements GatewayFilter {

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {

        logger.info("\n\nexchange.getAttributes():\n {}\n\n", exchange.getAttributes());

        ServerHttpResponse response = exchange.getResponse();
        DataBufferFactory dataBufferFactory = response.bufferFactory();

        ServerHttpResponseDecorator decoratedResponse = new ServerHttpResponseDecorator(response) {

            @Override
            public Mono<Void> writeWith(Publisher<? extends DataBuffer> body) {

                Flux<? extends DataBuffer> flux = (Flux<? extends DataBuffer>) body;

                Flux<? extends DataBuffer> f = flux.flatMap( dataBuffer  -> {

                    byte[] origRespContent = new byte[dataBuffer.readableByteCount()];
                    dataBuffer.read(origRespContent);

                    System.out.println("content::: " + (new String(origRespContent)));

                    //alocating a new buffer size does not help.
                    DataBuffer b = dataBufferFactory.allocateBuffer(256);
                    b.write("0123456789abcdefg".getBytes());

                    return Flux.just(b);
                });

                return super.writeWith(f);
            }
        };

        ServerWebExchange swe = exchange.mutate().response(decoratedResponse).build();
        return chain.filter(swe);
    }
}

I've posted this question on StackOverflow:
https://stackoverflow.com/questions/48470030/spring-cloud-gateway-modified-response-body-is-truncated
Related:
#47 Allow modification of the response body
#113 How to modify request / response body?

Hey @forGitMeNotPlease
Please check my answer here. It works for me to be able to make some actions on received response(wrap it in xml)

@malinovskiy-alex Did you use this within the Spring Cloud Gateway itself as a GatewayFilter, or did you test in an isolated test?  I'm still seeing that the response is truncated with your suggestion.

@spencergibb How could we use weirdbob filter in a route declaration? It needs to run with order -2 but if added in a route declaration its order cannot be a negative number

@samtonyclarke
It should look smth like this:
...
.route(r -> r
    ...
    .filters(f ->
        ...
        .filter(yourRequestChangeBodyFilter(), Integer.MIN_VALUE)
        .filter(yourResponseChangeBodyFilter(), NettyWriteResponseFilter.WRITE_RESPONSE_FILTER_ORDER - 1)
    )
)

Some later I'm going to commit some API for request/response body modification.

@samtonyclarke  You should be able to add a route programmatically.  I am passing a bean for this filter directly.
@re6exp Do you see any issues with what I'm doing in this issue?  Any idea why the response is being truncated?
    @Bean
    public RouteLocator customRouteLocator(RouteLocatorBuilder builder,
                                           @Qualifier("modifyBodyGatewayFilter")
                                                   ModifyBodyGatewayFilterImpl modifyBodyGatewayFilterImpl) {

        RouteLocator routeLocator = builder.routes()
                .route(r -> r.path("/test")
                        .filter(modifyBodyGatewayFilterImpl, NETTY_WRITE_RESP_FILTER_ORDER - 1)
                        .uri("http://localhost:8000")
                )
                .build();

        return routeLocator;
    }

@re6exp @forGitMeNotPlease
I mean if we are declaring the filters using the text definition of the route (example: https://github.com/spring-cloud/spring-cloud-gateway/blob/master/spring-cloud-gateway-core/src/test/resources/application.yml) then how can we control the order?
Filters are ordered in the order they appear starting at 0.
In other words https://github.com/spring-cloud/spring-cloud-gateway/blob/master/spring-cloud-gateway-core/src/main/java/org/springframework/cloud/gateway/route/RouteDefinition.java doesn't support full ordering of the filters?

@samtonyclarke  Unfortunately, I don't believe that it's supported via config for the time being.  I still use config for some filters, but for this one I had to do it programmatically.

@samtonyclarke
Try to use dataBufferFactory from your wrapped response inside your writeWith method.
DataBufferFactory bufferFactory = super.bufferFactory();

See spring-web and spring-core sources, there are many code examples you can find usefull.

@re6exp using parent class' DataBufferFactory doesn't help.  Including a test that may help to replicate.
Also, see my other test above that tests the code in isolation (removed from the Gateway codebase).  That works just fine.  Seems to be an issue only when passing via ServerWebExchange.
    @Test
    public void testModifyBodyGatewayFilter() throws Exception {

        String url = "/test";

        EntityExchangeResult<byte[]> result = webTestClient
                .get()
                .uri(url)
                .exchange()
                .expectStatus().isEqualTo(HttpStatus.OK)
                .expectBody().returnResult();

        String respBod = new String(result.getResponseBody());
        System.out.println("RESPONSE BODY: " + respBod);
    }

Thanks @re6exp @forGitMeNotPlease
Given that any sort of body manipulation might take some non trivial amount of time, then I assume we want to do the processing outside of the netty event thread. Is it possible?

Yes, it can be done. Just need to have the fluxBody publish on a scheduler.
fluxBody.publishOn(Schedulers.elastic()).map(dataBuffer ->

@samtonyclarke Did you test this in the gateway?  With your suggestion I'm still seeing that the response is the same amount of bytes in the original reponse:
Original resp:  99999
New buffer   :<PHONE_NUMBER>abcdefg
Modified res : 012345

Hey @forGitMeNotPlease
If you check my solution there I needed to wrap my response in some format - add custom json node to the start and to the end. So I was able to do that using buffer() method. In that way all the chunks from response would be added to the buffer and when we get latest chunk then some event would be initiated and we can add some custom information to the start and to the end after merging list of DataBuffers. You can find example here.

@malinovskiy-alex I get the same result with that approach.

I've added a sample project demonstrating this.  It uses your approach @malinovskiy-alex.
https://github.com/forGitMeNotPlease/truncatedBodySCG

@spencergibb Any further guidance on this?  Appears that the problem is way upstream and solution not easily achievable. The http client is setting the content-length and creating the buffer to the original response size. This is in the Netty code. I don’t see a way around it without some serious modifications to the SCG codebase.
The example git project that I provide demonstrates the problem.
If anyone has any ideas on how to solve this problem it would be hugely helpful to this project.  I know it's still a relatively new code base but it's an important feature to have on any gateway.

@forGitMeNotPlease - If i understand your problem correctly, possibly you could try ByteOutputStream to collect all buffers - For example -
`
public Mono filter(ServerWebExchange exchange, GatewayFilterChain chain) {
ServerHttpResponse originalResponse = exchange.getResponse();
DataBufferFactory bufferFactory = originalResponse.bufferFactory();
ServerHttpResponseDecorator decoratedResponse =
    new ServerHttpResponseDecorator(originalResponse) {
      @Override
      public Mono<Void> writeWith(Publisher<? extends DataBuffer> body) {
        if (body instanceof Flux) {
          Flux<? extends DataBuffer> flux = (Flux<? extends DataBuffer>) body;
          return super.writeWith(
              flux.buffer()
                  .map(
                      dataBuffers -> {
                        ByteOutputStream outputStream = new ByteOutputStream();
                        dataBuffers.forEach(
                            i -> {
                              byte[] array = new byte[i.readableByteCount()];
                              i.read(array);
                              outputStream.write(array);
                            });
                        // Do whatever you want to do with outputStream.getBytes() and then

                        return bufferFactory.wrap(outputStream.getBytes());
                      }));
        }

        return super.writeWith(body); // if body is not a flux. never got there.
      }
    };
return chain.filter(
    exchange.mutate().response(decoratedResponse).build()); // replace response with decorator

}
`

There is more efficient way - using encoders/decoders and/or readers/writers from Spring itself.
By the way, I wrote filters that change request/response body, see post here.
If you are interested, please vote to show team that we need this feature.
If this feature will been voted by many people, may be, spring team will come to decision to add it. In this case I am going to continue to port last readers/writers to cover all possible situations.

@forGitMeNotPlease Has the problem been solved?

@forGitMeNotPlease Has the problem been solved?

@forGitMeNotPlease @yangdd1205  have you reolved these problem ? wait for approach.

modify the response header "Content-Length"

Gateway has a filter that modifies the response body.
https://github.com/spring-cloud/spring-cloud-gateway/blob/eb950389527b7902ae117b0ad65adbfcf2e25a86/spring-cloud-gateway-sample/src/main/java/org/springframework/cloud/gateway/sample/GatewaySampleApplication.java#L98-L107
By using the webflux encoders/decoder the content length is set automatically.

@spencergibb Hello,I come from China. I want to set Content-Length in response header，this is my code：
f.modifyResponseBody(Object.class, Object.class,(exchange, object) -> {
 f.setRequestHeader(HttpHeaders.CONTENT_LENGTH,String.valueOf(String.valueOf(object).getBytes().length));
  return Mono.just(object);
});

However, some requests will report an exception.：
{
    "timestamp": "2019-07-12T06:17:05.375+0000",
    "path": "/v1/app",
    "status": 500,
    "error": "Internal Server Error",
    "message": "Content type 'text/html;charset=utf-8' not supported for bodyType=java.lang.Object"
}

Please help me, thank you.

See https://cloud.spring.io/spring-cloud-static/Greenwich.SR2/single/spring-cloud.html#_addresponseheader_gatewayfilter_factory


Retry on watch longpoll disconnect.

Partially addresses #2422
This PR introduces a Polly resilience pipeline to retry the watch resource long poll operation when it disconnects. We detect the disconnect by overving that the exception that is thrown has an inner exception of type EndOfStreamException. If we see this then we'll log a debug message for future supportability purposes, but otherwise exit.
We swallow OperationCancelledExceptions and exit cleanly. All other exceptions are logged.
Microsoft Reviewers: Open in CodeFlow

cc @martincostello


Is the configurater actual for iOS 11.x+

Hi,
I was setting up a  vpn.mobileconfig file and sent them to my mail address.
If want import the file, also over my website I get always a error "Safari can't open and install this profile (unknown error).
If I use this https://gist.github.com/noestreich/60f4248564e4801365183b79dbfce186#file-fritzboxvpn-mobileconfig-xml it works out of the box.
Something seems to be wrong in the generated file.
Any idea?
Best Theo

It is actual for IOS13 too

Working on the latest iOS14, perhaps an error in your input?

Working on the latest iOS14, perhaps an error in your input?


[2101] General Documentation Updates

Moved glossary to separate file
Add since version in UG
REORDER UG Sections alphabetically
Add include in repeated sort args as per @teclu's comment
Fix spelling errors
Add some glossary items

Coverage remained the same at 92.056% when pulling bf13d33e3136dc663809e34a4518198775e0e011 on ZY-Ang:AlexDocumentation into d4017e08a3baf681673001fc2db51292f1b4d9ea on CS2103AUG2017-W11-B2:GeneralDocumentation.


Introside: :memo: update confirmation text

Description
closes https://github.com/navikt/aksel/issues/3298
Component Checklist 📝

[x] JSDoc
[x] Examples
[x] Documentation / Decision Records
[x] Storybook
[x] Style mappings (@navikt/core/css/config/_mappings.js)
[x] Component tokens (@navikt/core/css/tokens.json)
[x] CSS class deprecations (@navikt/aksel-stylelint/src/deprecations.ts)
[x] Exports (@navikt/core/react/src/index.ts and @navikt/core/react/package.json)
[x] New component? CSS import (@navikt/core/css/index.css)
[x] Breaking change? Update migration guide. Consider codemod.
[x] Changeset (Format: <Component>: <gitmoji?> <Text>. E.g. "Button: :sparkles: Add feature xyz.")

Need to update the changelog for affected templates as well (this data is in sanity iirc?)

It's defined in the meta.json-file located with the templates https://github.com/navikt/aksel/blob/main/aksel.nav.no/website/pages/templates/soknad-introside/meta.json


Consider adding a transactEither method

As described in https://softwaremill.com/direct-style-bootzooka-2024-update/#database-access

Yes, this would be nice.
What would the method signature look like?  And would I be assured of never seeing an unhandled exception from it? Unless I'm reading too much into this it sounds like I'd be able to get rid of a lot of exception handling code by using it, though obviously I'd still have to handle any Lefts returned.

I haven't put too much thought into it yet, but something that rolls back the transaction on a Left() like
https://github.com/softwaremill/bootzooka/blob/master/backend/src/main/scala/com/softwaremill/bootzooka/infrastructure/DB.scala#L23
  /** Runs `f` in a transaction. The transaction is commited if the result is a [[Right]], and rolled back otherwise. */
  def transactEither[E, T](f: DbTx ?=> Either[E, T])(using IO): Either[E, T] =
    try com.augustnagro.magnum.transact(dataSource)(Right(f.fold(e => throw LeftException(e), identity)))
    catch case e: LeftException[E] => Left(e.left)

So it would still throw if for example a SqlException occurred.
Would love to have your feedback / thoughts.

Personally, I like to embrace exceptions with my User-defined / recoverable error types.
My error hierarchy looks like
enum Errors extends NoStackTrace:
  case Internal
  case InvalidJwt(msg: String)
  case Etc

And I still compose / validate with Either (or a type similar to Cats Validated), but then throw if Left.
My webserver then has a root-level exception handler that catches the Errors subclasses, and produces the error message, returning 500 for all others.

cc @adamw

Ah! Sorry should have looked in the open issues :)
So there's two aspects here:

programatically deciding, if the transaction should commit or rollback. Currently this can be done with exceptions (code block completes == commit, throws == rollback), but it could also be done on a value level (code block returns Commit[T] == commit, Rollback[E] == rollback). You could use a custom Commit/Rollback ADT, or reuse Either. The first one is more readable, but isomorphic to an either, so not sure if it's worth the effort

The downside of transact(Either[E, T]): Either[E, T] is that the return type also specifies if the transaction was committed or not (left - rolled back, right - committed). You could also imagine a `transact(RollbackOrCommit[E, T]): RolledBackOrCommited[E, T], which gives you the full information in a very clear way, but that's inventing way too many new types ;).

exception handling. If I understand @AugustNagro 's approach correctly, it's also a distinction that I like to make (following Rust's philosophy) of recoverable errors & panics. Anything recoverable becomes a value-level Either, anything not - an untyped exception throw. The fine print here is that what's a panic at one level might be recoverable at another, so an exception thrown in a thread processing a request becomes recoverable when it reaches the http server (which translates it into a 500). Though I'd think of this as a feature.

Now, you might imagine having a transactSafe method, which would guarantee that no exception is thrown; it would have to return sth like Error(Throwable) / Commited(T) / RolledBack(E), making it kind of a three-way either. But since it would be a simple try-catch wrapper on top of transact (sql drivers can always throw), probably no much sense in including this OOTB.

@lbialy you might also be interested :)

oh niiice

Good argument Adam. I know my philosophy is hard-core direct style, so bear with me;

Anything recoverable becomes a value-level Either, anything not - an untyped exception throw

I find that business validations are not typically recoverable, except at (1) the immediate callsite, or (2) the root exception handler, where they are translated into, for example, HTTP error codes.
Threading Either or Try vertically through the application quickly becomes tiresome, with little benefit if we never recover in the intermediate layers. Either becomes viral like any other monad, which results in more work for service methods to call other service methods. This breaks a programmer's zen flow.
Exceptions let us bypass this issue entirely. While there's a stigma against Exceptions for control flow, extending ControlThrowable nearly eliminates the overhead (https://shipilev.net/blog/2014/exceptional-performance/). Note that I'm not arguing for removing Either entirely, but rather to localize its usage (ie, don't return Either from service methods).
Here's an example of how this would look in Bootzooka. First, Fail would become throwable:
abstract class Fail extends ControlThrowable

Then we remove Either from the signature of service methods like UserService.changePassword;
def changePassword(userId: Id[User], currentPassword: String, newPassword: String)(using DbTx): ApiKey =
  ???
  val user = either {
    validateUserPassword(userId, currentPassword).ok()
    validateNewPassword().ok()
  }.toTry.get
  updateUserPassword(user, newPassword)
  invalidateKeysAndCreateNew(user)

Finally in the controller
  private val changePasswordEndpoint = authedEndpoint.post
    .in(UserPath / "changepassword")
    .in(jsonBody[ChangePassword_IN])
    .out(jsonBody[ChangePassword_OUT])
    .handle { id => data =>
      try
        val apiKeyResult = db.transact(userService.changePassword(id, data.currentPassword, data.newPassword))
        Right(ChangePassword_OUT(apiKey.id.toString))
      catch case f: Fail => Left(f)
    }

However, I understand that this is very much a preference, and I'm not opposed to having a value-based transact method. But this is a fun debate to have.

SqlExceptions may be transient (connection interruption, etc), and therefore recoverable. However, it greatly depends on the context so I wouldn't want to punish users by returning an Either or Try.
When consuming external REST APIs it can make a lot of sense to use Try + exponential retries, but your DB should have a reliable connection to your application; therefore I argue that it's better to fail fast on a connection issue.
The user can always retry again in the very rare cases of transiency, meanwhile retries may actually serve to mask an infrastructure issue. One exception is serialization anomalies, which might make sense to retry in the application. However that can be handled explicitly with a library like Ox.

Ha yes, error handling is an eternal and eternally unsolved problem ;). No approach is perfect: checked exceptions (for many reasons), unchecked exceptions (unreliable, "lying" method signatures), Eithers.
But, to provide some counter-points:

using Either in the result type is introducing yet another monad, however a monad which is easier to work with: you can pattern match on it (unlike IO), fold locally etc. In a way, it's "direct", vs. other asynchronous monads
boundary-break for eithers further helps when using direct-style, and unlike other "direct" macros for IO/ZIO, this works well with lambdas (with a slight footnote, that to be bullet-proof, you need capture checking)
invalid password / invalid new username etc. is in some ways a different error than an SQLException. First, it's a user error (4xx in HTTP terms), vs. an infrastructure/server error (5xx). Secondly, you can provide meaningful information back to the user, that is formulate a specific response, vs. just returning an "internal server error" and logging the stack trace. So I think it might make sense to treat them differently. But I agree that errors are rarely recoverable, at the level of an individual request.


cert-manager can't renew cert for rancher

What kind of request is this (question/bug/enhancement/feature request):
Cert-manager can't renew the cert for rancher. It's a bug. But i have no clue what's the cause.
Steps to reproduce (least amount of steps as possible):
Install rancher according to manual and wait until the cert is about to expire (30 days before expiry date).
Result:
cert-manager can't renew cert.
Error-Message:
E0902 12:08:54.544002 1 controller.go:180] certificates controller: Re-queuing item "cattle-system/tls-rancher-ingress" due to error processing: error creating x509 certificate: x509: only RSA and ECDSA public keys supported
Other details that may be helpful:
rancher v2.2.7
cert-manager v0.5.2 (recommended in manual)
kubernetes v1.12.10
I appreciate any help to find the issue

I also encountered the same problem.
I was able to renew cert by doing helm upgrade.
helm upgrade rancher rancher-latest/rancher --version 2.2.8 \
  --set hostname=rancher-hostname \
  --set ingress.tls.source=letsEncrypt \
  --set<EMAIL_ADDRESS>


Add Password expire date for OIDC (possibly in hybrid environments)

We are in a hybrid identity situation with EntraID, so should the account also get password expiry info if there is line of sight to the On-Prem domain?
From here (https://learn.microsoft.com/en-us/entra/identity-platform/optional-claims-reference#v20-specific-optional-claims-set) the password expiry value looks to be pwd_exp in Entra ID, at least.

https://github.com/twocanoes/xcreds/issues/165

dup. fixed and closing.


[CP - 3.14] Fix Xcode 15 build failure due to DT_TOOLCHAIN_DIR

Issue Link
https://github.com/flutter/flutter/issues/132755
Commit Hash
b52297d
Target
beta
PR Link
https://github.com/flutter/flutter/pull/132956
Changelog Description
Fix macOS build error with Xcode 15 and Cocoapods.
Impacted Users
All Flutter macOS developers using Xcode 15
Impact Description
Flutter cannot build macOS applications that use plugins with Xcode 15.
Workaround
Workaround would be to use an unreleased version of Cocoapods.
Risk
low
Test Coverage
yes
Validation Steps

Download and select Xcode 15
flutter create hello_world
flutter pub add path_provider
flutter run

Should not get an error.

@stuartmorgan and @christopherfujino could you take a look at this cherry-pick request?


[Meta Issue] Write tests for codebase

Summary

All functions need tests to ensure they do what we expect and to catch bugs before they get into the main branch.  The tests should cover all known cases and potential bugs.  If new bugs appear, new tests should be written to cover the new bug.
We should generally follow the suggested rules of testing given here:
https://docs.python-guide.org/writing/tests/
This issue covers everything except cli.py, which we will work on next.
Proposed implementation

This is a meta issue (aka Epic), so a new issue should be opened for each task.  Some tasks are in the list below, but other tasks can be added to this meta issue by editing this description.
There are data files in the tests directory that can be used in the tests.  Do not modify the data files.
All tests should be in the tests directory.
Each module in the repo will have a test file named tests_{module name}.py
Each test will be a function named test_{some function}.py

Acceptance Criteria

[ ] All functions and methods (except in cli.py) have tests (full coverage)

Tasks to complete

Tests should be written for each function:

[x] https://github.com/metrostar/ib_manifest_util/blob/main/ib_manifest_util/update_repository.py#L16 #18
[ ] https://github.com/metrostar/ib_manifest_util/blob/main/ib_manifest_util/create_dockerfile.py#L9
[ ] https://github.com/metrostar/ib_manifest_util/blob/main/ib_manifest_util/create_dockerfile.py#L63
[ ] https://github.com/metrostar/ib_manifest_util/blob/main/ib_manifest_util/create_hardening_manifest.py#L15
[x] https://github.com/metrostar/ib_manifest_util/blob/main/ib_manifest_util/download_conda_packages.py#L7 #6
[x] https://github.com/metrostar/ib_manifest_util/blob/main/ib_manifest_util/util.py#L18 #5
[x] https://github.com/metrostar/ib_manifest_util/blob/main/ib_manifest_util/util.py#L42
[x] https://github.com/metrostar/ib_manifest_util/blob/main/ib_manifest_util/util.py#L68 #16
[x] https://github.com/metrostar/ib_manifest_util/blob/main/ib_manifest_util/util.py#L78 #12
[x] dump_yaml function #15

Related to

Check here for coverage

Check here for coverage


SMT build error

This SMT build faulted in ext-QA.
\x1b[41m\x1b[30mfail\x1b[39m\x1b[22m\x1b[49m: Grpc.AspNetCore.Server.ServerCallHandler[6]
      Error when executing service method 'GetWordGraph'.
      System.Threading.Tasks.TaskCanceledException: A task was canceled.
         at SIL.Machine.Utils.ObjectPool`1.GetAsync(CancellationToken cancellationToken) in /app/src/SIL.Machine/Utils/ObjectPool.cs:line 56
         at SIL.Machine.Translation.Thot.ThotSmtModel.GetWordGraphAsync(String segment, CancellationToken cancellationToken) in /app/src/SIL.Machine.Translation.Thot/ThotSmtModel.cs:line 120
         at SIL.Machine.Translation.HybridTranslationEngine.GetWordGraphAsync(String segment, CancellationToken cancellationToken) in /app/src/SIL.Machine/Translation/HybridTranslationEngine.cs:line 208
         at SIL.Machine.AspNetCore.Services.SmtTransferEngineService.GetWordGraphAsync(String engineId, String segment, CancellationToken cancellationToken) in /app/src/SIL.Machine.AspNetCore/Services/SmtTransferEngineService.cs:line 127

\x1b[41m\x1b[30mfail\x1b[39m\x1b[22m\x1b[49m: SIL.Machine.AspNetCore.Services.ClearMLMonitorService[0]
      Build faulted (6545d4793627520aa516b7eb). Error: ValueError : Task object can only be updated if created or in_progress [status=stopped fields=['hyperparams']]

What is going on here?

I think this is a duplicate (in the first part) of https://github.com/sillsdev/serval/issues/81. Not sure why that would trigger an issue with the ClearMLMonitorService since it should only deal with NMT jobs on ClearML.

They may be separate issues that happened at the same time.  If the first issue was fixed in late September, why is it still happening?

OK, I see. I'll investigate the ValueError. As for the previous fix, I believe the solution was to filter exceptions on the Serval side. Would you like them to be filtered on machine as well? That might be a bit trickier.

In general, Serval should not log a "fail" for any combination of user inputs.  A "fail" means that something unexpected happened (and the total range of user inputs is by definition, expected).  It may be worthwhile to change the machine to log an "info" in this case, if it is possible.

In general, Serval should not log a "fail" for any combination of user inputs. A "fail" means that something unexpected happened (and the total range of user inputs is by definition, expected). It may be worthwhile to change the machine to log an "info" in this case, if it is possible.

This is a log from Machine though, right? But yes, I think I can manage to change this to an info log.
As for the ValueError, because of the nature of the error and the fact that it's something in machine.py, @ddaspit may be the best person to address it. I imagine it has something to do with the changes to the code that allow for the posting of updates to the build progress.

It looks like we are trying to use the ClearML task after it was closed.

We should probably spin off the second error into a separate issue.

I've spun off a separate issue. I'll address the logging concern.


x/tools/gopls: high cpu when call go mod tidy

Go version
go version go1.22.8 windows/amd64
Output of go env in your module/workspace:
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\user\AppData\Local\go-build
set GOENV=C:\Users\user\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\user\coding\golang\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\user\coding\golang
set GOPRIVATE=
set GOPROXY=https://goproxy.cn
set GOROOT=C:\Users\user\.version-fox\cache\golang\current
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLCHAIN=auto
set GOTOOLDIR=C:\Users\user\.version-fox\cache\golang\current\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.22.8
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Users\user\coding\golang\src\yaklang\go.mod
set GOWORK=C:\Users\user\coding\golang\src\yaklang\go.work
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=C:\Users\user\AppData\Local\Temp\go-build78613495=/tmp/go-build -gno-record-gcc-switches

What did you do?
When I plan to write code normally and use some libraries from the current file, gopls will automatically import the library for me, which will cause the CPU to soar, causing the entire gopls to be temporarily paralyzed (highlighting, completion, prompts)
What did you see happen?
Sudden CPU surge
go mod tidy -modfile=C:\Users\lang\AppData\Local\Temp\gopls-tempmod246022334\go.mod

What did you expect to see?
gopls running normally

Related Issues

x/tools/gopls: high cpu usage all the time #70172
x/tools/gopls: windows cpu use 100% #66945 (closed)
x/tools/gopls: massive cpu and memory usage #46897 (closed)
x/tools/gopls: go list causes 100% CPU load #39116 (closed)
x/tools/go/packages: high CPU usage while processing cgo file #32821
x/tools/gopls: high cpu usage when make changes in package which have some cgo package dependents #50151
x/tools/gopls: autocomplete - unstoppable import  #54296 (closed)
x/tools/gopls: "stuck" at 100% CPU #34569 (closed)
x/tools/gopls: uses too much memory (60G) #37790 (closed)
x/tools/gopls: excessive invocations of "go mod tidy" #38816 (closed)

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

extra info:
Extra information, a new computer purchased a day ago
CPU: AMD Ryzen Al 9 HX 370

I found that go list also cause the CPU to soar..

Thank you. This is almost certainly related to go list: when you add an import we run go list to list package information.
Does this occur for even a small module? If so, then it may indicate some real bug in go list (for example, mismatching paths that cause spurious cache misses).

Thank you. This is almost certainly related to go list: when you add an import we run go list to list package information.
Does this occur for even a small module? If so, then it may indicate some real bug in go list (for example, mismatching paths that cause spurious cache misses).

Without further testing, I have moved to WSL to work on it, and while there is no evidence, I suspect Windows Defender is involved


moduledoc support

Maybe I have misconfigured something but it doesn't seem like the plugin knows about @moduledoc.

What is more, if I type everything manually it completes extra " making me having to always delete " on the first and last lines. This is very frustrating.

For me this is the most annoying bug with this otherwise excellent extension because it is actually counterproductive (typing it out plain text is better than having to delete, move cursor, delete again).

This should propably be a snippet.

This has been added as a snippet


cast http requet data

Hello,
I wonder if you have a mechanism to automatically cast data from HTTP request.
I did my own, parsing the doc comment of  generated classes by Sailor, but maybe you did it yourself better than me.
Many thanks in advance.
Eric

What data type are you want to cast?

Hello @vuongxuongminh
"data from HTTP request", i mean data that comes from a form.
I wish you a nice day.


Expanded Calendar View

I am adding points as the calendar scrolls to another month. However, the calendar doesn't refresh and show those new points unless it's closed an open again. Is there a way around this?

@mBigFlower
Please help to verify.
I think it's just missing a refresh call, right ?

Yes

@mBigFlower Is there a method that will refresh it?
@SpongeBobSun I know you've asked before but my hope is to push this to beta next week.

refreshDrawableState() ?

Hmm let me check. Seems like my buddy is taking a rest.
Get Outlook for iOShttps://aka.ms/o0ukef
On Sat, Nov 12, 2016 at 11:25 PM +0800, "Jason H Caruso" <notifications<EMAIL_ADDRESS>wrote:
refreshDrawableState() ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/SpongeBobSun/mCalendarView/issues/34#issuecomment-260128228, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALABrgASO9REL77G2EK-8u1c1MWZTu2nks5q9dptgaJpZM4KwCSW.

Have you tried MarkedDates.notifyObservers() ? As I recall we've registered our adapters as observers.

@SpongeBobSThat appears to have done it I'll test more to verify.

@SpongeBobSun  We're live!

@jcaruso001 WOW, Congrats!


[Autocomplete] Add equivalent to openOnFocus for click events

[x] I have searched the issues of this repository and believe that this is not a duplicate.

Summary 💡
openOnFocus does not affect what happens when an out-of-focus input is clicked. In the comments of #20286, it was argued that this is by design. As such, I hereby request an equivalent (suggested: disableOpenOnClick) prop that disables opening the suggestions on click.
Motivation 🔦
I had to add this to the embedded TextField
          [...]
          onMouseDownCapture={ (e) => e.stopPropagation()}
          [...]

This sounds like a good idea. What's your use case for this behavior?

Should this rather be a single prop that handles both? Otherwise we should specify use cases for all combinations:

openOnFocus={false} openOnClick={false}
openOnFocus={false} openOnClick={true}
openOnFocus={true} openOnClick={false}
openOnFocus={true} openOnClick={true}

@eps1lon These combinations look valid.
I have updated the proposal to disableOpenOnClick to match our naming convention. react-select has a openMenuOnClick prop, it was reported early on: https://github.com/JedWatson/react-select/issues/1131.

These combinations look valid.

It's trivial that these are valid. My point is what the use case for each of these combinations would be.

open on focus was asked to prevent masking the UI when quickly tabbing between the fields
open on click was asked to delay XHR requests. On https://next.material-ui.com/components/autocomplete/#google-maps-place you don't want to open the popup and display "no options" we will use the prop there.

open on focus was asked to prevent masking the UI when quickly tabbing between form fields.
open on click was asked to delay XHR requests.

These are the same?

I think that we could bundle them under a single prop if the expected default behavior was identical for the two events. It's not the case. By default, we respond to click but not focus events.

think that we could bundle them under a single prop if the expected default behavior was identical for the two events.

This is what I've been saying. Props, like components, should be declarative.

By default, the Autocomplete responds to click but not focus events.

Why though? I don't see the distinction since depending on the user these are somewhat equivalent interactions.

@eps1lon Initially, the behavior of the Autocomplete was openOnFocus={true} (no click logic), which developers complained about: #18815. In https://github.com/mui-org/material-ui/issues/18815#issuecomment-569277122 you made a great point around how Chrome's autofill/autocomplete behaves by default. We went on reproducing it. Later on, developers complained about the open on click logic to not be complete, missing the case for handling already selected values: #20730.
Does it answer the question?

Hi I'd like to try working on this issue :)

@oliviertassinari Hey mind if i pick this issue up?

@RiyaNegi Feel free to work on it. I have reread the thread, it seems to still be relevant and bring value.

@oliviertassinari So just to be clear, we want a disableOpenOnClick prop to disable dropdown of options on a mouse click event. Rather it would open when the user types something in the Autocomplete input box.

@RiyaNegi Correct

@oliviertassinari Hey so I added the disableOpenOnClick prop and it is working for every use case as far as I know. To tackle the use case where both openOnFocus and disableOpenOnClick are true, I created a state flag called isClicked which is set to true when onMouseDown is triggered. This was needed because the onFocus event needs something to differentiate between an event triggered by key press or by a mouse press.
This was my solution, however if you have any suggestions for better implementation, do let me know. :)

It seems to me the behavior requested can already be implemented in a controlled Autocomplete with
open={open}
onOpen={(event) => {
  if (event.type !== "mousedown") {
    setOpen(true);
  }
}}
onClose={() => setOpen(false)}

-- https://codesandbox.io/s/autocomplete-disableopenonclick-g0bt3?file=/src/Demo.tsx
If that is the case, then we shouldn't bloat the interface and implementation for such a niche feature.

@eps1lon @oliviertassinari Hey so I checked out your solution and it works, but it blocks the dropdown completely( doesn't open on either mouse click or focus). Also, the bigger issue is if you add openOnFocus prop to your solution, it opens the dropdown on both mouse click and keyboard click.
I solved this particular problem in the library with the introduction of disableOpenOnClick.
--https://codesandbox.io/s/autocomplete-disableopenonclick-forked-g43i1?file=/src/Demo.tsx  (demo with openOnFocus prop)

@eps1lon @oliviertassinari Hey so I checked out your solution and it works, but it blocks the dropdown completely( doesn't open on either mouse click or focus). Also, the bigger issue is if you add openOnFocus prop to your solution, it opens the dropdown on both mouse click and keyboard click.
I solved this particular problem in the library with the introduction of disableOpenOnClick.
--codesandbox.io/s/autocomplete-disableopenonclick-forked-g43i1?file=/src/Demo.tsx (demo with openOnFocus prop)

Fixed with
-if (event.type !== "mousedown") {
+if (event.type !== "mousedown" && event.type !== 'focus') {
  (true);
}

@eps1lon @oliviertassinari  I added your suggestion to the demo but it's not working. The dropdown doesn't open regardless of mouse click or focus.
--https://codesandbox.io/s/autocomplete-disableopenonclick-forked-3hsfs?file=/src/Demo.tsx
I know it's a niche case to be covering (disableOpenOnClick and openOnFocus together), but I guess someone might need it?

The dropdown doesn't open regardless of mouse click or focus.

But that was the point of the codesandbox. People were asking for disableOpenOnClick which I implemented in the codesandbox.

I think there is slight misunderstanding. The plan is to cover a case where both(disableOpenOnClick and openOnFocus) are used together right?
Maybe I am missing something, but currently in the examples you are giving, only one use case is getting covered.
if the following is added along with openOnFocus prop, it doesn't work well.
if (event.type !== "mousedown") {
 (true);
}

this is because when openOnFocus is passed, it always sets setOpen(true). So adding any external code along with openOnFocus, won't make a difference.

A reminder that this issue is still not clearly outlined.
Currently it's ignored that a click causes a focus and how a conflict of openOnFocu and disableOpenOnClick should be resolved.
There is not "intuitive" solution for this so we need to define clearer what the expected behavior is and how an API for that might look like. It sounds like we want to further split between keyboard-focus and click-focus.

Use case for disable onClickFocus: you have freeSolo mode and user wants to type a new value, in a form the popup does cover the form inputs beneath and so the user has to click-away the popup


Some search strategies get stuck

Describe the issue:
I'm trying to run NNI NAS to optimize a neural architecture, but I'm encountering an issue where the program gets stuck after all trials have been completed or the max duration has been reached (I have tried both). Specifically, the program appears to be running indefinitely without providing any further output or results. Note that the experiment reaches the DONE status. I've tried running the program with different configurations and strategies. PolicyBasedRL and RegularizedEvolution seem to be the ones where this issue arises. Is this related to #5202? Is there any workaround for this problem? I have tried creating a thread to stop the experiment after the max duration had been reached and although the experiment did end, the program kept running indefinitely.
Environment:

NNI version: stable
Training service (local|remote|pai|aml|etc): local
Client OS: Windows 10
Python version: 3.10.9
PyTorch/TensorFlow version: PyTorch 1.13.1
Is conda/virtualenv/venv used?: conda
Is running in Docker?: no

Configuration:

Experiment config (remember to remove secrets!):

{
  "params": {
    "experimentType": "nas",
    "trialCommand": "C:\\Users\\Leonardo\\anaconda3\\envs\\omnia\\python.exe -m nni.retiarii.trial_entry py",
    "trialCodeDirectory": "C:\\Users\\Leonardo\\Documents\\Universidade Leo\\5º ano\\tese\\Omnia\\omnia-deep-learning\\tests",
    "trialConcurrency": 1,
    "trialGpuNumber": 1,
    "maxExperimentDuration": "180s",
    "useAnnotation": false,
    "debug": false,
    "logLevel": "info",
    "experimentWorkingDirectory": "C:\\Users\\Leonardo\\nni-experiments",
    "trainingService": {
      "platform": "local",
      "trialCommand": "C:\\Users\\Leonardo\\anaconda3\\envs\\omnia\\python.exe -m nni.retiarii.trial_entry py",
      "trialCodeDirectory": "C:\\Users\\Leonardo\\Documents\\Universidade Leo\\5º ano\\tese\\Omnia\\omnia-deep-learning\\tests",
      "trialGpuNumber": 1,
      "debug": false,
      "useActiveGpu": true,
      "maxTrialNumberPerGpu": 1,
      "reuseMode": false
    },
    "executionEngine": {
      "name": "py"
    }
  },
  "execDuration": "3m 5s",
  "nextSequenceId": 100,
  "revision": 24
}

Search space:

{
  "hidden_dim1": {
    "_type": "choice",
    "_value": [16, 32, 64, 128, 256]
  },
  "dropout1": {
    "_type": "choice",
    "_value": [0, 0.25, 0.5]
  },
  "model_1": {
    "_type": "choice",
    "_value": [0, 1]
  },
  "hidden_dim2": {
    "_type": "choice",
    "_value": [16, 32, 64, 128, 256]
  },
  "dropout2": {
    "_type": "choice",
    "_value": [0, 0.25, 0.5]
  },
  "model_2": {
    "_type": "choice",
    "_value": [0, 1]
  },
  "block_iter": {
    "_type": "choice",
    "_value": [1, 2, 3, 4, 5]
  },
  "block_values0": {
    "_type": "choice",
    "_value": [16, 32, 64, 128, 256]
  },
  "block_dropout0": {
    "_type": "choice",
    "_value": [0, 0.1, 0.2]
  },
  "output_dim": {
    "_type": "choice",
    "_value": [16, 32, 64, 128, 256]
  },
  "block_values1": {
    "_type": "choice",
    "_value": [16, 32, 64, 128, 256]
  },
  "block_dropout1": {
    "_type": "choice",
    "_value": [0, 0.1, 0.2]
  },
  "block_values2": {
    "_type": "choice",
    "_value": [16, 32, 64, 128, 256]
  },
  "block_dropout2": {
    "_type": "choice",
    "_value": [0, 0.1, 0.2]
  },
  "block_values3": {
    "_type": "choice",
    "_value": [16, 32, 64, 128, 256]
  },
  "block_dropout3": {
    "_type": "choice",
    "_value": [0, 0.1, 0.2]
  },
  "block_values4": {
    "_type": "choice",
    "_value": [16, 32, 64, 128, 256]
  },
  "block_dropout4": {
    "_type": "choice",
    "_value": [0, 0.1, 0.2]
  },
  "cell/op_1_0": {
    "_type": "choice",
    "_value": ["0", "1"]
  },
  "cell/op_2_0": {
    "_type": "choice",
    "_value": ["0", "1"]
  },
  "cell/op_3_0": {
    "_type": "choice",
    "_value": ["0", "1"]
  },
  "cell/op_4_0": {
    "_type": "choice",
    "_value": ["0", "1"]
  },
  "cell/input_1_0": {
    "_type": "choice",
    "_value": [0]
  },
  "cell/input_2_0": {
    "_type": "choice",
    "_value": [0, 1]
  },
  "cell/input_3_0": {
    "_type": "choice",
    "_value": [0, 1, 2]
  },
  "cell/input_4_0": {
    "_type": "choice",
    "_value": [0, 1, 2, 3]
  }
}

Log message:

nnimanager.log:
nnimanager.log

How to reproduce it?:
My code is similar to the Hello, Nas! tutorial, with different data, model space, and using the PyTorch Lightning Regression evaluator. I can provide my code if needed.

I think the latest refactored version will no longer have those problems. You can try the preview here: https://github.com/ultmaster/nni/tree/nas-nn-refactor
Currently a workaround is to decrease max_trial_number or increase algo-specific budget (e.g., max_collect).

Thank you for the feedback!
And how exactly can I kill an experiment?
Right now what I have is:
def start_experiment():
    exp = RetiariiExperiment(model_space, evaluator, [], search_strategy)
    exp_config = RetiariiExeConfig('local')
    exp_config.experiment_name = 'nas_nni_dnn'
    exp_config.trial_gpu_number = 1
    exp_config.training_service.use_active_gpu = True

    exp_config.max_experiment_duration = str(MAX_EXP_DURATION) + "s"
    exp_config.trial_concurrency = 1

    def get_status():
        while True:
            time.sleep(20)
            if exp.get_status() == 'DONE':
                exp.stop()
                break

    thread = threading.Thread(target=get_status)
    thread.start()

    exp.run(exp_config, 8081)
    thread.join()
    return exp

However, this isn't working because exp.stop() only pauses the experiment...

What you need is a way to interrupt the main thread.

That does fix the issue, except for the PolicyBasedRL strategy. It seems it is ignoring the SIGINT. Any ideas on how to avoid this using this threading approach?
Also, any idea when that refactored version might be released?

RL strategy creates new threads internally. That's why this approach is not working.
I think the new release will be available sometime between April and May.

I'm closing the issue since it is related to #5202, and there was a workaround for the most part.
I appreciate the help and the info!


docs: fix provider type in injectable service snippet

Information

Type
Breaking change

Doc
No

Description
Correct the provider type to ProviderType.SERVICE in the sample snippet of how to use @Injectable for services. The previous version had ProviderType.CONTROLLER and would throw an INJECTION_ERROR with TypeError: provider.hasParent is not a function.

:tada: This PR is included in version 6.10.4 :tada:
The release is available on:

v6.10.4
GitHub release

Your semantic-release bot :package::rocket:


Thumbnail zoom

The thumbnail zoom code could do with some improvements IMO.  Today there are 3 behaviors (I think) when you click on an attachment thumbnail

do nothing since the image is already full size, ie they attached a thumbnail
zoom it up in the same window if it fits full size in the current viewport or window size.  In so doing it moves the other unzoomed thumbnails around as needed, messing up the view
open a new window or tab with the image, if it was to large to zoom in place

And of course you can get all three behaviors on a single post.
I'd propose we use a "poor mans" lighbox to zoom all of them the same.  Dropping the 1 and 3 behaviors all together.

Maybe: https://github.com/elkarte/Elkarte/commit/8d9d552053faf98c384e2e68c76333f60a6ebb31

Is this patch already in or is in a branch?

its in a branch in my repo (linked above) ... but that only covers some of the behavior (meaning the sources are still deciding a few things like new window, zoom or do nothing) so I still need to look at that a bit.

Ohhh!
Okay, got it!

So it was #2292 ?

Yep


ERROR Error("data did not match any variant of untagged enum LambdaRequest")

I was going to try to write a lambda using rust and found this and i have had a really hard time with getting lambdas to work and how to develop a lambda locally.
i dont know if im stupid or if lambdas are very convoluted, complicated and strange.
when running:
cargo lambda new test-lambda

i answer yes, and then HTTP REST. i then cd into the folder test-lambda
and run:
cargo lambda watch

and then:
cargo lambda invoke --data-ascii '{ "command": "hi" }'

and what i get back is
ERROR Error("data did not match any variant of untagged enum LambdaRequest", line: 0, column: 0)
[Finished running. Exit status: 0]

And the entire lambda crashes and i need to restart it.
i also try to use curl:
curl http://localhost:9000/lambda-url/test-lambda

and i get nothing back.
What am i missing?

Http lambas run behind one of many AWS services that do the actual http handling. Those services package up all the http info and some AWS context as a JSON object which is what your lambda is expecting.
Here's an example.

okey.
I have switched from using cargo lambda to using aws sam instead as the documentation was a lot clearer for my particular usercase (http -> lambda -> dynamoDB).
The example you posted is just a random json file without any documentation, information about properties, any use case or any explanation.


Ads: expiration is reset upon ad editing

if you go in and edit your ad it extends the life of the ad

Actually it maybe useful to reset expiration upon editing, so it maybe option for this is needed.


MeshLoader inconsistency on import  switching

Hello,
Depending on the order of import, instances of MeshLoader have differents methods. To reproduce, you can do:
code 1:
import hppfcl
import pinocchio as pin
meshloader = hppfcl.MeshLoader()
print(dir(meshloader))

result:
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'create']

code 2:
import pinocchio as pin
import hppfcl
meshloader = hppfcl.MeshLoader()
print(dir(meshloader))

result:
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'load']

Notice the create method in first case and the load method in the second case. This bug seens related to the interaction between pinocchio and hppfcl, it seems appropriate to also tag @jmirabel .

I encountered this some time ago and I believe I solved it. Which version do you use ?

#1084 should have solved this.

I am using (pinocchio 2.3.1)[https://anaconda.org/conda-forge/pinocchio] and (hpp-fcl 1.3.0) [https://anaconda.org/conda-forge/hpp-fcl] which are the latest versions available on conda-forge.

I checked and there hasn't been any release since the fix. The bindings you should use are the one from hppfcl so you should import pinocchio before hppfcl.

Yes, that is what I've been doing for a while. I checked today if things have changed and as it has not I opened this issue so that there is a trace of this problem somewhere.

As mentioned by @jmirabel, this issue has been solved in #1084. I will close it as no action is no more required.

As there is no release, this is still a problem you will run into with the available packages. I opened it because I was not the only one to run into the problem.


Improve export/import with multiple formats: GRAPHML, GRAPHJSON and SQL

We'd need a more powerful export/import that support format. Right now we use a OrientDB proprietary JSON format, but it would be nice supporting:

SQL: this would be similar to RDBMS, where re-running the export.sql would produce an identical database
GRAPHML, we could use the Graph API that already support it
GRAPHJSON, we could use the Graph API that already support it (https://groups.google.com/forum/#!topic/gremlin-users/SD7yXKT8DXY)

+1

In last 2.2.9 import is already able to import from GraphML and GraphSON. It's still missing:

export in GraphML (easy to do because it's already in TinkerPop libs)
export in GraphSON (easy to do because it's already in TinkerPop libs)
export in SQL. This requires more code.


SVG and power-of-two mipmap

Following discussion in russian html5gamedev channel in telegram.
@busyrev has a great idea:
When we make texture from SVG, add an option to make power-of-two texture with mip levels made from the same SVG source.

May be it will be better to use gl.TEXTURE_MIN_FILTER with gl.LINEAR_MIPMAP_NEAREST value, because mip levels generated separatly from vector.


Message causes confusion

pREST version (or commit ref): master
pREST endpoint: -
PostgreSQL version: 10
Operating system: linux
Go version: 1.10.3
Log gist:

Description
Using this prest.toml, but set PREST_PG_USER in environment with "blah", prest uses env as priority but shows that using prest.toml file, causing some confusion.

thanks @cassiobotaro


Fix for when body of pull request is null

The body of a PR can is a null literal when the PR's body is completely blank.
This causes a .trim() invocation to fail later on by null reference.

Also, why was some code written in dist/index.js absent from src/index.js?

I see npm run package was just not done after modifying source and releasing... oof.
Updates dist/index.js to remove ignoreActionLabel as commit https://github.com/zattoo/output/commit/174c5d048ed0e8cfcb86cc330ec64aab0e19b615 on `main does.


IPFS icons

@ericronne Assuming the latest is here right?
https://github.com/ipfs-shipyard/ipfs-css/tree/master/icons

Those icons were designed for ipfs desktop/webui. They're likely a good jumping-off point for further development.


added support for user-data when using AWS

For cross platform support I still want to use cloudbase-init without making specific cloud-configs

Thank you for contributing to openstack/cloudbase-init!
openstack/cloudbase-init uses Gerrit for code review.
If you have never contributed to OpenStack before make sure you have read the
getting started documentation:
http://docs.openstack.org/infra/manual/developers.html#getting-started
Otherwise please visit
http://docs.openstack.org/infra/manual/developers.html#development-workflow
and follow the instructions there to upload your change to Gerrit.

Hello,
Thank you for the contribution. I opened a feature request on Launchpad [1]. We are currently working to have this merged in cloudbase-int.
Thanks.
[1] https://launchpad.net/cloudbase-init

Hello,
AWS userdata support has been merged: https://blueprints.launchpad.net/cloudbase-init/+spec/user-data-support-for-aws
You can download the latest cloudbase-init installer and you will have the feature there.
Thank you,
Adrian Vladu


Added documentation for ProviderObserver

Close #132

@rrousselGit Let me know what do you think. Also I noticed that when a value of provider is logged it show Instance of 'StateController' maybe should add toString() method to StateController to expose the value instead ?

Thanks a lot!
A few things:

I don't think we need to remind users what a provider or Consumer is.
We should list all the life-cycles of this object (so didAddProvider & co)
I like to start these pages with a description of "A use-case for this class" – typically logging & devtools

@rrousselGit I'll make those changes. When you say that we don't need to remind users of what a consumer or provider is you mean in the example ?

Yes. I was referring to the comments, which explains what Consumer/context.read are

@rrousselGit I think it's good now let me know if it needs any changes :)

Thanks!


Populate enqueued_at when enqueuing jobs

This needs to be done when we enqueue through Verk.enqueue and when jobs move from the scheduled set to the queues.
enqueued_at is just an unix time of the moment we enqueued.

Kia Ora!
I had some time to work on this. According to https://redis.io/commands/eval we can't get system time in Lua script itself:

In order to enforce this behavior in scripts Redis does the following:
Lua does not export commands to access the system time or other external state.
Redis will block the script with an error if a script calls a Redis command able to alter the data set after a Redis random command like RANDOMKEY, SRANDMEMBER, TIME. This means that if a script is read-only and does not modify the data set it is free to call those commands. Note that a random command does not necessarily mean a command that uses random numbers: any non-deterministic command is considered a random command (the best example in this regard is the TIME command).

<IP_ADDRESS>:6379> eval "local t = redis.call('TIME'); redis.call('LPUSH', 'test_queue', t[1]); return t[1]" 0
(error) ERR Error running script (call to f_23907c1ac8a9f92f2e36a750baa9b1ac885dc9d6): @user_script:1: @user_script: 1: Write commands not allowed after non deterministic commands

So it have to be passed to the script as argument, which is already done anyway - it's used for the sorted set score.
I did not want to make the script aware for the the difference between the retry and schedule sets, so we can either change it to set enqueued_at for all jobs that don't have this already (assuming these will be coming only from the schedule set), which I did in https://github.com/krasio/verk/tree/issues/109 or we can change the ScheduleManager to pass additional param when handling fetch_scheduled - https://github.com/krasio/verk/tree/issues/109-2.
The downside of the former approach is that at after upgrade all jobs from the retry set will get their enqueued_at set to the time they were retried first time after upgrade.
Let me know what do you think and I'll send a pull request.

Hey @krasio! I think it's fine. Can you set up a PR? Sorry for the delay

Which one of the approaches will you prefer?

Oops sorry I forgot to specify which one. I would simply populate enqueued_at when the data is missing. The downside is that people upgrading will see a wrong enqueued_at when there's a retry but after some time it will be fine. What do you reckon?

I'm fine with that, will submit PR soon.


FUGR includes with author SAP are skipped

We noticed that some very old includes are not serialized in one of our repositories. I debugged why and came upon this:
https://github.com/larshp/abapGit/blob/d1c5e332d4d6fdeb78c68994d734c9ec59a3385b/src/objects/zcl_abapgit_object_fugr.clas.abap#L519-L524
This is something we would like to serialize, it just happens that the author is listed as SAP for historical reasons.
What is the purpose of this logic?
Would it possible to utilize a different field such as last changed by to do this exclusion?

try creating a table maintenance function group, perhpas this comment is related:
https://github.com/larshp/abapGit/issues/15#issuecomment-134636714

The generated includes seem to start with LSVIM. Could this be good enough criteria for excluding them?

Looking at last changed by would work for our purposes, but would not correctly exclude all of the table maintentance includes:

LSVIM, yea, lets try

This is what remains from the generated group:
https://github.com/FreHu/abapgit-fugr-test/tree/master/src
There are no changes in the repo when I run abapgit without the fix.

cool, suggest adding a comment in the code so we can remember what LSVIM is


[Bug] Prompt with trailing whitespace may hurt model performance

Prerequisite

[X] I have searched Issues and Discussions but cannot get the expected help.
[X] The bug has not been fixed in the latest version.

Type
I have modified the code (config is not considered code), or I'm working on my own tasks/models/datasets.
Environment
{'CUDA available': True,
 'CUDA_HOME': '/usr/local/cuda',
 'GCC': 'gcc (GCC) 9.2.1 20200522 (Alibaba 9.2.1-3 2.17)',
 'GPU 0,1,2,3': 'NVIDIA A100-SXM4-80GB',
 'MMEngine': '0.10.3',
 'MUSA available': False,
 'NVCC': 'Cuda compilation tools, release 12.1, V12.1.105',
 'OpenCV': '4.9.0',
 'PyTorch': '2.1.0+cu121',
 'PyTorch compiling details': 'PyTorch built with:\n'
                              '  - GCC 9.3\n'
                              '  - C++ Version: 201703\n'
                              '  - Intel(R) oneAPI Math Kernel Library Version '
                              '2022.2-Product Build 20220804 for Intel(R) 64 '
                              'architecture applications\n'
                              '  - Intel(R) MKL-DNN v3.1.1 (Git Hash '
                              '64f6bcbcbab628e96f33a62c3e975f8535a7bde4)\n'
                              '  - OpenMP 201511 (a.k.a. OpenMP 4.5)\n'
                              '  - LAPACK is enabled (usually provided by '
                              'MKL)\n'
                              '  - NNPACK is enabled\n'
                              '  - CPU capability usage: AVX512\n'
                              '  - CUDA Runtime 12.1\n'
                              '  - NVCC architecture flags: '
                              '-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90\n'
                              '  - CuDNN 8.9.3\n'
                              '    - Built with CuDNN 8.9.2\n'
                              '  - Magma 2.6.1\n'
                              '  - Build settings: BLAS_INFO=mkl, '
                              'BUILD_TYPE=Release, CUDA_VERSION=12.1, '
                              'CUDNN_VERSION=8.9.2, '
                              'CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, '
                              'CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 '
                              '-fabi-version=11 -fvisibility-inlines-hidden '
                              '-DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO '
                              '-DLIBKINETO_NOROCTRACER -DUSE_FBGEMM '
                              '-DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK '
                              '-DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE '
                              '-O2 -fPIC -Wall -Wextra -Werror=return-type '
                              '-Werror=non-virtual-dtor -Werror=bool-operation '
                              '-Wnarrowing -Wno-missing-field-initializers '
                              '-Wno-type-limits -Wno-array-bounds '
                              '-Wno-unknown-pragmas -Wno-unused-parameter '
                              '-Wno-unused-function -Wno-unused-result '
                              '-Wno-strict-overflow -Wno-strict-aliasing '
                              '-Wno-stringop-overflow -Wno-psabi '
                              '-Wno-error=pedantic -Wno-error=old-style-cast '
                              '-Wno-invalid-partial-specialization '
                              '-Wno-unused-private-field '
                              '-Wno-aligned-allocation-unavailable '
                              '-Wno-missing-braces -fdiagnostics-color=always '
                              '-faligned-new -Wno-unused-but-set-variable '
                              '-Wno-maybe-uninitialized -fno-math-errno '
                              '-fno-trapping-math -Werror=format '
                              '-Werror=cast-function-type '
                              '-Wno-stringop-overflow, LAPACK_INFO=mkl, '
                              'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, '
                              'PERF_WITH_AVX512=1, '
                              'TORCH_DISABLE_GPU_ASSERTS=ON, '
                              'TORCH_VERSION=2.1.0, USE_CUDA=ON, USE_CUDNN=ON, '
                              'USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, '
                              'USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, '
                              'USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, '
                              'USE_OPENMP=ON, USE_ROCM=OFF, \n',
 'Python': '3.8.18 (default, Sep 11 2023, 13:40:15) [GCC 11.2.0]',
 'TorchVision': '0.16.0+cu121',
 'numpy_random_seed':<PHONE_NUMBER>,
 'opencompass': '0.2.1+',
 'sys.platform': 'linux'}

Reproduces the problem - code/configuration sample
Evaluating my own model.
Reproduces the problem - command or script
python run.py --datasets agieval_gen \
--models $MY_MODEL \
--model-kwargs device_map='auto' \
--tokenizer-path $TOKENIZER_PATH \
--tokenizer-kwargs padding_side='left' truncation='left' use_fast=False trust_remote_code=True \
--max-out-len $MAX_OUT_LEN \
--max-seq-len 2048 \
--batch-size 8 \
--no-batch-padding \
--work-dir $WORK_DIR \

Reproduces the problem - error message
None
Other information
I'm evaluating on AGIEval and notice a performance drop under default config. Dig into predictions, I find that model generates unusual tokens, like multi white spaces or "\n".
https://github.com/open-compass/opencompass/blob/ba7cd58da3317bdec233d097153e2ab92c5f5dd5/configs/datasets/agieval/agieval_gen_64afd3.py#L72
The issue is gone when I remove the trailing whitespace. It seems like an OOD problem when a base model tries to predict under a situation not seen in the pre-training stage, which is also mentioned in this video. Go back to the original repo of AGIEval, there're no trailing whitespaces.

You are right. The LLMs are sensitive to the prompt.

@tonysy Is it considered a bug of OpenCompass and getting fixed in future release? I've noticed several other datasets with prompt configured similarly, which could cause possible performance downgrade.

I think is not a bug, it's the issue of the LLM other than the evaluation. Actually, we may need introduce several different prompts  to improve the robustness of the evaluation.

@tonysy I agree with this view. However, I want to point out that OpenCompass could give different results compared to the original version, and the prompts change with different datasets, such as those with or without trailing whitespace. But, we can partially fix this issue in the tokenization stage.

Right, we are working on the prompt sensitivity and will provide multi-prompt result recently. Stay tuned.

@yzlnew It a problem related to bpe dropout. Our paper has discussed this problem.

@longxudou Thanks, seems a simple but effective fix during the tokenization stage.


Terraform State Lock not released

We've hit an issue where the State lock was not released due to the runner pod not shutting down correctly (assumption here) due to a node refresh on an EKS Cluster. So we ended up with a Kubernetes Lease that was still held.
$ kubectl -n flux-system get lease
NAMESPACE     NAME                      HOLDER                                AGE
...
flux-system   lock-tfstate-default-sg1                                        3d7h
flux-system   lock-tfstate-default-sg2  f2ab685b-f84d-ac0b-a125-378a22877e8d  3d7h

The Terraform resource was reporting the following:
error running Plan: rpc error: code = Unknown desc = error acquiring the state lock: Lock Info: ID: f2ab685b-f84d-ac0b-a125-378a22877e8d Path: Operation: OperationTypePlan Who: controller@sg2-tf-runner Version: 1.1.9 Created: 2022-06-15 22:27:10.244794577 +0000 UTC

Which was repeated in the tf-controller and tf-runner logs. We were able to get things running again by deleting the Lease in question.
So the ask here is to support the Terraform force-unlock feature.
I think the best approach might be to add a new field to the TerraformSpec, e.g. forceUnlock, the value of which should be the Lock Info: ID: [value], in the above example this would be forceUnlock: f2ab685b-f84d-ac0b-a125-378a22877e8d. This ID corresponds to the key spec.holderIdentity on a  Lease.
The tf-controller/runner should be tasked with acting on this, allowing for unlocking other Terraform backends that support locking if they become supported by the tf-controller.
This could then be supported in the tfctl e.g. tfctl force-unlock f2ab685b-f84d-ac0b-a125-378a22877e8d which would set the field on the Terraform resource and force a reconciliation. Is there a potential race condition here?

Another possible solution would be to have the tf-controller watching the Leases to see if a tf-runner completes but fails to release the Lease, the tf-controller could then force the release of the Lease, this wouldn't require user intervention and could maybe be a configuration of the tf-controller or another configuration option of the Terraform resource.

We found two issues that were the cause of this in two separate cases:
{
  "level": "error",
  "ts": "2022-06-15T20:14:34.824Z",
  "logger": "runner.terraform",
  "msg": "error creating the plan",
  "error": "exit status 1
    Error: Error releasing the state lock
    Error message: etcdserver: request timed out
    Terraform acquires a lock when accessing your state to prevent others
    running Terraform to potentially modify the state at the same time. An
    error occurred while releasing this lock. This could mean that the lock
    did or did not release properly. If the lock didn't release properly,
    Terraform may not be able to run future commands since it'll appear as if
    the lock is held.

    In this scenario, please call the \"force-unlock\" command to unlock the
    state manually. This is a very dangerous operation since if it is done
    erroneously it could result in two people modifying state at the same time.
    Only call this command if you're certain that the unlock above failed and
    that no one else is holding a lock."
}

And
I0615 22:29:32.020709       7 request.go:665] Waited for 1.027123219s due to client-side throttling, not priority and fairness, request: GET:https://<IP_ADDRESS>:443/apis/node.k8s.io/v1?timeout=32s
{\"level\":\"info\",\"ts\":\"2022-06-15T22:29:40.658Z\",\"logger\":\"runner.terraform\",\"msg\":\"creating a plan\"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xd165ea]

goroutine 359 [running]:
github.com/hashicorp/terraform-exec/tfexec.(*Terraform).buildTerraformCmd(0x0, {0xc0005e0280, 0x1}, 0x162e028, {0xc000616400, 0x47d9be, 0x0})
  /go/pkg/mod/github.com/tf-controller/terraform-exec@v0.15.1-0.20220114135154-c06c64609ee0/tfexec/cmd.go:174 +0x2a
github.com/hashicorp/terraform-exec/tfexec.(*Terraform).planCmd(0xc0003fc120, {0x1847728, 0xc00063a3c0}, {0xc0003fc130, 0x1, 0x10})
  /go/pkg/mod/github.com/tf-controller/terraform-exec@v0.15.1-0.20220114135154-c06c64609ee0/tfexec/plan.go:179 +0xb6f
github.com/hashicorp/terraform-exec/tfexec.(*Terraform).Plan(0x1498420, {0x1847728, 0xc00063a3c0}, {0xc0003fc130, 0xf, 0x0})
  /go/pkg/mod/github.com/tf-controller/terraform-exec@v0.15.1-0.20220114135154-c06c64609ee0/tfexec/plan.go:100 +0x31
github.com/weaveworks/tf-controller/runner.(*TerraformRunnerServer).Plan(0xc000ad63f0, {0x18477d0, 0xc0005247e0}, 0xc0003c3270)
  /workspace/runner/server.go:384 +0x4e6
github.com/weaveworks/tf-controller/runner._Runner_Plan_Handler({0x15dcdc0, 0xc000ad63f0}, {0x18477d0, 0xc0005247e0}, 0xc0005c15c0, 0x0)
  /workspace/runner/runner_grpc.pb.go:489 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0005c8000, {0x1873270, 0xc000603380}, 0xc0004c0000, 0xc000614cc0, 0x23c99c0, 0x0)
  /go/pkg/mod/google.golang.org/grpc@v1.45.0/server.go:1282 +0xccf
google.golang.org/grpc.(*Server).handleStream(0xc0005c8000, {0x1873270, 0xc000603380}, 0xc0004c0000, 0x0)
  /go/pkg/mod/google.golang.org/grpc@v1.45.0/server.go:1619 +0xa2a
google.golang.org/grpc.(*Server).serveStreams.func1.2()
  /go/pkg/mod/google.golang.org/grpc@v1.45.0/server.go:921 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
  /go/pkg/mod/google.golang.org/grpc@v1.45.0/server.go:919 +0x294

To expand on the above I'm looking at adding the following to the spec:
spec:
  backendConfig:
    state:
      forceUnlock: [holder-id]

By default, and following the same path as Terraform, this would require manual intervention by an operator, who would need to put this [holder-id] in place in the the Terraform resource. This can be done by changing the resource in the Flux Source or by running tfctl force-unlock [terraform-resource] [holder-id], which would set the value and then trigger a reconcile for that resource.
 Ideally we'd like the tfctl force-unlock to trigger a reconciliation immediately so that the Terraform resource doesn't get reset by Flux before the reconcile happens.
I think it also makes sense to allow an automated in which case you would set spec.backendConfig.state.forceUnlock: auto, this allows for the use case where the assumption is that nothing else will ever be using this state, so we are okay with the tf-controller automatically unlocking the state. Obviously putting all the warnings around this in the documentation.
@chanwit @tomhuang12 @phoban01 what do you think about the above?

I have changed the above noted structure as it impacts the backendConfig in a way that seems to complicate it more than I'd like.
So I've moved spec.backendConfig.state out to spec.terraformState and changed the structure of this a little to the following:
spec:
  terraformState:
    forceUnlock: (yes|no|auto)
    lockIdentifier: [holder-id]

spec.terraformState.forceUnlock defaults to no

Thank you @Nalum
This design is excellent. It gives us auditable activities for force unlocking.
spec:
  terraformState:
    forceUnlock: (yes|no|auto)
    lockIdentifier: [holder-id]

There are a few things to improve UX.

I would change terraformState to tfstate so users would be able to reason that they're working with tfstate at the moment.
What are the semantics of forceUnlock=yes and forceUnlock=auto? How are they different?

Thanks for the feedback 👍

There are a few things to improve UX.

I would change terraformState to tfstate so users would be able to reason that they're working with tfstate at the moment.

Ah yes, this makes sense, will make this change 👍

What are the semantics of forceUnlock=yes and forceUnlock=auto? How are they different?

The purpose of auto is to have the tf-controller automatically unlock the state, in the case where the user is happy for this to happen, and yes is a manual trigger, the expectation being that it is triggered from the tfctl.
This has just made me think about some of the implementation and I need to revisit it.

Closing as #268 has been merged!


Implements #[allow(dead_code)]

Description
Added support for new attribute #[allow(dead_code)]
Checklist

[x] I have linked to any relevant issues.
[x] I have commented my code, particularly in hard-to-understand areas.
[ ] I have updated the documentation where relevant (API docs, the reference, and the Sway book).
[x] I have added tests that prove my fix is effective or that my feature works.
[x] I have added (or requested a maintainer to add) the necessary Breaking* or New Feature labels where relevant.
[x] I have done my best to ensure that my PR adheres to the Fuel Labs Code Review Standards.
[x] I have requested a review from the relevant team or maintainers.

Can you also add more testing around methods?

Added test for method.

@tritao created PR https://github.com/FuelLabs/sway/pull/4203 that adds docs to all attributes including #[allow(dead_code)].


WebAPI integration

Do you consider WebAPI integration instead of MVC?
One advantage of WebAPI is; it is built for single page angular2 projects, mobile apps, etc.

This project is based on .Net Core where Microsoft team has merged MVC and WebApi together. This project is using backend as webapi.


TensorFlow import

Tensor flow report issue:
I am getting the following error  when importing tensorflow.
import tensorflow as tf
Traceback (most recent call last):
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 18, in swig_import_helper
return importlib.import_module(mname)
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\importlib_init_.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 978, in _gcd_import
File "", line 961, in _find_and_load
File "", line 950, in _find_and_load_unlocked
File "", line 648, in _load_unlocked
File "", line 560, in module_from_spec
File "", line 922, in create_module
File "", line 205, in _call_with_frames_removed
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 41, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 21, in
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 20, in swig_import_helper
return importlib.import_module('pywrap_tensorflow_internal')
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\importlib_init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_pywrap_tensorflow_internal'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow_init_.py", line 24, in
from tensorflow.python import *
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python_init_.py", line 49, in
from tensorflow.python import pywrap_tensorflow
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 52, in
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 18, in swig_import_helper
return importlib.import_module(mname)
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\importlib_init_.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 978, in _gcd_import
File "", line 961, in _find_and_load
File "", line 950, in _find_and_load_unlocked
File "", line 648, in _load_unlocked
File "", line 560, in module_from_spec
File "", line 922, in create_module
File "", line 205, in _call_with_frames_removed
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 41, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 21, in
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 20, in swig_import_helper
return importlib.import_module('pywrap_tensorflow_internal')
File "C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib\importlib_init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_pywrap_tensorflow_internal'
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/install_sources#common_installation_problems
for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.

(1) Please review tensorflow guidelines here before raising an issue.
(2) Please go to stackoverflow for more help.
(3) If you are trying to install tensorflow-gpu then have to follow specific guidelines here.
(4) If you are trying to install tensorflow without gpu support then read and follow installation guidelines here.

It seems module named '_pywrap_tensorflow_internal' is missing or cannot found.
Therefore, your TensorFlow runtime failed to load.
Try to fix it by going to your C:\Users\Deena Awny\AppData\Local\Programs\Python\Python36\lib directory and locate the missing file.

This issue is automatically closed due to lack of activity. Please re-open if this is still an issue for you. Thanks!


Angular 5.x

While I'm not sure if this PR is wanted or not, these are the changes I made in order to use this library with an Angular 5 project that I'm developing.
Full Disclosure:
I'm not sure how or if it's possible to make these changes work with an Angular 4.x project.

Originally, I scaffolded the project with the Angular CLI, and never updated the dependencies. I wanted to have an environment where I could run npm start for my dev environment and develop the library against the demo code.
We could probably move all of the dependencies into devDependencies since they are only there for use at development time. Especially stuff like @angular/router.
Then we could move the actual dependencies that ARE used by the library into peerDependencies.
Also, the versions should probably be marked as `"@angular/core": ">=4.0.0", and whatever the other appropriate versions are for Angular 4.x libs (like zone, rxjs, and core-js). This should allow backwards compatibility.
I don't believe the project is using any API's that are new to Angular 5.x.

We could probably move all of the dependencies into devDependencies since they are only there for use at development time. Especially stuff like @angular/router.

Then we could move the actual dependencies that ARE used by the library into peerDependencies.

I agree, this is the approach I'm taking with my projects as well. That, with the combination of your >= suggestion should work nicely.

On further inspection it appears there is this issue regarding modules built against angular 4 dependencies vs angular 5 dependencies: https://stackoverflow.com/a/47534611
I'm not sure the exact dependency/version change that causes the metadata version mismatch, but it seems to be a common issue. If there isn't a viable solution for supporting both, would you be open to a major version bump to accommodate a change such as this?

Yeah, that'd be fine. I'm planning on releasing a new version soon so accommodate horizontal forms, and in working on that, I decided to introduce an actual component that needs to be placed where you want the error messages to appear. So that's gonna be a breaking change as well.
Do you want to update the PR?

component that needs to be placed where you want the error messages to appear

What do you think about checking for a viewChild and if it doesn't exist, add the component automatically with the current placement behavior. That would allow people to continue to use the library as it is today, but also opting in to custom error message placement.

Also, yes I'll work on updating this PR  ( :