override virtual methods

In Vala, we can override virtual methods and this helps to achieve things like this:
public class App : Adw.Application {
  public App() {
    Object(application_id: "org.app");
  }

  static int main(string[] args) {
    return new App().run(args);
  }

  protected override void activate() {
    // ...
  }

  protected override void startup() {
    base.startup();

    // ...
  }
}

In this case, this skips the step of connecting functions to signals and the app just will run without problems, is this possible in gintro?

Unfortunately with gintro we can not do all that what is possible in C or in Vala. (I think Rust is closest to C and Vala for GTK.)
For example, in current gintro we can not create new GObjects with new properties. And the GtkExpression is not supported in gintro or in the Python bindings. This may be generally possible with Nim GTK bindings, but it is difficult and would need some work.
When you already have learned Vala, I would recommend just to continue with Vala, as Vala was created for GTK. When you think that you  need "override virtual methods" in Nim, first step would be to provide a minimal but working example in plain C for that, then we can start thinking about how it can be done in Nim. Unfortunately I am not familiar with Vala.

I see, thanks anyway, i'll try to check and see how to do that in C and provide an example

I think for override virtual methods, we have to support creating and subclassing GObjects, as described in https://docs.gtk.org/gobject/tutorial.html. It would be nice to have support for that in the gintro bindings, but is in not really easy. In C a lot of complicated and nested macros like G_DECLARE_FINAL_TYPE() and many more are used, which are generally not available for language bindings. For Nim, we could try to expand the C macros into plain function calls, and then try to do the same in Nim. I was playing with that many years ago when I created the oldgtk3 bindings, and wrote the old NEd editor. It was really difficult. I have seen how GTK subclassing is used from Python, looks not too nice. For doing it in Nim, I would have to understand all the GObject and C macro stuff very well, and then think about a nice Nim API. And then create the Nim macros. I am sure that I would be not able to do all that in less than 100 hours. I am slow, so I should better assume 200 hours when doing it myself. Maybe someone can do it in 100 hours, maybe Mr Rumpf, Mr. Disruptec, Mr. Döhring? Or someone from the GTK team, Mr. Bassi, or Mr. Droege from the Rust team? You may contact that people.
You may know, that we initial intended to create a new Nim binding set based on the Rust bindings, as the Rust bindings may be the most advanced. I would be willing to donate up to 1000 Euro for that task, so with some more supporters, we had some hope to collect some 10k Euro to hire some experts. But with the current state of Nim, GTK, and gintro, I have no hope for such a project any more. I guess the actual number of users of gintro is one or two currently. And actually I looked at some Rust GTK code recently -- using GTK from Rust is really ugly.
When you really should intent to use Nim with GTK, and need low level GObject access, you may consider mixing plain C and Nim. It is possible and not so difficult, I gave an example some years ago in the gintro issue tracker, will see if I can find it. Yes, here it is: https://github.com/StefanSalewski/gintro/issues/141 When you have some basic C and Nim skills, you may be able to solve your issues.
Maybe a very different approach is, to use low level bindings. My oldgtk3 is still available. Or some people have generated low level bindings generators, like the futhark of P. Munch, or earlier tools of people who have left Nim. I think there was something like toast or nimbindgen some years ago?
Or use Vala, Rust, or Python. Or a native Nim GUI toolkit, maybe that of treeform and elcritch -- figma or fidget or somethink like that?

Makes sense, well, I just was being curious. And since you said to keep going using Vala in your first response, I'll do that. I known making bindings for GTK is a really hard thing, so there's not much I can do about this

I am under the impression that overriding virtual methods is the only way to perform certain tasks...
I looked around and the GObject type system is described here in more detail : https://docs.gtk.org/gobject/concepts.html this is basically what is behind the macros.

The provided link should be enough to subclass a GObject using the gobject bindings manually without using any of the C macros. Basically subclassing an object seems to go down to:

declare a struct for the subclass vtable linking to the parent class vtable and containing the virtual method function pointers
declare a struct for the subclass instance object containing the instance variables
declare a _get_type() function that returns a GType based on a GTypeInfo, the first time it is called, it initializes the GType.
when the GType is initialized, function pointers to init and finalize both the class vtable (to set the function pointers) and the object instance (to initialize instance variables) are used

With that, you should be close to ready to subclass any GObject.

Thanks for your comment. I think I remember your name. But you should be aware, that with comments like https://forum.nim-lang.org/t/10312#68650 you support their behaviour with had seriously damaged the whole Nim project. As you are observing the Nim project for some years already, you might know their impolite behaviour well, as discussed in https://www.reddit.com/r/nim/comments/ywxsbz/this_is_disappointing_to_read_coming_from_the_nim/. Not to mention all the personal attacks like https://forum.nim-lang.org/t/10101#66729
I think I have seen much better comments from you in the past, but I can not remember details, sorry.
For gintro, I just answered to a request of Mr. Arkan and explained why I am not any longer work on Gintro or use Nim, see https://github.com/StefanSalewski/gintro/issues/224
Your provided link is interesting, it seems to be new content, or at least a new layout. At least I can not remember having seen that page. I may read it in next winter, thanks.


Other Services doesnt work

I have compiled the code, and have been playing around with it a bit. Everything works fine for me, but if i want to change SWITCH_S  service (button_led DEMO project) for example to MOTION_SENSOR_S
      chas=addService(      sers,++iid,APPLE,MOTION_SENSOR_S);      addCharacteristic(chas,aid,++iid,APPLE,NAME_C,"motion",NULL);      addCharacteristic(chas,aid,++iid,APPLE,MOTION_DETECTED_C,"1",NULL);
pairing doesnt work anymore and for other services - the same issue. What am i doing wrong? Thx in advance .

At least you have not violated the minimum list of characteristics
RequiredCharacteristics
00000022 Motion Detected
OptionalCharacteristics
00000075 Status Active
00000077 Status Fault
0000007A Status Tampered
00000079 Status Low Battery
00000023 Name
00000085-0000-1000-8000-0026BB765291 Motion Sensor
BUT, if you look inside the source code (hkc.h) you will see that MotionDetected is not yet implemented since no info is behind it
#define MOTION_DETECTED_C                           0x22
#define NAME_C                                      0x23    //64    string  r
e.g. Name does have that
This means no code is inside the driver so far, explaining the failure
I’ll see when I get around it, or you do it yourself… and file a push request
bool	R CU  00000022 Motion Detected
Good luck,
Hack

what the hack, I just pushed the update, please try again?

Thx for reply!
Hm, still doesnt want to pair, seems that iphone disconects the connection, not the ESP... And the wierd thing is that i undo all the changes  (returned SWITCH_S back), but no pair cmplete anymore...

Uh, sorry, made it work! Thx! You did a great job!

Thing is that there is still a weird behaviour in the pairing that didn’t happen before.
I think it is iOS11 that checks bonjour midway.
One of my lower priority experiments is to flip the pairing flag after pair setup step 1 and see if that improves stuff
Anyway, glad to be of service.
Please report to the Demo repo with the details of your experiment so others can enjoy it as well

Can you help me with CURRENT_RELATIVE_HUMIDITY_C
I add MOTION_SENSOR_S:
//service 3
chas=addService(      sers,++iid,APPLE,HUMIDITY_SENSOR_S);
addCharacteristic(chas,aid,++iid,APPLE,NAME_C,"humidity",NULL);
addCharacteristic(chas,aid,++iid,APPLE, ,"0",humidity);
but pairing doesn't work.

@medmile It looks like in your code sample you did not specify proper characteristic type (see double commas after APPLE: (chas,aid,++iid,APPLE, ,"0",humidity);). That makes whole accessory invalid and thus controller refuses to pair.

@maximkulkin Thank you. It’s mistake)
To solved it I add this to hkc.c this code:
case CURRENT_RELATIVE_HUMIDITY_C:{
strcpy(format,FLOAT);        perm=5;
cJSON_AddNumberToObject(acc_items[iid].json, "minValue",   0);
cJSON_AddNumberToObject(acc_items[iid].json, "maxValue",   100);
cJSON_AddNumberToObject(acc_items[iid].json, "minStep",    1);
cJSON_AddStringToObject(acc_items[iid].json, "unit", "celsius");
} break;

BTW, If you guys missed my announcement, I've released another HomeKit library for ESP8266 (and demo). You might want check it out.


Link myuw-developers , uportal-user, and uportal-dev Google groups.

Coverage remained the same at 15.733% when pulling aed6fafa45ba31ff71b6efc25b504f78860a6bb9 on apetro:link-goog-group into 6dec8635a30dee78fe4646cdf129ffd4f6725e0a on UW-Madison-DoIT:master.

:+1:


[Neural Speed] Cont Batching in Offline and Server (GPT-J & Beam Search First)

Type of Change
feature or bug fix or documentation or others
As title
Description
detail description
Jira: 829 && 1152

How to use?
python usage please see here
Do other models support cont-batching?
we will support cont-batching in other models in the future and this cont-batching will be the main batching inference way.
Does it improve performance
It should be since it saves padding dealing time in offline scenario and makes inferences on the fly in server scenario. we will update GPT-J MlPerf results soon.
Will it affect logits?
We think it would not (for-loop rope & kv-concat & self-attn). However, we found it seems to have some different logits (but not too much). It can pass MLPerf offline and server scenarios accuracy tests (int4-int8-per-channel & int4-bf16-per-channel). Will raise another PR if it's a bug in the future.

Expected Behavior & Potential Risk
the expected behavior that triggered by this PR
How has this PR been tested?
how to reproduce the test (including hardware information)
Dependency Change?
any library dependency introduced or removed

gpt-j-6b ext test: https://inteltf-jenk.sh.intel.com/job/neural_speed_extension/43/

gpt-j-6b ext test: https://inteltf-jenk.sh.intel.com/job/neural_speed_extension/43/

no cont_batching in extension test, so we will test cont_batching though python API from now on?

gpt-j-6b ext test: https://inteltf-jenk.sh.intel.com/job/neural_speed_extension/43/

no cont_batching in extension test, so we will test cont_batching though python API from now on?

yes. Added cont-batching offline generation result test in the test folder. As for offline perf and ModelServer, I will discuss with @DDEle @zhenwei-intel how to add a test and how to measure its perf after enabling more important models (llama, qwen, etc.)


The Deploy freezes after a success Build

We have problems with some deploying. The provisioning and the build finish successfully, but the Deploy freezes without any information about any problem. We tried to set up in different regions in EU but the problem still.

@itdoh , I'm sorry that you are having this issue. Would you mind share your appId and region? I'll investigate your issue.

Hi @Joycehao19 thank you for your response. These are the information:
appId: arn:aws:amplify:eu-central-1:000098351948:apps/d3lfc9pdtt3gfv
region: eu-central-1
Thanks!

Closing this to track in #115.

Hi @swaminator ,
The issue persists. Now we are trying in a new APP with the same code that other APP that is Deploying well:
These are the APP with problems to deploy:
ARN: arn:aws:amplify:eu-central-1:000098351948:apps/d2k00by58w5diy
region: eu-central-1
These are the APP Deploying well:
ARN: arn:aws:amplify:eu-central-1:000098351948:apps/d2fbd8k0un5fqf
region: eu-central-1
The two APP have the same code.
Thanks!


Interactive mode in scope start

Current work on scope start allows to pass the input data with pipe or redirect mechanism e.g. scope start < example_filter.yml or cat example_filter.yml | scope start
The mechanism described below can be overridden with --force option
It would be nice to mechanism that by default would inform the user about the action which is performed and ask about confirmation e.g.

scope start < example_filter.yml

Following actions will be performed:
- extraction of the libscope.so to /tmp/libscope.so on the host and on the containers
- extraction of the filter inptu to /tmp/scope_fitler.yml on the host and on the containers
- setup etc/profile script to use LD_PRELOAD=/tmp/libscope.so on the host and on the containers
- setup the services which meet the allow list conditions on the host and on the containers
- attach to the processes which meet the allow list conditions on the host and on the containers
Are you want to proceed? Y/n

Closed since the issue is duplicated mentioned in #1112, as:
add y/n are you sure in cli with --force/-f as an override

The issue will be possibly resolved by https://github.com/criblio/appscope/commit/db3b64328497bb7af0eab8c80e9f4ed0bda8547d


modifed h/vflip assert to only check if 2D tensors are contiguous (maintaining backward compat.)

Thanks to @Atcold for suggesting this.
This is a temporary fix.  It certainly can't go back to not having the assert but someone will need to fix it eventually.

Tnx @jonathantompson. So, troubles arise only with 2D Tensors? What about 4 and 5D? I've just tried 3D.

Hflip and vflip only support 2D/3D. Flip supports up to 5D but all tensor types (except 5D) must have contiguous inputs.
Sent from my iPhone.

On Apr 24, 2015, at 5:08 PM, Alfredo Canziani<EMAIL_ADDRESS>wrote:
Tnx @jonathantompson. So, troubles arise only with 2D Tensors? What about 4 and 5D? I've just tried 3D.
—
Reply to this email directly or view it on GitHub.

@jonathantompson, alright.


add update changelog history to the extension and extesion page

it will help people understand the issues fixed in preview version as well

Hey @fisforfaheem we have them:

https://marketplace.visualstudio.com/items/sourcegraph.cody-ai/changelog
https://github.com/sourcegraph/cody/releases

but it needs to be updated visually and improved with dates and details.
Also update the dekstop app
On Fri, Oct 27, 2023 at 2:27 AM Justin Dorfman @.***>
wrote:

Hey @fisforfaheem https://github.com/fisforfaheem we have them:

https://marketplace.visualstudio.com/items/sourcegraph.cody-ai/changelog

https://github.com/sourcegraph/cody/releases

[image: Image 2023-10-26 at 2 24 54 PM png]
https://user-images.githubusercontent.com/398230/278479443-8e4de80f-9361-4099-9b61-349b45c3b381.png
—
Reply to this email directly, view it on GitHub
https://github.com/sourcegraph/cody/issues/1512#issuecomment-1781918725,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIRXJSD6LESL2JXPRNB6PKLYBLIU5AVCNFSM6AAAAAA6RVCLWCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBRHEYTQNZSGU
.
You are receiving this because you were mentioned.Message ID:
@.***>

@fisforfaheem you are right, it should be uniform with the other changelog. @abeatrix is this something we can start doing? If I can help in anyway LMK.

Also update the dekstop app

We will no longer be updating or supporting the desktop app. We will eventually be bringing that functionality into the plugins.

@jdorfman i think he's referring to changelog for App and not just VS code.
The vs code changelog includes link to PR for each change already, and we also have blog and release notes.

Okay got it.
Kindly focus on improving the chat and it should work better alongside
copilot and intellisence as well
On Sat, Oct 28, 2023, 10:19 AM Beatrix @.***> wrote:

@jdorfman https://github.com/jdorfman i think he's referring to
changelog for App and not just VS code.
The vs code changelog includes link to PR for each change already, and we
also have blog and release notes.
—
Reply to this email directly, view it on GitHub
https://github.com/sourcegraph/cody/issues/1512#issuecomment-1783706369,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIRXJSEMVVYOXTYN43BKJGDYBSIVXAVCNFSM6AAAAAA6RVCLWCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBTG4YDMMZWHE
.
You are receiving this because you were mentioned.Message ID:
@.***>


refine job only runs on juicedata/juicefs

Codecov Report

Merging #1812 (d47a723) into main (51301b5) will decrease coverage by 0.03%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##             main    #1812      +/-   ##
==========================================
- Coverage   61.82%   61.79%   -0.04%
==========================================
  Files         135      135
  Lines       22963    22968       +5
==========================================
- Hits        14198    14193       -5
- Misses       7172     7179       +7
- Partials     1593     1596       +3

Impacted Files
Coverage Δ

pkg/meta/redis.go
73.21% <66.66%> (-0.08%)
:arrow_down:

pkg/chunk/utils_unix.go
73.33% <0.00%> (-20.00%)
:arrow_down:

pkg/chunk/cached_store.go
76.70% <0.00%> (-0.74%)
:arrow_down:

pkg/meta/sql.go
67.04% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c00a9ee...d47a723. Read the comment docs.


(PUP-5835) Make node params String if they are Symbol

Before this, if a node parameter had a Ruby Symbol value it would leak
to Scope, and into variables. This was especially true for the
$environment variable after at enviroment name is made into a Symbol.
This commit fixes the problem both by ensuring that when a node sets a
parameter (in itself) based on the environment's name it transformed the
symbolic name to a Strig, and in the general case, the compiler when
setting parameters for a node, will also transform a Symbol value to
Strig. The later to ensure the same kind of mistake does not leak out to
Scope.
This commit also includes unit tests that asserts that environmet name
as a symbol dos not leak, nor any other node parameter.

Note that this change causes this test to fail:
https://github.com/puppetlabs/puppet/blob/a81f968650748442a2ccf6c9acd656c78b94c4e6/spec/unit/node_spec.rb#L90-L96
We could tweak that test of course, but it does highlight the magic around to_sing node parameters. Thoughts?

Also, some nits:

can you rebase to pick up the travis.yml change?
can you correct the spelling of "String" and "environment" in the commit message?

meh, I missed that other failure

Ping @kylog  it is green now.


Get-AzureADMSPrivilegedRoleSetting filters not working

On the page https://docs.microsoft.com/en-us/azure/active-directory/privileged-identity-management/powershell-for-azure-ad-roles#retrieving-and-updating-role-settings it has the sample below.
$setting = Get-AzureADMSPrivilegedRoleSetting -ProviderId 'aadRoles' -Filter "roleDefinitionId eq 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'"
$setting.UserMemberSetting.justificationRule = '{"required":false}'

When running it, it outputs an error as below.  It seems the parameter "ResourceID" is needed, but its not an accepted parameter on the command.
$setting = Get-AzureADMSPrivilegedRoleSetting -ProviderId 'aadRoles' -Filter "roleDefinitionId eq '0964bb5e-9bdb-4d7b-ac29-58e794862a40'"

Get-AzureADMSPrivilegedRoleSetting : Error occurred while executing GetAzureADMSPrivilegedRoleSettings
Code: ResourceId
Message: The required parameters ResourceId is missing.
InnerError:
  RequestId: 6cc4f9d8-9978-42c7-9e71-3be11d6d3565
  DateTimeStamp: Mon, 17 Aug 2020 06:17:24 GMT
HttpStatusCode: BadRequest
HttpStatusDescription: Bad Request
HttpResponseStatus: Completed
At line:1 char:12
+ $setting = Get-AzureADMSPrivilegedRoleSetting -ProviderId 'aadRoles'  ...
+            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Get-AzureADMSPrivilegedRoleSetting], ApiException
    + FullyQualifiedErrorId : Microsoft.Open.MSGraphBeta.Client.ApiException,Microsoft.Open.MSGraphBeta.PowerShell.GetAzureADMSPrivilegedRoleSetting

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 51dbe039-2ab8-b9a3-531d-71de3082cf2c
Version Independent ID: 020b443d-d664-9bad-bbc9-74dbe35c8ae2
Content: PowerShell for Azure AD roles in PIM - Azure AD
Content Source: articles/active-directory/privileged-identity-management/powershell-for-azure-ad-roles.md
Service: active-directory
Sub-service: pim
GitHub Login: @curtand
Microsoft Alias: curtand

@bubbletroubles Thank you for your query. We will investigate and update the thread.

@bubbletroubles is correct, Try including ResourceId in filter parameter like this as workaround:
Get-AzureADMSPrivilegedRoleSetting -ProviderId 'aadRoles' -Filter "(RoleDefinitionId eq '$RoleDefnId') and (ResourceId eq '$TenantId')"

@bubbletroubles  This has been recorded in our internal item and should be processed soon. Thanks.


Add DAI live DASH example to DEMO app

Closes: https://github.com/google/shaka-player/issues/3154

@ismena :)

@av

@ismena :)

At your service!


IoT added in resources

i'm a little unclear on the utility of exposing the iot things as resource, there's quite possibly millions of things extant. could you elaborate on a use case?

Like some can create IoT things and use them so account owner will be charged more so for monitoring purpose this can be useful.


test(*): test using miminum supported database versions

Versions from https://www.prisma.io/docs/reference/database-reference/supported-databases
Closes https://github.com/prisma/prisma/issues/15737

PostgreSQL 9.4 seems not supported by Migrate / Introspection engines
See https://github.com/prisma/prisma/issues/15320

Closing to clean up the open PRs list.


In README.mdhttps://github.com/IdoPort/TechToMed/pull/1#discussion_r1453903730:

+|  15 | FD | Fully Diluted | Functional Dyspepsia |

+|  16 | TOS | Terms of Service | Thoracic outlet syndrome |
+|  17 | ARR | Annual Recurring Revenue | Aldosterone:Renin Ratio |
+|  18 | IRR | Internal Rate of Return | Infusion-Related Reactions |
We have talked with them again and they accepted the ARR.
Please remove the rest and keep only this one.
—
Reply to this email directly, view it on GitHubhttps://github.com/IdoPort/TechToMed/pull/1#discussion_r1453903730, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ASG4EAEOHJHKLMS5SM2JGCDYO3HSPAVCNFSM6AAAAABB5KKW56VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTQMRUG4ZDQMJWGI.
You are receiving this because you authored the thread.Message ID: @.***>


Resource observer has failed: Ran out of valid credentials

I set up a fresh Kubernetes server, installed CusterSecret, and the ClusterSecret pod enters a crash loop on start up.
Kubernetes version: 1.29.3
Host OSs: Linux 6.6.25-1-lts x86_64
Helm chart version: 0.4.1
Application version: 0.0.10
[2024-04-10 09:50:44,104] kopf._core.engines.a [INFO    ] Initial authentication has been initiated.
[2024-04-10 09:50:44,107] kopf.activities.auth [INFO    ] Activity 'login_via_client' succeeded.
[2024-04-10 09:50:44,107] kopf._core.engines.a [INFO    ] Initial authentication has finished.
[2024-04-10 09:50:44,122] kopf._core.engines.a [INFO    ] Re-authentication has been initiated.
[2024-04-10 09:50:44,123] kopf.activities.auth [INFO    ] Activity 'login_via_client' succeeded.
[2024-04-10 09:50:44,123] kopf._core.engines.a [INFO    ] Re-authentication has finished.
[2024-04-10 09:50:44,123] kopf._core.reactor.r [ERROR   ] Resource observer has failed: Ran out of valid credentials. Consider installing an API client library or adding a login handler. See more: https://kopf.readthedocs.io/en/stable/authentication/
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/errors.py", line 148, in check_response
    response.raise_for_status()
  File "/usr/local/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 1005, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 401, message='Unauthorized', url=URL('https://<IP_ADDRESS>:443/apis')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/auth.py", line 45, in wrapper
    return await fn(*args, **kwargs, context=context)
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/api.py", line 85, in request
    await errors.check_response(response)  # but do not parse it!
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/errors.py", line 150, in check_response
    raise cls(payload, status=response.status) from e
kopf._cogs.clients.errors.APIUnauthorizedError: ('Unauthorized', {'kind': 'Status', 'apiVersion': 'v1', 'metadata': {}, 'status': 'Failure', 'message': 'Unauthorized', 'reason': 'Unauthorized', 'code': 401})

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/aiokits/aiotasks.py", line 107, in guard
    await coro
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/reactor/observation.py", line 113, in resource_observer
    resources = await scanning.scan_resources(groups=group_filter, settings=settings, logger=logger)
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/scanning.py", line 31, in scan_resources
    resources.update(await coro)
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 611, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/scanning.py", line 68, in _read_new_apis
    rsp = await api.get('/apis', settings=settings, logger=logger)
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/api.py", line 111, in get
    response = await request(
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/auth.py", line 47, in wrapper
    await vault.invalidate(key, exc=e)
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/structs/credentials.py", line 261, in invalidate
    raise LoginError("Ran out of valid credentials. Consider installing "
kopf._cogs.structs.credentials.LoginError: Ran out of valid credentials. Consider installing an API client library or adding a login handler. See more: https://kopf.readthedocs.io/en/stable/authentication/
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/errors.py", line 148, in check_response
    response.raise_for_status()
  File "/usr/local/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 1005, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 401, message='Unauthorized', url=URL('https://<IP_ADDRESS>:443/apis')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/auth.py", line 45, in wrapper
    return await fn(*args, **kwargs, context=context)
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/api.py", line 85, in request
    await errors.check_response(response)  # but do not parse it!
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/errors.py", line 150, in check_response
    raise cls(payload, status=response.status) from e
kopf._cogs.clients.errors.APIUnauthorizedError: ('Unauthorized', {'kind': 'Status', 'apiVersion': 'v1', 'metadata': {}, 'status': 'Failure', 'message': 'Unauthorized', 'reason': 'Unauthorized', 'code': 401})

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/kopf", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/kopf/cli.py", line 57, in wrapper
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/kopf/cli.py", line 104, in run
    return running.run(
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/reactor/running.py", line 58, in run
    loop.run_until_complete(operator(
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/reactor/running.py", line 135, in operator
    await run_tasks(operator_tasks, ignored=existing_tasks)
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/reactor/running.py", line 416, in run_tasks
    await aiotasks.reraise(root_done | root_cancelled | hung_done | hung_cancelled)
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/aiokits/aiotasks.py", line 237, in reraise
    task.result()  # can raise the regular (non-cancellation) exceptions.
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/aiokits/aiotasks.py", line 107, in guard
    await coro
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/reactor/observation.py", line 113, in resource_observer
    resources = await scanning.scan_resources(groups=group_filter, settings=settings, logger=logger)
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/scanning.py", line 31, in scan_resources
    resources.update(await coro)
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 611, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/scanning.py", line 68, in _read_new_apis
    rsp = await api.get('/apis', settings=settings, logger=logger)
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/api.py", line 111, in get
    response = await request(
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/auth.py", line 47, in wrapper
    await vault.invalidate(key, exc=e)
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/structs/credentials.py", line 261, in invalidate
    raise LoginError("Ran out of valid credentials. Consider installing "
kopf._cogs.structs.credentials.LoginError: Ran out of valid credentials. Consider installing an API client library or adding a login handler. See more: https://kopf.readthedocs.io/en/stable/authentication/

User issue, one of the nodes had it's time desynchronised, and this was a result certification issues.


Get-VstsInput Powershell Core Support???

I created a custom azure devops extension that manages all of our terraform deployments.  The only steps for our development teams to deploy their application and all of the supporting cloud infrastructure is to point the extension to the main.tf file in their code repo and map a variable group that's used to set environment variables.
I recently converted all of the scripts inside the extension to use Powershell core.  Testing on a private windows build server works perfectly, but I cannot get the extension working on a Linux VM running Powershell Core.
It seems that my issue is the VstsTaskSdk is only supported on Windows.  Is there another module I can use that gives me the Get-VstsInput cmdlet or are there any other alternative solutions that would allow me get user input from a custom extension that can run on a linux VM or container?
Any help would be greatly appreciated.
Here is the error I am getting:

2019-08-28T04:58:22.6199963Z ##[error]The current operating system is not capable of running this task. That typically means the task was written for Windows only. For example, written for Windows Desktop PowerShell.

are there any other alternative solutions that would allow me get user input from a custom extension that can run on a linux VM or container

Hey @TechnologyAnimal , Our recommendation for writing cross platform tasks is to use Node.


Fix Postgres discovery duplicated foreign keys

PR Info

Resolve #28
unwrap panic demo #31

Ok, this is interesting... to demo the panic I created #31
And the test result suprise me. Seems that the behaviour of information schema changed since Postgres 12? Running discovery & writer on Postgres version 9, 10, 11 don't panic but version 12 & 13 panic.

https://github.com/SeaQL/sea-schema/actions/runs/1553156144


Indexing fails in IntelliJ

Before submitting your bug report

[X] I believe this is a bug. I'll try to join the Continue Discord for questions
[X] I'm not able to find an open issue that reports the same bug
[X] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info
- OS: MacOS
- Continue version: 0.0.81
- IDE version: 2024.2.4
- Model: nomic-embed-text:latest
- config.json:

{
  "models": [
    {
      "apiKey": "<>",
      "deployment": "gpt-4o",
      "apiBase": "<>",
      "apiType": "azure",
      "model": "gpt-4o",
      "title": "GPT-4o",
      "systemMessage": "You are an expert software developer. You give helpful and concise responses. Omit greetings.",
      "provider": "azure",
      "apiVersion": "2024-08-01-preview"
    }
  ],
  "tabAutocompleteModel": {
    "title": "codellama13b",
    "provider": "ollama",
    "model": "codellama:13b"
  },
  "embeddingsProvider": {
    "title": "nomic",
    "provider": "ollama",
    "model": "nomic-embed-text"
  },
  "customCommands": [
    {
      "name": "test",
      "prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
      "description": "Write unit tests for highlighted code"
    }
  ],
    "contextProviders": [
      {
        "name": "diff",
        "params": {}
      },
      {
        "name": "folder",
        "params": {}
      },
      {
        "name": "codebase",
        "params": {}
      }
  ],
  "slashCommands": [
    {
      "name": "edit",
      "description": "Edit selected code"
    },
    {
      "name": "comment",
      "description": "Write comments for the selected code"
    },
    {
      "name": "share",
      "description": "Export the current chat session to markdown"
    },
    {
      "name": "commit",
      "description": "Generate a git commit message"
    }
  ],
  "allowAnonymousTelemetry": false
}

Description
After update to the latest version of continue plugin, it can not successfully index the codebase.
The logs show this error a lot
[2024-11-13T14:19:01] LanceDBIndex, skipping /Users/<>/.continue/index/lancedb/Users<>.continueindexNONEvectordbOllamaEmbeddingsProvidernomic-embed-textlatest.lance/data/60a98c79-61fc-4c24-835b-92d6553970ea.lance: Error: did not chunk properly
[2024-11-13T14:18:58] LanceDBIndex, skipping /Users/<>/.continue/index/lancedb/Users<>DocumentsProjekte<>.ideamainvectordbOllamaEmbeddingsProvidernomic-embed-textlatest.lance/data/008be3ed-b4d2-4502-8b64-d95afaa8fb3b.lance: Error: did not chunk properly
I have tried both a local ollama model and a cloud embedding model (which was working perfectly fine before the update). Both show the same error.
I have also deleted the ~/.continue/index/ folder and tried to reindex, also did not work.
To reproduce

configure indexing
open project
hit index in the continue settings

Log output

[2024-11-13T14:41:58] LanceDBIndex, skipping /Users/<>/.continue/index/index.sqlite-wal: Error: did not chunk properly
[2024-11-13T14:41:58] LanceDBIndex, skipping /Users/<>/.continue/index/index.sqlite-shm: Error: did not chunk properly

[2024-11-13T14:42:07] LanceDBIndex, skipping /Users/<>/.continue/index/lancedb/Users<>LibraryApplicationSupportJetBrainsIntelliJIdea2024.2workspaceNONEvectordbOllamaEmbeddingsProvidernomic-embed-textlatest.lance/data/45260cde-cd07-4627-9f10-b687f3b1d354.lance: Error: did not chunk properly

[2024-11-13T14:19:01] LanceDBIndex, skipping /Users/<>/.continue/index/lancedb/Users<>.continueindexNONEvectordbOllamaEmbeddingsProvidernomic-embed-textlatest.lance/data/60a98c79-61fc-4c24-835b-92d6553970ea.lance: Error: did not chunk properly

[2024-11-13T14:18:58] LanceDBIndex, skipping /Users/<>/.continue/index/lancedb/Users<>DocumentsProjekte<>.ideamainvectordbOllamaEmbeddingsProvidernomic-embed-textlatest.lance/data/008be3ed-b4d2-4502-8b64-d95afaa8fb3b.lance: Error: did not chunk properly

@florian-hansen-mw what are the effects of this in Continue, other than the logs? For example, when you use @ codebase what happens? And how does the indexing progress bar behave in the more page when you start indexing?


add basic JSON API for creating local copy of public issue data

So that external users can clone and incrementally update a copy of the public issue data, we need two basic API endpoints:

a full list of visible issues, with the ability to sort by the last update timestamp for each issue
a full JSON object containing at least all of the (sanitised) data used by the HTML renderer to produce the HTML view of a bug

Change for review: https://cr.joyent.us/#/c/6851

Testing Notes
I've run a copy of this change against our production JIRA instance and confirmed that the same number of issues are displayed (i.e., those with the public label) both before and after.  I've tried the new API endpoints and made sure they contain only the expected information (e.g., no restricted issue comments).


[EOC-260] [SOCKET] Adding members to a list

Socket logic (regarding adding cohort member) was refactored

Socket logic (regarding adding cohort member) was refactored


[Migration] Summary on completion

Is your feature request related to a problem? Please describe.
I'm not able to understand what happened during the migration.
Describe the solution you'd like
Upon completion, show a summary of what was migration, what was not and why, what are next steps

This issue is closed as the project is no longer maintained


Logoff on non-main inventory world... return to main world to destroyed inventory

Anything with special characters in the item title (colors, etc.) results in this.  Screen shot examples:
https://imgur.com/a/Q9QE2
Most items I picture are SlimeFun related which liked to use color titles, but any item with "special" characteristics qualifies for destruction: vote keys from CratesPlus... Signed books (such as "colors" -- book bundled with Essentials).  Items lose special properties or entirety of function (book is no longer a book (not useable)... Slimefun items are useless other than as "skulls", crate keys no longer access crates.
I have two groups... one inventory group for all 'survival' worlds, one for all 'creative' worlds.  If you log out while in a creative world with "special items" in your (not currently active survival) inventory, they are destroyed when you return to the survival world.
mv version output:
[20:42:49] [Server thread/INFO]: [Multiverse-Inventories] Multiverse-Inventories Version: 2.5.0-SNAPSHOT-b443
[20:42:49] [Server thread/INFO]: [Multiverse-Inventories] First Run: false
[20:42:49] [Server thread/INFO]: [Multiverse-Inventories] Using Bypass: false
[20:42:49] [Server thread/INFO]: [Multiverse-Inventories] Default Ungrouped Worlds: false
[20:42:49] [Server thread/INFO]: [Multiverse-Inventories] Using GameMode Profiles: false
[20:42:49] [Server thread/INFO]: [Multiverse-Inventories] === Groups ===
[20:42:49] [Server thread/INFO]: [Multiverse-Inventories] default: {Worlds: [world, world_the_end, world_nether, pvp], Shares: [hit_points, economy, food_level, saturation, exhaustion, xp, total_xp, lvl, inventory_contents, armor_contents, bed_spawn, maximum_air, remaining_air, fall_distance, fire_ticks, potion_effects, last_location, ender_chest, off_hand]}
[20:42:49] [Server thread/INFO]: [Multiverse-Inventories] CreativeMaps: {Worlds: [staff_creative, event, creative, pixelmap], Shares: [hit_points, economy, food_level, saturation, exhaustion, xp, total_xp, lvl, inventory_contents, armor_contents, bed_spawn, maximum_air, remaining_air, fall_distance, fire_ticks, potion_effects, last_location, ender_chest, off_hand]}
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core] Multiverse-Core Version: 2.6.0-SNAPSHOT-b736
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core] Bukkit Version: git-Spigot-5695bca-53fccdf (MC: 1.12.2)
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core] Loaded Worlds: [{"MVWorld@248304490":{"Gen":"null","Type":"NORMAL","Env":"THE_END","Name":"world_the_end"}}, {"MVWorld@1466964761":{"Gen":"null","Type":"NORMAL","Env":"NORMAL","Name":"world"}}, {"MVWorld@1340554846":{"Gen":"null","Type":"NORMAL","Env":"NORMAL","Name":"Creative"}}, {"MVWorld@740268914":{"Gen":"null","Type":"FLAT","Env":"NORMAL","Name":"PixelMap"}}, {"MVWorld@1668123099":{"Gen":"null","Type":"NORMAL","Env":"NETHER","Name":"world_nether"}}, {"MVWorld@714176366":{"Gen":"PlotSquared","Type":"NORMAL","Env":"NORMAL","Name":"PixelMapTwo"}}, {"MVWorld@48304298":{"Gen":"null","Type":"FLAT","Env":"NORMAL","Name":"staff_creative"}}, {"MVWorld@327531482":{"Gen":"null","Type":"FLAT","Env":"NORMAL","Name":"Event"}}, {"MVWorld@884419041":{"Gen":"null","Type":"FLAT","Env":"NORMAL","Name":"pvp"}}, {"MVWorld@910201301":{"Gen":"null","Type":"NORMAL","Env":"NORMAL","Name":"RolePlay"}}]
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core] Multiverse Plugins Loaded: 2
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core] Economy being used: Essentials Economy
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core] Permissions Plugin: Bukkit Permissions (SuperPerms)
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core] Dumping Config Values: (version 2.9)
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core]  messagecooldown: 10000
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core]  teleportcooldown: 10000
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core]  worldnameprefix: false
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core]  worldnameprefixFormat: [%world%]%chat%
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core]  enforceaccess: true
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core]  displaypermerrors: true
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core]  teleportintercept: true
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core]  firstspawnoverride: true
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core]  firstspawnworld: world
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core]  debug: 0
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Core] Special Code: FRN002
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Portals] Multiverse-Portals Version: 2.5.2-SNAPSHOT-b762
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Portals] Loaded Portals: 19
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Portals] Dumping Portal Values: (version 2.7)
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Portals]  wand: 271
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Portals]  useonmove: true
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Portals]  enforceportalaccess: true
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Portals]  portalsdefaulttonether: false
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Portals]  portalcooldown: 10000
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Portals]  clearonremove: false
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Portals] Special Code: FRN001
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Inventories] Multiverse-Inventories Version: 2.5.0-SNAPSHOT-b443
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Inventories] First Run: false
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Inventories] Using Bypass: false
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Inventories] Default Ungrouped Worlds: false
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Inventories] Using GameMode Profiles: false
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Inventories] === Groups ===
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Inventories] default: {Worlds: [world, world_the_end, world_nether, pvp], Shares: [hit_points, economy, food_level, saturation, exhaustion, xp, total_xp, lvl, inventory_contents, armor_contents, bed_spawn, maximum_air, remaining_air, fall_distance, fire_ticks, potion_effects, last_location, ender_chest, off_hand]}
[20:42:49] [Server thread/INFO]: [Multiverse-Core] [Multiverse-Inventories] CreativeMaps: {Worlds: [staff_creative, event, creative, pixelmap], Shares: [hit_points, economy, food_level, saturation, exhaustion, xp, total_xp, lvl, inventory_contents, armor_contents, bed_spawn, maximum_air, remaining_air, fall_distance, fire_ticks, potion_effects, last_location, ender_chest, off_hand]}

mv-inv config:
# Multiverse-Inventories Settings

# ===[ Multiverse Inventories Config ]===
settings:
  # This is the locale you wish to use.
  locale: en
  # If this is true it will generate world groups for you based on MV worlds.
  first_run: false
  # If this is set to true, it will enable bypass permissions (Check the wiki for more info.)
  use_bypass: false
  # If set to true, any world not listed in a group will automatically use the settings for the default group!
  default_ungrouped_worlds: false
  # The default and suggested setting for this is FALSE.
  # False means Multiverse-Inventories will not attempt to load or save any player data when they log in and out.
  # That means that MINECRAFT will handle that exact thing JUST LIKE IT DOES NORMALLY.

  # The biggest potential drawback here is that if your server crashes, player stats/inventories may be lost/rolled back!
  save_load_on_log_in_out: false
  # If this is set to true, players will have different inventories/stats for each game mode.
  # Please note that old data migrated to the version that has this feature will have their data copied for both game modes.
  use_game_mode_profiles: false
shares:
  # When set to true, optional shares WILL be utilized in cases where a group does not cover their uses for a world.
  # An example of this in action would be an ungrouped world using last_location.  When this is true, players will return to their last location in that world.
  # When set to false, optional shares WILL NOt be utilized in these cases, effectively disabling it for ungrouped worlds.
  optionals_for_ungrouped_worlds: true
  # You must specify optional shares you wish to use here or they will be ignored.
  # The only built in optional share is "economy"
  use_optionals: []

moved to Multiverse Inventories Issue #322

Verified this only happens during server restarts when player was last in the not-main inventory group, so either a startup or shutdown routine that copies to/from the .json files appears to be responsible for this issue... adds funny characters before all color codes or after line breaks in written books.

resolved as character set issue?
Change character set in use to en-US.utf-8 to resolve issue
locale  <-- command to see current setting
enter these two commands to change character set temporarily...
LANG=en_US.utf-8
LANGUAGE=en_US.utf-8:
Make the changes permanent, browse to /etc/default/locale and change contents to:
/etc/default/locale
LANG="en_US.utf-8"
LANGUAGE="en_US.utf-8:"
Reboot and all should be well going forward (any previously glitched stuff will still be glitched so... rip).

https://github.com/Multiverse/Multiverse-Inventories/issues/322 not #322

thanks to SliimeDog on Discord for this solution.


Model grammar support via BNF

We will implement based on this.
https://crates.io/crates/bnf

@EricLBuehler do you have further steps of it planned?
Looking at similar designs:
Outlines generates a FSM from regex (using https://github.com/MegaIng/interegular), transforms the FSM from text-based to token-based https://github.com/outlines-dev/outlines/blob/main/outlines/fsm/regex.py#L47 which is later used to influence the logits by vLLM: https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/guided_logits_processors.py#L28
https://github.com/sgl-project/sglang also uses Outlines FSM but SGLang also implements a jump-forward optimization mechanism https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/constrained/jump_forward.py#L8 https://lmsys.org/blog/2024-02-05-compressed-fsm/
Llama.cpp seems to compute the state of the FSM as it walks it https://github.com/ggerganov/llama.cpp/blob/master/llama.cpp#L12510 after building this data structure https://github.com/ggerganov/llama.cpp/blob/master/llama.cpp#L11954 after parsing the grammar

vLLM roadmap also shows they plan to integrate lm-format-encoder, that does not pre-build the FSM https://github.com/vllm-project/vllm/issues/3713
And also AICI https://github.com/vllm-project/vllm/pull/2888
AICI is a Rust library, I wonder if it wouldn't be simpler for mistral.rs to implement it, or implement it alongside other setups.

@lucasavila00, I plan on working on this next week. Would you be able to put together a skeleton PR? This looks like an exciting development.

Sure, I'll try to integrate AICI over the weekend.

Implementing AICI has shown to be too complex for me to build quickly.
What I learned on https://github.com/EricLBuehler/mistral.rs/pull/81 :

AICI is too young, there is no library published to crates.io, the protocol might change (eg: https://github.com/microsoft/aici/discussions/68) and so on
AICI supports fast forwarding and it is complicated to implement. SGLang does fast-forwarding by appending text, re-tokenizing it and running a new request. SGLang has RadixAttention where it can cache the requests very granuarly, so for SGLang the only overhead is re-tokenization.
Configuring the side channel messages is trivial

To run the server on that PR use
./target/release/mistralrs-server --aicirt=PATH_TO_AICIRT --port 1234 --log target/output.txt mistral-gguf

where PATH_TO_AICIRT is the binary from Github Releases https://github.com/microsoft/aici/releases/tag/v0.0.10
Use mistral-gguf because I hardcoded some settings instead of building the code that reads them from the config.json, tokenizer.json.

Thank you for working on this! We will probably begin work on this early next week.
Like you said, AICI is pretty complicated, and we will probably not use it. I will look into the SGLang method you mentioned. If you have any ideas for implementing grammars, please let me know!

I have this https://github.com/lucasavila00/LmScript project that supports both vLLM and SGLang. So I really only know about these.
The RadixAttention used by SGLang makes it faster to use prompts that generate many small pieces. For example, an ad-hoc structured XML generation https://github.com/lucasavila00/LmScript/blob/main/packages/client/src/backends/executor.ts#L78-L136
Also, SGLang has 2 different select-from-choices operations.
If one uses a regex with multiple options the backend will eagerly-per-token select the most probable choice. This might be unintuitive (eg: https://github.com/guidance-ai/guidance/issues/564)
If one uses the native SGLang select operation then it calculates the probability for each of the options as whole: https://github.com/sgl-project/sglang/blob/ff99c38a0711ee82926840129db840a70e91f0d9/python/sglang/backend/runtime_endpoint.py#L191-L242
SGLang select is amazing regarding result quality, and it is implemented with just a few extra settings to the server, that tell the server which tokens it should return the logprobs of. Of course, this only works efficiently because the RadixAttention cache can re-use the computation of the common prefix.
I like SGLang a lot. The only issue is that it takes a long time to pre-compile the regexes, which are then saved to disk and available for re-use. I agree with https://github.com/vllm-project/vllm/issues/3713 "For endpoint products running model as a service with customers supplying many different schemas, the cost might not be acceptable."
I would like if SGLang could compile regexes fast or reject them due to excessive complexity, that's the major flaw I see with it. Besides bad error messages when it receives invalid regexes and so on.
I did look into rust's regex-automata for the regex compilation https://github.com/rust-lang/regex/blob/master/regex-automata/src/dfa/mod.rs#L265-L270 but as the linked line says it is hard to make this compilation efficient.

Ah, SGLang doesn't do token healing. Only Guidance does https://towardsdatascience.com/the-art-of-prompt-design-prompt-boundaries-and-token-healing-3b2448b0be38
It would be amazing if SGLang's design also had token healing.
It does require a lot of changes to be supported though, one needs to remove parts of the initial prompt and then let the model re-generate the removed part.
If this is a desired feature, you might want to keep it in mind while building the other parts to make an eventual implementation easier.

Other interesting things I found:
Latest version of TGI https://github.com/huggingface/text-generation-inference also uses Outlines FSM, supports regex decoding
Llama.cpp has performance issues on its grammar implementation https://github.com/ggerganov/llama.cpp/issues/4218#issuecomment-1836540046
Kalosm implemented regex-based decoding using the regex-automata DFA, as I mentioned above https://github.com/huggingface/candle/issues/1945#issuecomment-2027242005
vLLM has cache prefix now https://github.com/vllm-project/vllm/issues/2614 (Unfortunately I couldn't test it as my RTX2070 is too old for it, so I can't tell if this works as well as SGLang Radix Attention. I'm in the process of replacing the GPU and I should be able to compare the approaches in a week or so)

Thanks for the links. I really like the idea of using BNF, maybe it could be converted to a regex. After looking at the Kalosm implementation and this issue, I think we could implement this with the logit bias and just parse the BNF. Reading the llama.cpp issue, I think there are a few considerations we need to take into account when implementing:

Use a polynomial bounded time parsing algorithm
Run normal sampling and if that returns a token that does not match the grammar, only then spend the time computing the grammar logit bias

Potentially, we could use a regex DFA to handle the matching. What do you think about that plan?
Regarding prompt caching, I think that is workable, as we could just implement an eviction policy for KV caches. However, that is a separate topic, and perhaps you could raise a tracking issue for it?

I really like the idea of using BNF

Outlines has an example of .lark files. Terminals are regexes and one can build the parser on top of the terminals, https://github.com/outlines-dev/outlines/blob/main/outlines/grammars/json.lark https://github.com/outlines-dev/outlines/blob/main/outlines/grammars/common.lark https://github.com/outlines-dev/outlines/blob/main/outlines/fsm/guide.py

Potentially, we could use a regex DFA to handle the matching. What do you think about that plan?

Generating the FSM for a "big" regex expression is slow. This SGLang example https://github.com/sgl-project/sglang?tab=readme-ov-file#json-decoding of a JSON object with ~10 fields takes a minute to compile.
A mixture of an "interpreter" for the BNF/gramamar, where each terminal is a compiled regex could work. We would cache the regexes FSM and build big JSON objects without a big compilation delay. But I'm not sure if this is feasible to implement in token-space...

Regarding prompt caching, I think that is workable, as we could just implement an eviction policy for KV caches. However, that is a separate topic, and perhaps you could raise a tracking issue for it?

Sure.

True, compiling a big BNF would be very costly, I like the idea of only compiling the terminals. I wonder if we can multithread the logit bias calculation with a thread pool, we can do something like:

https://github.com/EricLBuehler/mistral.rs/blob/067316b7d0d9c57a4a286e5b65b9a1691eae243a/mistralrs-core/src/utils/varbuilder_utils.rs#L36

And then accumulate them:

https://github.com/EricLBuehler/mistral.rs/blob/067316b7d0d9c57a4a286e5b65b9a1691eae243a/mistralrs-core/src/utils/varbuilder_utils.rs#L135-L139

The vocab size for a Mistral model is 32000, so iterating over that would be expensive in the sampling hotloop! We are currently ~3 ms/T slower than llama.cpp on an A10 + Mistral Q4_K_M GGUF, so I want to improve performance.

On performance, I think the explanation of the AICI protocol is good:
...
- the LLM schedules some of the non-supended sequences to compute logits for
- the LLM informs AICIrt about the scheduled sequences;...
- the LLM starts computing logits
- AICIrt sends the logit biases to the LLM
- LLM adds computed logits and biases and samples tokens
...

It should be done in a different thread, concurrently to the logit calculation, and it should not start calculation at sampling time.
According to AICI, if done this way, there is little cost: https://github.com/microsoft/aici?tab=readme-ov-file#performance
For example, computing allowed token set in the 32000-strong vocabulary of Llama model takes:

- about 2.0ms for Yacc grammar of the C programming language
- about 0.3ms for a regular expression
- about 0.2ms for a substring constraint, from 4kB string

Ah, that is great: that way the logit biases are ready before the logits are! Even with this, I think it would still be best to sample w/o applying the logit biases first as an optimization, as that needs to be sequential and may iterate over most of the vocab size. In fact, while the initial sampling is occurring a worker thread can apply the logit bias.
If you want to open a PR laying the groundwork for some of this, such as the logit bias application and dual-sampling setup, please feel free! I am hoping to implement grammar support this week.

Awesome!
I'll have a busy week. I fear I'll only be available next weekend.
Once I have the time I'll look around for what I can help with.

I was reading a bit more about DFA, FSM and so on thinking about how to implement this, and I stumbled upon the details of the AICI ABI:
https://github.com/microsoft/aici/tree/main/controllers/aici_abi
It works pretty much like Kalosm where it runs the regex for every logit. However, they do it on a token trie, so if tokens share a common prefix they won't require re-computation.
They provide the grammar: https://github.com/microsoft/aici/tree/main/controllers/aici_abi#lr1-grammars
Regex: https://github.com/microsoft/aici/tree/main/controllers/aici_abi#regular-expressions
And low level API like required in the sampler: https://github.com/microsoft/aici/tree/main/controllers/aici_abi#regular-expressions
It requires an instance of TokTree that can be built like I did in the previous AICI MR https://github.com/EricLBuehler/mistral.rs/pull/81/files#diff-8e7ab085145c61f5613962a6b30db55a222fa6bf6e432e82df9ab90dbfb4627aR900-R903
And an instance of the Recognizer, like this regex https://github.com/microsoft/aici/blob/main/controllers/aici_abi/src/rx.rs#L20 or grammar https://github.com/microsoft/aici/blob/main/controllers/aici_abi/src/cfg.rs
@EricLBuehler what are your thoughts?
To me it looks like we can do this by adding aici-abi as a dependency. (notice that the previous MR also added aici-runtime, which I'm not proposing to add anymore)
We could also start with copying the code and editing it. However after reading it for a bit I think using it as a dependency should be good enough...

I also published kalosm-sample as a separate library if you want to re-use the logic. It currently walks the dfa as it generates tokens so each individual token will take a constant amount of time to validate regardless of the generation length. If you are generating a series of different but similar sequences I would be happy to merge support for a prompt cache.
One technique kalosm uses that I haven't seen mentioned here is constraint accelerated batching. If you see that the next n tokens must be a specific string, you can load all of those tokens into the kv cache at once in a larger batch to accelerate generation

@lucasavila00, I think it would be good to do something similar to the token trie + kalosm-style regex matching.
@ealmloff, thanks for the link! If you would be able to merge support for a prompt cache, that would be much appreciated! I was wondering, in the kalosm internals, do you apply the Parser to every token in the vocab to make a logit bias, or do you do someting else?

I was wondering, in the kalosm internals, do you apply the Parser to every token in the vocab to make a logit bias, or do you do something else?
Currently, yes I do that here. If you use top_k, you could only parse tokens until you get at least k tokens instead of every logit

Implemented in #103, thank you @lucasavila00 !


Exclude doesn't support individual resources

AFAIK there is no way to exclude just one resource (individual VM or instance) using the exclude parameter: is this expected ?
E.g: I tried the following in a rule to prevent public IPs on CloudSQL, but the exclude is not working:
match:
    target: ["organization/*"]
    exclude: ["//cloudsql.googleapis.com/projects/<PROJECT>/instances/my-db"]

or
match:
    target: ["organization/*"]
    exclude: ["organization/<ORG_ID>/folder/<FOLDER_NUMBER>/project/<PROJECT_NUMBER>/cloudsqlinstance/<INSTANCE_NUMBER>"]

but excluding on the project level works (but might be inadequate for certain contexts):
match:
    target: ["organization/*"]
    exclude: ["organization/<ORG_ID>/folder/<FOLDER_NUMBER>/project/<PROJECT_NUMBER>"]

Correct, projects are the most granular unit you can exclude (similar to the target semantics).


chore: bump spin version to 1.5.0

Bump spin 1.4.1 -> 1.5.0.

@cpuguy83 @devigned PTAL

IIUC, there's a problem building a dependency introduced by the new LLM feature of spin.

Ok, I digged a bit into it.
What's failing is building llama-cpp in aarch64 musl.
It builds fine with gnu ABI adding the -Ctarget-feature=+fp16 (what upstream spin does).
But it fails with musl ABI.
There are a few diferences, including the gcc version that cross uses for those targets.
The main issue is that llama-cpp is using the  vld1q_u8_x4 and vld1q_s8_x4 functions, which should be SIMD intrinsics exposed by the compiler.
According to this issue https://github.com/xmrig/xmrig/pull/2882, that intrinsic is not present in gcc <= 11.
I am wondering what the right fix is, since spin doesn't provide a feature flag to disable llm.

Need a rebase

minor comment: add a readme on polyfill folder to describe what this does and when shall we remove it.

If there's an update to the compiler in cross-rs, the polyfill will become a no-op as it does a feature test before acting.
I left a link to the upstream issue in spin.
There are many different solutions, off the top of my head I can think of:

a feature gate in spin
a compiler update in cross
replacing cross with something like xx, similar to what we do in runwasi (although cross is really nice, but would also allow us to build with systemd and seccomp support here)

sorry, changed laptops and hadn't set up git correctly!
it's signed now :-)


Potential fix for the failed code quality check comment workflow

What does this PR do?

It seems there is an issue with actions/download-artifact@v4, now doing something similar to https://github.com/huggingface/doc-builder/blob/fcea09086f6dec68ef9ce419f164c6c33e8fb5f4/.github/workflows/upload_pr_documentation.yml#L48.
Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests?

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


B314 since Python 3.6 is not valid

Describe the bug
I checked the python module xml processing and the vulns that bandit report is no longer there. maybe you should remove it:

xml.etree.ElementTree.parse
xml.etree.ElementTree.iterparse
xml.etree.ElementTree.fromstring
xml.etree.ElementTree.XMLParser
https://docs.python.org/3/library/xml.html#xml-vulnerabilities
I also checked billion laughs and quadratic blowup no longer there even the doc above said so.

Reproduction steps

Using xml.etree

import xml.etree.ElementTree as ET
import xml as xml

tree = ET.parse('country_data.xml')
print(tree.findtext('body'))

output:
>> Issue: [B314:blacklist] Using xml.etree.ElementTree.parse to parse untrusted XML data is known to be vulnerable to XML attacks. Replace xml.etree.ElementTree.parse with its defusedxml equivalent function or make sure defusedxml.defuse_stdlib() is called
   Severity: Medium   Confidence: High
   CWE: CWE-20 (https://cwe.mitre.org/data/definitions/20.html)
   More Info: https://bandit.readthedocs.io/en/1.7.5/blacklists/blacklist_calls.html#b313-b320-xml-bad-elementtree
   Location: main.py:5:7
4
5       tree = ET.parse('country_data.xml')
6       print(tree.findtext('body'))

Expected behavior
Maybe remove [B314] or explain your concern there.
Bandit version
1.7.5 (Default)
Python version
3.11
Additional context
No response

So after some digging on this topic. Seems Python has a vendored version of expat embedded. This is the default version (at least for macOS and Windows). However, it does seem permitted
to use an external system level version. So maybe there is still a window of vulnerability there? In any case, the latest minor versions of Python 3.8 to 3.12 all vendor expat 2.5.0 now.
So it might make sense to deprecate or remove this check.

To see the vendored version of expat embedded in Python you can browse to:
https://github.com/python/cpython/blob/3.8/Modules/expat/expat.h#L1056C20-L1056C20
At the bottom of this header file you'll find the expat major, minor, and micro version.

However, I also found that the oldest version of Python 3.8, 3.8.0, which Bandit officially still supports does include an older, vulnerable version of expat:
https://github.com/python/cpython/blob/v3.8.0/Modules/expat/expat.h#L1016

Expat 2.4.1 wasn't added to Python 3.8 till 3.8.12. Similar for 3.9, not until 3.9.7.

I also checked  the latest patch for python 3.6 (3.6.15) and 3.7 (3.7.17), both are safe. only from (3.5 <=) latest patch didn't include the fix. I couldn't find their repo branches that I can refer to, but I just tested by running all possible 5 possible xml vulns.
might be useful to know.


Converting a SBOM looses the algorithm type for added checksums

What happened:
I was working on a project to test diffrent SBOM converters and also looked into Syft. I think you do a really good job, better then most other tools that provide converters for SBOMs.
One thing that looked like a simple bug to me was, that syft looses the algorithm of a checksum when converting. This only happens while converting from SPDX to CycloneDx. From CycloneDx to SPDX the value is mapped correctly.
Input:
{
  "name": "commons-compress",
  "SPDXID": "SPDXRef-Package-java-archive-commons-compress-591e913d9a6a50d4",
  "checksums": [
    {
      "algorithm": "SHA1",
      "checksumValue": "4af2060ea9b0c8b74f1854c6cafe4d43cfc161fc"
    }
  ],
  "sourceInfo": "acquired package info from installed java archive: /usr/share/jenkins/jenkins.war",
  "versionInfo": "1.23.0",
  "externalRefs": [
    {
      "referenceType": "cpe23Type",
      "referenceLocator": "cpe:2.3:a:apache:commons-compress:1.23.0:*:*:*:*:*:*:*",
      "referenceCategory": "SECURITY"
    },
    {
      "referenceType": "cpe23Type",
      "referenceLocator": "cpe:2.3:a:apache:commons_compress:1.23.0:*:*:*:*:*:*:*",
      "referenceCategory": "SECURITY"
    },
    {
      "referenceType": "cpe23Type",
      "referenceLocator": "cpe:2.3:a:apache:compress:1.23.0:*:*:*:*:*:*:*",
      "referenceCategory": "SECURITY"
    },
    {
      "referenceType": "cpe23Type",
      "referenceLocator": "cpe:2.3:a:apache:commons:1.23.0:*:*:*:*:*:*:*",
      "referenceCategory": "SECURITY"
    },
    {
      "referenceType": "purl",
      "referenceLocator"<EMAIL_ADDRESS>      "referenceCategory": "PACKAGE-MANAGER"
    }
  ],
  "copyrightText": "NOASSERTION",
  "filesAnalyzed": false,
  "licenseDeclared": "LicenseRef-https---www.apache.org-licenses-LICENSE-2.0.txt",
  "downloadLocation": "NOASSERTION",
  "licenseConcluded": "NOASSERTION"
}

Output:
{
  "bom-ref": "pkg:maven/org.apache.commons/commons-compress@1.23.0?package-id=218cd71a5496eeb9",
  "type": "library",
  "name": "commons-compress",
  "version": "1.23.0",
  "licenses": [
    {
      "license": {
        "name": "LicenseRef-https---www.apache.org-licenses-LICENSE-2.0.txt"
      }
    }
  ],
  "cpe": "cpe:2.3:a:apache:commons-compress:1.23.0:*:*:*:*:*:*:*",
  "purl"<EMAIL_ADDRESS>  "externalReferences": [
    {
      "url": "",
      "hashes": [
        {
          "alg": "",
          "content": "4af2060ea9b0c8b74f1854c6cafe4d43cfc161fc"
        }
      ],
      "type": "build-meta"
    }
  ],
  "properties": [
    {
      "name": "syft:package:language",
      "value": "java"
    },
    {
      "name": "syft:package:metadataType",
      "value": "JavaMetadata"
    },
    {
      "name": "syft:package:type",
      "value": "java-archive"
    },
    {
      "name": "syft:cpe23",
      "value": "cpe:2.3:a:apache:commons_compress:1.23.0:*:*:*:*:*:*:*"
    },
    {
      "name": "syft:cpe23",
      "value": "cpe:2.3:a:apache:compress:1.23.0:*:*:*:*:*:*:*"
    },
    {
      "name": "syft:cpe23",
      "value": "cpe:2.3:a:apache:commons:1.23.0:*:*:*:*:*:*:*"
    }
  ]
}

Round Trip
  {
  "name": "commons-compress",
  "SPDXID": "SPDXRef-Package-java-archive-commons-compress-218cd71a5496eeb9",
  "versionInfo": "1.23.0",
  "downloadLocation": "NOASSERTION",
  "checksums": [
    {
      "algorithm": "",
      "checksumValue": "4af2060ea9b0c8b74f1854c6cafe4d43cfc161fc"
    }
  ],
  "sourceInfo": "acquired package info from installed java archive: ",
  "licenseConcluded": "LicenseRef-LicenseRef-https---www.apache.org-licenses-LICENSE-2.0.txt",
  "licenseDeclared": "LicenseRef-LicenseRef-https---www.apache.org-licenses-LICENSE-2.0.txt",
  "copyrightText": "NOASSERTION",
  "externalRefs": [
    {
      "referenceCategory": "SECURITY",
      "referenceType": "cpe23Type",
      "referenceLocator": "cpe:2.3:a:apache:commons-compress:1.23.0:*:*:*:*:*:*:*"
    },
    {
      "referenceCategory": "SECURITY",
      "referenceType": "cpe23Type",
      "referenceLocator": "cpe:2.3:a:apache:commons_compress:1.23.0:*:*:*:*:*:*:*"
    },
    {
      "referenceCategory": "SECURITY",
      "referenceType": "cpe23Type",
      "referenceLocator": "cpe:2.3:a:apache:compress:1.23.0:*:*:*:*:*:*:*"
    },
    {
      "referenceCategory": "SECURITY",
      "referenceType": "cpe23Type",
      "referenceLocator": "cpe:2.3:a:apache:commons:1.23.0:*:*:*:*:*:*:*"
    },
    {
      "referenceCategory": "PACKAGE-MANAGER",
      "referenceType": "purl",
      "referenceLocator"<EMAIL_ADDRESS>    }
  ]
}

What you expected to happen:
It feels like a bug that the algorithm type is lost. Looks like CycloneDx and SPDX both support this field.
Steps to reproduce the issue:
This are the command and SBOMs i used:
Command:
syft convert original.syft.spdx.json -o cyclonedx-json=converted.bySyft.toCdx.json
syft convert converted.bySyft.toCdx.json -o spdx-json=roundTrip.bySyft.toSpdx.json

sboms.zip
Anything else we need to know?:
I put some notes together in a blogpost about my tests with all tools. There are more minor things that were not mapped correctly but this issue feels like a bug while the others more look like it's hard to map all SPDX and CycloneDx fields. And you guys already stated that converting with Syft is experimental and will lose data. So i guess you already know about that.
https://mariuxdeangelo.gitlab.io/website/#/post/20230925-SBOM-Convertion-Tools
Environment:

Output of syft version:
OS (e.g: cat /etc/os-release or similar):

Thanks for reporting! An initial glance shows that we need to consider casing when we are writing out cyclonedx documents https://github.com/anchore/syft/blob/38d5ef2c84e9bb8245c657daf2a3f97facde0a8f/syft/formats/common/cyclonedxhelpers/external_references.go#L95C46-L95C46

Hi! I'm going to try to fix this issue if that's alright.


Prompt buffer - Ctrl-Shift-W doesn't delete the word

Steps to reproduce
:h prompt-buffer. The documentation says

The CTRL-W key can be used to start a window command, such as CTRL-W w to
switch to the next window.  This also works in Insert mode (use Shift-CTRL-W
to delete a word). When leaving the window Insert mode will be stopped.  When
coming back to the prompt window Insert mode will be restored.

Here is how to reproduce

vim
:new | set buftype=prompt | startinsert
Type any word and then try to use Ctrl-Shift-W to delete it(while being in the insert mode)
This doesn't delete the previous word

Expected behaviour
It should delete the previous word, but it doesn't. Pressing Ctrl-Shift-W twice switches the window.
So essentially Ctrl-Shift-W is behaving exactly like Ctrl-W
Version of Vim
VIM - Vi IMproved 9.0 (2022 Jun 28, compiled Aug  5 2023 11:23:13) Included patches: 1-1677
Environment
Ubuntu 22.04.3
GNOME Terminal - Version 3.44.0 for GNOME 42
bash - GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)
echo $TERM
screen-256color
Logs and stack traces
N/A

GNONE Terminal doesn't support Ctrl-Shift-W

can you plese verify, if this works in xterm or the gui version?

@chrisbra I can confirm that this works with xterm. Couldn't check on gvim, since I don't have it installed(and no root access to install it either)
I think we can close this?

GNONE Terminal doesn't support Ctrl-Shift-W

Not only that, but as well judging by the value of TERM environment variable it all happens under gnu-screen terminal multiplexer - and gnu-screen in between would not allow vim to interact even with xterm on such a level that CTRL+SHIFT+W can be distinguished from CTRL+W.


[Doc] Add explanations for inactive topics

Fix https://github.com/apache/pulsar/issues/10932

@MarvinCai could you please help review? Thanks


Migrate to Go Modules

This PR migrates the package to Go Modules and updates the path according to Semantic Import Versioning
Generated by https://github.com/marwan-at-work/mod

@marwan-at-work why is this a hard requirement? we shouldn't be changing the path of the directories where the packages are stored in this manner. It is not correct and prone to import issues.
Let's not take this PR in until the Go modules has been moved away from experimental and we don't want any dependency on the mod tool. We prefer if Go provided a mechanism already natively through their own tooling.

@harshavardhana totally understandable to wait until Go modules is out of experimental :) feel free to close the PR.
Just FYI, Go Modules is backwards compatible with the latest Go 1.9 and 1.10. In other words, it can still be imported without the /v6 and Go would resolve it correctly. And of course, you'd need to tag this with v6.x.x because according to Semantic Import Versioning, the git tag and the import path must match exactly on the major part.

For now, let's close this, we will revisit this when we move to go1.11 modules.

We will move when go modules become stable and fully supported.


Update DRP products from 1.1p to 1.2p in tutorial notebooks

Update the butler repo in tutorial notebooks to point to 1.2p in tutorial notebooks.
I think this upgrade can be carried out now, am I right, @wmwv?
(cc @reneehlozek as this is one of the hackuDC2 preparation task)

Yesish.
We need to provide a Jupyter kernel that provides the correct shifter image and obs_lsstCam version of the Run 1.2p processing.  @heather999

@yymao @heather999 I confirm that a recent version of the stack: w_2018_39 can still read the Run1.1p data.
The key difference is that Run 1.1p obs_lsstSim camera mapper, while Run 1.2p uses the obs_lsst[Cam] mapper.  There's nothing that intentionally broken things for obs_lsstSim, but that package is not CIed.
There's no reason this shouldn't have been true, but checking makes me feel much more reassured.

We have an initial shifter image that will handle the ongoing Run1.2p reprocessing with w_2018_30+a version of obs_lsstCam.  I'm updating that shifter image to include the python packages that Michael requested.  Once that image is installed at NERSC, then we need to add an additional script that will allow users to add this new image to a jupyter kernel.

@heather999 @wmwv thanks. I have two may-be-naive questions:

Does this mean that we will have a new kernel other than the desc-stack that we have been asking people to use? Can we just update this kernel instead of creating a new one?
(on a different note) what's the path of the 1.2p repo for butler?

That's the question.. can we just rename the "desc-stack" kernel? I suppose if the new one is really compatible with Run1.1p, it is ok... but at some point that I expect we will diverge and a new kernel may not be compatible with older data.  I'd like to test to see what happens when we execute "jupyter kernelspec install"and the kernel name already exists in the user's $HOME/.local/share/ area
The Run1.2p data is appearing here (which is a symlink to the projecta area, but I understand that the compute nodes and projecta space + butler do not work well together, so we have link to the scratch area):  /global/cscratch1/sd/desc/DC2/in2p3/Run1.2p/w_2018_30

I set up a desc-stack-run1.2p kernel, unfortunately I am unable to log into jupyter-dev, even with MFA set up on my account. While I have opened a ticket, given that it is the weekend, I don't expect to hear from NERSC until tomorrow. There is a new shifter image for w_2018_30 called lsstdesc/stack-jupyter:w_2018_30-run1.2p. I’ve created a new “dev” setup file while we test things out: /global/common/software/lsst/common/miniconda/kernels/setup-dev.sh This should set up a new kernel called desc-stack-run1.2p.  Perhaps someone else can try running setup-dev.sh to see if a new kernel is successfully installed that can be run on jupyter-dev.

Closed by #29.


SPFx 1.15 - Getting error when use form customiser to view/edit/new - in Australia East Region

Target SharePoint environment
SharePoint Online
What SharePoint development model, framework, SDK or API is this about?
💥 SharePoint Framework
Developer environment
Windows
What browser(s) / client(s) have you tested

[ ] 💥 Internet Explorer
[ ] 💥 Microsoft Edge
[ ] 💥 Google Chrome
[ ] 💥 FireFox
[ ] 💥 Safari
[ ] mobile (iOS/iPadOS)
[ ] mobile (Android)
[ ] not applicable
[ ] other (enter in the "Additional environment details" area below)

Additional environment details

browser version
SPFx version
Node.js version
etc

Describe the bug / error
I deployed a Form customiser and associated component Id to the content type for new, edit and display. After that, getting the below error while trying to create a new item/edit an item/view an item.

Before associating content type, the form works without any issues on debug mode (gulp serve). But after content type association, getting the same error in debug mode.
Looks like an issue with Form customiser in our region, which is Australia East.
I saw a similar issue reported with classic mode before(
https://github.com/SharePoint/sp-dev-docs/issues/8272). But this issue exists even in modern view.
Could you please have a look and help to get this fixed
Steps to reproduce

Create a SharePoint List and associate a content type
Create a Form customiser extension and deploy to the SharePoint site.
Test the customiser in debug mode to see everything works alright
Update either or all of the NewFormClientSideComponentId ,  EditFormClientSideComponentId , DisplayFormClientSideComponentId  of the  content type to the component id of the Form customiser component.
Test the customiser in debug mode and it will not work
Check the New/Edit/Disp form of the list and it will not work.

Expected behavior
The form loads up both at the list form level and in debug mode after associated to content type

Updated screen shot with correlation ID

Hello @harinvp,
Sorry for the long delay!
Could you provide more details?
I've just tested simple form customizer and it's working perfectly in Australia tenant. Both when deployed in production and debug modes.
Are you installing it to tenant app catalog or site collection app catalog?
Are you using it with list or document library?
Are there any properties assigned to the form customizer?
What approach are you using to set the properties of the content type?


Jay - Refactor Documentation + Refactor Solution

fixed in #46

Accidentally closed. Needs to refactor 2 functions.

Apparently we only need to refactor one function...


npm install -dev Error: No compatible version found<EMAIL_ADDRESS><0.3.0-'

In my package.json file I moved zombie from "dependencies" to "devDependencies" (since I use it for dev test). Then I tried running "npm install -dev", which always resulted in the aforementioned error/failure.
I've tried this on OSX with Node 0.5.7 + NPM 1.0.30, and also Node 0.6.6 + NPM 1.1.0-beta-4. Same error in both cases.
It installs fine if I have it in my package.json as a regular dependency (vs. devDepencies).

npm install installs everything list in devDependencies.
npm install --dev installs all dev dependencies recursively, so it will try to install nodeunit because it's a dev dependency of JSDOM, itself a dependency of Zombie.
Zombie itself doesn't depend on Jake.

Ah! Thanks for explaining.


Add update command to cli

Need to add an update command to the available cli command, e.g.,
bash$ openfido update

run the install command to update the install scripts.

Right now update only updates the pipelines, not the CLI itself.


ARROW-207: Extend BufferAllocator interface to allow decorators around BufferAllocator

Currently AllocationManager needs BufferAllocator instance reference for accounting purposes.
Having a decorator around the BufferAllocator disturbs the accounting in AllocationManager.
Add an unwrap method to allow getting the reference to inner BufferAllocator instance and use
this method in AllocationManager to get the actual BufferAllocator.
/**

Unwrap the class so that exposes the provided interface, if possible. Otherwise, throw Exception.
@param c

     The class or interface that you want this class to implement/extend.

@return The instance of that class related to 'this'
*/
 T unwrap(Class c);

@StevenMPhillips: Could you please review this patch?

If AllocationManager assumes the the BufferAllocator is a specific implementation class, what is the point of the interface? It seems to me we should either fix the interface or get rid of it.

I agree, current AllocationManager depends on the implementation of BufferAllocator, BaseAllocator, than the interface.

I suggest we abandon this and come up with a different approach for the extensibility you're trying to achieve.


Modified README.md file.

Issue Resolved:

Issue #134

1. Modifications in README.md file
1) Add animated line separators

2) Added GSSoC 2024 Extended & Hacktoberfest 2024 Acceptance banners.

3) Modified the Contributors Mention Section. [Faced a conflict while working on this]

4) Replaced the Contact section with the new Team section in both the README.md and CONTRIBUTING.md files

5) Added a redirect to the CONTRIBUTING.md file from the README.md file

Reference Video:
https://github.com/user-attachments/assets/e189dea5-6084-4565-a841-e6cbed4a573b

Please review the PR and assign the PR to me along with the necessary labels.

@UTSAVS26 Please close the issue #134 as the PR has been merged


add support for --env-file command line flag

the --set flag works well for passing in a few vars. This gets unwieldy for more than a few vars tho.
Docker supports the --env-file flag which would load a file of VAR=value pairs.
Would ya'll be open to adding something like this to apex? seems like the vars are already in a file somewhere in the lambda zip

That's right env vars are stored in separate file inside zip but I'm not sure if we should add possibility to specify that file. I'm still +1 for possibility to specify --env-file though.

Hmm my opinion there is that if you just have a big list of config (as json or similar), you might as well just require('./config.json') etc. Or use our "environment" object in project.json (this is what I do). I don't think --set is even very useful haha, other than for one-offs

@tj yea I see where you're coming from for sure. In my case supporting the .env style config would make certain configs portable between my docker projects and lambda projects so I could use the same file sometimes

you could also have it in npm (if you're using node) or just in a Go package etc, but I prefer having a less "global" config, I've had a lot of issues with that in the past haha. At Segment we had one massive config package used by every program

I feel the struggle :) might just use dotenv instead

We'd love this feature right now - we worked around it by reading from a file and converting each line to --set flags but it is a bit ugly.

I suppose another hack would be --env $(cat myenv) and use foo=bar,bar=baz in there. I can see how people would want to keep sensitive stuff out of project.json.
I think I'm 👍 on it, otherwise people would have to symlink a shared config file into the functions I guess, bit more of a pain.

I'm stuck here too. I think a quick hacky fix would be to have a command that parses the .env file into json using dotenv and injects it into project.json then removes it after the command is written.
Then one could use the hooks :/

would be cool to get this in there soon. Anyone have opinions on foo=bar\nbar=baz or just go with JSON? I think the former is probably the best, to match /etc/environment

an --env-file would certainly be the most useful for my team……  occasionally people put spaces into these env vars, which is proving to be a bit tricky to reliably pass through the CLI.

Also comments! I recommend json
On Jun 10, 2016 7:37 PM, "Matt Andrews"<EMAIL_ADDRESS>wrote:

an --env-file would certainly be the most useful for my team……
occasionally people put spaces into these env vars, which is proving to be
a bit tricky to reliably pass through the CLI.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/apex/apex/issues/387#issuecomment-225320040, or mute
the thread
https://github.com/notifications/unsubscribe/AASHXk1FW_ENEmFw9MMPG1xWu0dqBiY4ks5qKfU0gaJpZM4ISNVa
.

@matthew-andrews any opinion on flat file vs json?

i don't mind, .env is a nice standard but i know that it ultimately ends up as a .env.json file anyway so why not keep it as that all the way through.

What about allowing function.json and package.json to also be js then we could let env = require('env.json'). I know this isn't a great solution for those who don't want to use javascript, but it would allow room for an env.json file that could live outside of the .json and be ignored from git. This way we could have functions with their own scope of environment variables and not have to provide a global set.

I like the idea of having an environment.json or similar in the dirs, that would play nice with hooks to load from external sources. For now I've added --env-file but I'll open an issue for files

Is this really implemented? I can't find --env-file in the docs and my apex installation (v.0.14.0) does not know this flag. :( What am I missing?


GUI support

I would like to add GUI support to ALPHA-AI. Please assign me to work on this with a Hacktoberfest label. Thank you!

@sachinl0har please mark it as Hacktoberfest accepted


[Textures] Greenification of plant textures on loaded packs

Especially for https://github.com/satoshinm/NetCraft/issues/113, but even with texture atlases from pre-1.5 MC, such as loading 1.7.3 beta terrain.png here - or many other packs - the leaves and grass appear gray instead of green:

A green tint likely needs to be applied. Might as well change the default texture to gray, too, and make it green at runtime (in the shader?).

https://www.reddit.com/r/Minecraft/comments/368lzd/my_biome_color_specific_blocks_all_changed_to/ describes a similar problem:

and suggests it has to do with the "tintindex" of the resources

Something like this may help? From CraftNG: https://github.com/twetzel59/CraftNG/commit/b715b8c319e5650dabc8597ef05af464cf46d3f6

Requested on https://www.spigotmc.org/threads/websandboxmc.232807/page-3#post-2428619  "when will green grass and green leaves be implemented
just wondering:)"

If you not yet want to make biome and the green color change by biome, you and simply choose a color in colormap, and then overlay it on the grayscale picture.


Enhancing Readme.md

Hi,
I’ve been exploring the project and appreciate the work you’ve done so far. I’d like to propose an update to the README.md file as It seems a bit boring to read and I want to improve it  to provide a more comprehensive overview of the website. Here are a few ideas I have in mind:

A detailed description of the project.
A clear vision for the final project and any future enhancements.
Embedded links for additional reference.

I believe these updates could enhance user understanding and engagement with the project. If you're open to it, I’d love to contribute.
I’m committed to continually improving the quality of our documentation.
Thank you!

Assign me this issue.I have worked on this and I m a beginner

I want to work on it. Can you please assign it to me?

Hey @neeru24, I see your PR was closed #171 with the label duplicate.
Could you have a look at the current readme and see areas of improvement ?
If you could restyle the current version, we can keep this issue open or else will close and create a new one. Do let me know here. Thanks in advance.

Hi, Yes
I can improve this readme.  Should I start my work and create a PR?
Or I've to create another issue?

Hey @neeru24, I see your PR was closed #171 with the label duplicate.
Could you have a look at the current readme and see areas of improvement ?
If you could restyle the current version, we can keep this issue open or else will close and create a new one. Do let me know here. Thanks in advance.

Hi, Yes
I can improve this readme.  Should I start my work and create a PR?
Or I've to create another issue?

This issue is fine, do create a new PR.

Hey @neeru24, I see your PR was closed #171 with the label duplicate.
Could you have a look at the current readme and see areas of improvement ?
If you could restyle the current version, we can keep this issue open or else will close and create a new one. Do let me know here. Thanks in advance.

Hi, Yes
I can improve this readme.  Should I start my work and create a PR?
Or I've to create another issue?

This issue is fine, do create a new PR.

Sure, I will create a PR by evening.

Hey @neeru24, I see your PR was closed #171 with the label duplicate.
Could you have a look at the current readme and see areas of improvement ?
If you could restyle the current version, we can keep this issue open or else will close and create a new one. Do let me know here. Thanks in advance.

Hi, Yes
I can improve this readme.  Should I start my work and create a PR?
Or I've to create another issue?

This issue is fine, do create a new PR.

@akbatra567 I've submitted the PR that got merged but it is not there in leaderboard, can you assign me labels again and mark it as completed or close the issue

Hey @neeru24, I see your PR was closed #171 with the label duplicate.
Could you have a look at the current readme and see areas of improvement ?
If you could restyle the current version, we can keep this issue open or else will close and create a new one. Do let me know here. Thanks in advance.

Hi, Yes
I can improve this readme.  Should I start my work and create a PR?
Or I've to create another issue?

This issue is fine, do create a new PR.

@akbatra567 I've submitted the PR that got merged but it is not there in leaderboard, can you assign me labels again and mark it as completed or close the issue

There is no PR that got merged yet, I've added a requested some changes to the PR. Do have a look at that.
Also as a practice in future do tag issue with your PR so it automatically closes once the PR is merge.
For now I've added:  closes #134 in PR description, which automatically closes the issue as soon as PR is merged.


Tower key management

Currently, the tower keypair (identity + sk) have to be manually created on first tower boostrap. However, this could easily be automated so the user does not have to bother with it (better UX).
Furthermore, options to overwrite the key pair and to import and external one (either to create it independently or to reuse it from an old build) should be added.

I think we can squeeze this in  v0.0.1 since it is a pretty trivial change that would make bootstrapping way easier.


TrayButton support for a different action per mouse button

Right now tray buttons only support running a single action.  It would be nice to be able to specify an action for each mouse button (see issue #11).
The most complicated part of this is deciding how to configure it.  One approach would be:
 <TrayButton icon=“myicon.png”>
   <Button mask=“123”>exec:actionA</Button>
   <Button mask=“4”>exec:actionB</Button>
   <Button mask=“5”>exec:actionC</Button>
 </TrayButton>

This would have a tray button that ran the same command when buttons 1, 2, or 3 were clicked and different actions for the scroll wheel.

This is implemented as of snapshot 1072.
For backwards compatibility and ease of configuration, you can still use the old method of configuration to get mask “123”.

Sorry to be a bother here.
Could you make the release notes a bit more verbose?
You have:
"Added the ability to have seperate actions per mouse button for tray buttons (issue #171)."
Is the approach to use what you describe in your initial comment?  Or did you do something slightly different?
Is this a for TrayButton only, or does this fix the scroll wheel switching desktops issue that users have as well?  Sorry If I missed something obvious here.

I just updated the release notes.
This is only for tray buttons (where it fixes the scroll wheel issue because the default button mask is now “123” instead of any button), though I could do the same thing for the clock.  The clock has been fixed just by ignoring all buttons other than 1, 2, or 3.

Ok thank you!
I see what you did, this is amazing!  I am getting some really interesting ideas of what is possible now.
Does this also work with root? or only exec?

This should work with root as well.
I just now updated the “Clock” component to support binding different commands to different buttons.  I’m considering making “Clock” work the same way as TrayButton such that it can show menus too, but that would be another change that would break old configurations.

A menu could be useful in the clock, but I do not think it is extremely important at this point.
I am rebuilding my PPA of jwm (I am using a minimal Ubuntu distro), so I will test the new clock to make sure that the scrolling issue is fixed

@joewing I had a thought recently.  I have made a few different menus that I generate automatically from scripts in /etc/menu-methods.  I use almost all the possible menus I can have (AFAIK).
I was wondering if there was a way to use more than 1-9.  I know if I chain together numbers those menus are all linked together onroot="23" for example.
With Traybuttons supporting different actions for different buttons, is it possible to make an onroot="a"  or some other option.  I would like to directly link right clicking on my 'Applications menu' Traybutton to a menu with items to configure it (change menu items, configure icon, etc...)
Basically I would like to implement something akin to what larger DE do (LXDE,XFCE, etc...) where a right click on different parts of the Tray show a menu to configure that specific part.
If this would be too major an undertaking I fully understand not being able to.

You can try just changing the count from 10 to some arbitrary number.
https://github.com/joewing/jwm/blob/master/src/root.c#L24
I'm not sure if there is anything more to it.
(This value was changed since 1.x though, but I don't recall if it added/subtracted from the count)
Note however that your "important" menus should still be 1-3 for mouse button association.

Adding support is something I’ve thought about doing before.  It should be fairly easy to do.  I created an issue for it (issue #172).
Just changing that constant should work, but then you would need to specify strange characters (or up the constant to a large enough number).  The next 7 ASCII characters are punctuation, but then the upper-case letters start.


fix wrong CHANGELOG.md link in integrations guide

Description (required)
fix wrong CHANGELOG.md link in integrations guide(mdx --> tailwind's CHANGELOG.md)

Hello! Thank you for opening your first PR to Astro’s Docs! 🎉
Here’s what will happen next:

Our GitHub bots will run to check your changes.
If they spot any broken links you will see some error messages on this PR.
Don’t hesitate to ask any questions if you’re not sure what these mean!

In a few minutes, you’ll be able to see a preview of your changes on Netlify 🥳

One or more of our maintainers will take a look and may ask you to make changes.
We try to be responsive, but don’t worry if this takes a few days.


cmd/redirector: add cloudbuild.yaml

This builds/deploys gcr.io/cloudrun/button-deployer:$SHORT_SHA image
to cloud-run-button-deploy Service.

I'll go ahead and merge this as I manually verified, I need this merged to create the Build Trigger.