CERTIFICATE_VERIFY_FAILED while connecting to the databricks sql endpoint using python through M2M auth

main.py content
from dotenv import load_dotenv, dotenv_values
load_dotenv()
from databricks.sdk.core import Config, oauth_service_principal
from databricks import sql
import os

server_hostname = os.getenv("DATABRICKS_HOST")

def credential_provider():
  config = Config(
    host          = f"https://{server_hostname}",
    client_id     = os.getenv("DATABRICKS_CLIENT_ID"),
    client_secret = os.getenv("DATABRICKS_CLIENT_SECRET"))
  print(config)
  return oauth_service_principal(config)

with sql.connect(server_hostname      = server_hostname,
                 http_path            = f'{os.getenv("DATABRICKS_HTTP_PATH")}',
                 credentials_provider = credential_provider,
                _tls_no_verify=True,
                 ) as connection:
    with connection.cursor() as cursor:
        cursor.execute("SELECT 1 as a")
        print(cursor.fetchall())
        cursor.close()

connection.close()

.env content
DATABRICKS_HOST="adb-XXXXXXXXXXXXX.azuredatabricks.net"

DATABRICKS_CLIENT_ID="XXXXXXX"

DATABRICKS_CLIENT_SECRET="XXXXXXX"

DATABRICKS_HTTP_PATH="/sql/1.0/warehouses/XXXXXXXXXX"

ERROR
Traceback (most recent call last):
  File "/Users/saurabhkumar/PycharmProjects/pythonProject/test.py", line 28, in <module>
    credential_provider()
  File "/Users/saurabhkumar/PycharmProjects/pythonProject/test.py", line 21, in credential_provider
    config = Config(
  File "/Users/saurabhkumar/PycharmProjects/pythonProject/.venv/lib/python3.8/site-packages/databricks/sdk/config.py", line 127, in __init__
    raise ValueError(message) from e
ValueError: default auth: oauth-m2m: HTTPSConnectionPool(host='adb-XXXXXXXXXXXX.azuredatabricks.net', port=443): Max retries exceeded with url: /oidc/.well-known/oauth-authorization-server (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1131)'))). Config: host=https://adb-XXXXXXXXXXX.azuredatabricks.net, client_id=XXXXXXXXXXX, client_secret=***. Env: DATABRICKS_HOST, DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRET

Packages
cachetools==5.5.0
certifi==2024.8.30
charset-normalizer==3.3.2
databricks==0.2
databricks-connect==13.0.1
databricks-sdk==0.32.1
databricks-sql-connector==3.4.0
et-xmlfile==1.1.0
google-auth==2.34.0
googleapis-common-protos==1.65.0
grpcio==1.66.1
grpcio-status==1.66.1
idna==3.8
lz4==4.3.3
numpy==1.24.4
oauthlib==3.2.2
openpyxl==3.1.5
pandas==1.5.3
pip-system-certs==4.0
protobuf==5.28.1
py4j==<IP_ADDRESS>
pyarrow==16.1.0
pyasn1==0.6.1
pyasn1_modules==0.4.1
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
pytz==2024.2
requests==2.32.3
rsa==4.9
six==1.16.0
thrift==0.20.0
urllib3==2.2.2
wrapt==1.16.0

@benc-db can you please take a look?

@kravets-levko Yes, it works fine when I use access_token.

I think the earlier error was due to some request quota exceed. Now I am getting different error with same code.
<Config: host=https://adb-XXXXXXXX.azuredatabricks.net, client_id=XXXXXXXXX, client_secret=***, auth_type=oauth-m2m. Env: DATABRICKS_HOST, DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRET>
<Config: host=https://adb-XXXXXXXX.azuredatabricks.net, client_id=XXXXXXXXX client_secret=***, auth_type=oauth-m2m. Env: DATABRICKS_HOST, DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRET>
/Users/saurabhkumar/PycharmProjects/pythonProject/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1099: InsecureRequestWarning: Unverified HTTPS request is being made to host 'adb-XXXXXXXXX.azuredatabricks.net'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
Traceback (most recent call last):
  File "/Users/saurabhkumar/PycharmProjects/pythonProject/test.py", line 30, in <module>
    with sql.connect(server_hostname      = server_hostname,
  File "/Users/saurabhkumar/PycharmProjects/pythonProject/.venv/lib/python3.8/site-packages/databricks/sql/__init__.py", line 90, in connect
    return Connection(server_hostname, http_path, access_token, **kwargs)
  File "/Users/saurabhkumar/PycharmProjects/pythonProject/.venv/lib/python3.8/site-packages/databricks/sql/client.py", line 247, in __init__
    self._open_session_resp = self.thrift_backend.open_session(
  File "/Users/saurabhkumar/PycharmProjects/pythonProject/.venv/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 549, in open_session
    response = self.make_request(self._client.OpenSession, open_session_req)
  File "/Users/saurabhkumar/PycharmProjects/pythonProject/.venv/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 478, in make_request
    self._handle_request_error(error_info, attempt, elapsed)
  File "/Users/saurabhkumar/PycharmProjects/pythonProject/.venv/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 308, in _handle_request_error
    raise network_request_error
databricks.sql.exc.RequestError: Error during request to server

I have found a work around:
Downgraded the databricks-sql-connector version to 2.7.1.dev2 as mentioned
here: https://github.com/databricks/databricks-sql-python/issues/169
Now, I am getting permission error:
databricks.sql.exc.RequestError: Error during request to server: PERMISSION_DENIED: e5ecc97a-d6e2-4f0b-a29d-17ef69544871 is not authorized to use this SQL Endpoint. Please contact your administrator.
I will update here after testing the script with proper permission.

@saurabhdataeng this warning comes from urllib and normally should be just printed, but not interrupt your script. Do you have anything that may override that behaviour and change warnings to exceptions? Also, try to disable warnings and see if anything changes:
import warnings
...
warnings.simplefilter("ignore")

@kravets-levko
No, I am just using the boilerplate code copied from azure databricks documentation. I downgraded the databricks-sql-connector version as mentioned and now I am not getting any execptions and warnings.

This is not working for me either @kravets-levko . I'm using the documentation within the Databricks workspace and the connection just hangs. The GO client works fine...
from databricks import sql
import os

connection = sql.connect(
                        server_hostname = "dbc-360a72da-4376.cloud.databricks.com",
                        http_path = "/sql/1.0/warehouses/b9e5450fbdfc8b49",
                        access_token = "<access-token>")

cursor = connection.cursor()

cursor.execute("SELECT * from range(10)")
print(cursor.fetchall())

cursor.close()
connection.close()

Also experiencing this issue with the boiler plate code, opened a new issue.

I think I found a solution for this in my issue

Just wanted to add some signal that the boilerplate M2M code appears to be non-functional on main. I'm working on an integration now (which we'll hopefully publish in the near future) and had to scrap the boilerplate here. For reference, the U2M example does appear to work. But M2M is broken.


ros2_example_bazel_installed: Should have simple cc,py example of using RMW isolation

Per Anzu 9275, we should have a simple cc and py test that exercises RMW isolation.
e.g. rmw_isolation_example_cc_test, rmw_isolation_example_py_test
Both can simply publish a float, and do no other verification (for now).
They should then be referenced by:
https://github.com/RobotLocomotion/drake-ros/tree/eba927fd3ecf0ce2819200e0a8176e7d3f366e2a/bazel_ros2_rules/ros2#tools
\cc @IanTheEngineer

I see a test here for isolated listener and talker in both C++ and python : https://github.com/RobotLocomotion/drake-ros/blob/main/bazel_ros2_rules/ros2/resources/rmw_isolation/test/isolated_listener.py. To we require something more to resolve this issue ? cc @sloretz

Also a duplicate issue probably : https://github.com/RobotLocomotion/drake-ros/issues/77

Do we require something more to resolve this issue ? cc @sloretz

I think I recall this issue was to have an example test in the ros2_example_bazel_installed folder.
That folder is a separate bazel workspace, which matches what our downstream users would use. The C++/Python files will probably look similar, so IIUC the value in putting tests there is in showing what bazel rules to load() and run in the WORKSPACE and BUILD.bazel files.

Yes, a very clear, user-based example of both C++ and Python of how to use isolation is my intent.
Having it in a more focused example workspace makes sense too! note that existing rmw_isolation tests are more for unittesting, less for exemplars.


Feat/component detail analytics

Changes to Component Details Page.
Things to do

[x] Use QueryAPI graphQL to pull developer information

Head to near.org//widget/ComponentDetailsPage?src=devgovgigs.near/widget/gigs-board.pages.Feed to test

@roshaans Is there an associated ticket for this work?

@roshaans Is there an associated ticket for this work?

yes, Here it is

Thanks for linking the ticket @roshaans - I guess I'm curious what the product reasoning is behind renaming and removing some of the tabs. I didn't see any discussion about that in the linked ticket.

Thanks for linking the ticket @roshaans - I guess I'm curious what the product reasoning is behind renaming and removing some of the tabs. I didn't see any discussion about that in the linked ticket.

@TiffanyGYJ can provide more insights here

Thanks for linking the ticket @roshaans - I guess I'm curious what the product reasoning is behind renaming and removing some of the tabs. I didn't see any discussion about that in the linked ticket.

@calebjacob We are doing a special NEARCON App Upvoting Event, more details here
Google Doc on the event
Blog Post with simplified explanation for DApp owners
TLDR:

this is a NEARCON Event in the form of a NCON Task that ask people to upvote qualified apps and decide who gets featured
We are using a fork of the App Library and populate it with Apps that are qualified (description required + tags<Earn,play,develop,engage> required, so we are running a data filtering pipeline on the back to grab those.
Then we will enable a upvote button on the details page of these qualified apps.
Users at NEARCon can visit this App Upvoting Event component, look at the qualified apps, upvote the ones they like.
Result is top 8 upvotes will get featured in the App Library (near.org/applications)

Thanks for the info @TiffanyGYJ. Going forward, it would be nice to have PR's link to issues with those requirements listed so that other developers can understand the context for the changes.

Thanks for the info @TiffanyGYJ. Going forward, it would be nice to have PR's link to issues with those requirements listed so that other developers can understand the context for the changes.

Yep Great advice, will include these in the future, thank you for the help @calebjacob !


TSH progresses the wrong player from a challonge bracket

Some players are flipped around in their set in the bracket widget, e.g. in challonge they will be displayed as
Player 1
Player 2
but in the TSH bracket widget they will be displayed as
Player 2
Player 1
So when a player progresses to the next round the wrong player progresses in the bracket widget like shown in these screenshots.

Hi! Can you post the link to your bracket here so we can investigate this behavior please?

Sure
https://ausanime.challonge.com/OCEP4U40

Should be fixed in 5.21


hello i got a server error when i use redisson to connet AliyunRedis

the error like this:
"ERR unknown command eval. channel: [id: 0x73e1562f, /<IP_ADDRESS>:35465 => 126748df08bd4eea.m.cnhza.kvstore.aliyuncs.com/<IP_ADDRESS>:6379] command: CommandData [promise=DefaultPromise@40a65eef(incomplete), command=(EVAL), params=[local v = redis.call('hget', KEYS[1], ARGV[1]); redis.call('hset', KEYS[1], ARGV[1], ARGV[2]); return v, 1, loginInfo, admin, com.expresscne.parcelconfig.data.model.UserAndPermission.UserPermissionObject@175b609e], codec=org.redisson.codec.JsonJacksonCodec@5bcaeaef]",
"path": "/system/userPermissionEndPoint"
I ask Ali's tech support about it. He told me this issue was caused by AliyunRedis not supported EVAL operation.
How can I do?

hi! Sorry but eval is necessary operation for Redisson. But you can try old version 1.2.1 it doesn't use eval commands but losing performance.

Maybe it would be better try another hosting with full-featured redis support?

Ok! thank you！I got it！


xxx

I decided to remove this message and close ticket. This is poor analysis and I would like to apologize.

Closing issue.


Update Help message displayed for --usage

This should reflect the actual features available in the recipe.

:tada: This issue has been resolved in version 1.0.0 :tada:
The release is available on:

GitHub Release
NuGet Package


Please fix the Bluetooth data transfer issue on Android 13 devices.

I use the Jayps app on my Samsung device running Android 13. When the screen of my phone is turned off, the data displayed on my Pebble watch only lasts for a few seconds until I press the power button, after which it continues to display for a few more seconds. I have already allowed the Jayps app to run in the background, but it still doesn't work.

You're using the 3.0.0 apk? I had issues with the 3.1.0 beta


dashboard/app: provide self-contained repro scripts

Reflecting on the syzbot experience, some developers say that the reproducers don't reproduce bugs for them. There are probably some differences in setup or something, because in most cases I can reproduce these bugs locally in qemu. It would be useful to generate scripts that will do all necessary setup (e.g. checking out kernel, building it with the right compiler and config, starting qemu with the right image, etc) and trigger the bug. This can be especially useful for syzkaller reproducers as they are harder to run (we could provide all necessary flags for syz-execprog too).

Mark Rutland suggested that this script also needs an option to accept a custom kernel (that a developer built themselves). Which is reasonable for debugging/fix testing.


SelectableList 'Link to' container element requiring two clicks to fire

Hi,
I am using a Selectable List within a Drawer as a fixed side navigation component for my app. Until recently I was using the href property to provide the navigation but noticed that it was reloading the page on each click. To get around that I want to use React Router's Link functionality like so:
<SelectableList defaultValue={1}>
        <ListItem
            value={1}
            primaryText="Dashboard"
            leftIcon={<ActionDashboard />}
            containerElement={<IndexLink to="/" />}
        />
</SelectableList>

However on first click the URL is updated but a blank page is rendered. Then when I click the nav list item again it loads the page.
I thought it maybe related to the SelectableList but I'm also using it on my horizontal AppBar MenuItems and I'm getting the same thing:
<AppBar
        style={styles}
        iconElementRight={
            <IconMenu
                iconButtonElement={<IconButton><MoreVertIcon /></IconButton>}
                targetOrigin={{horizontal: 'right', vertical: 'top'}}
                anchorOrigin={{horizontal: 'right', vertical: 'top'}}
                >
                <MenuItem
                    primaryText="Sign Out"
                    onTouchTap={ handleLogout }
                />
                <MenuItem
                    primaryText="Manage Users"
                    containerElement={<Link to="/manageusers" />}
                />
            </IconMenu>
        }>
    </AppBar>

Any ideas?
Thanks!

We have been porting the component on the v1-beta branch. We reimplemented it from the ground-up. While we haven't tested it, I think that the issue is most likely fixed on that branch. Hence, I'm closing it.
Still, we will accept PR fixes until v1-beta takes over the master branch.


KeyError 'error' in docstring when using Resnets

Hi, first of all, thanks for putting out these models! They are really helpful and save me some time.
However to be able to use the library with Tensorflow 2.4.1 a small fix has to be made. The PREPROCESS_INPUT_DOC includes an additional field error which is not provided as parameter to .format(..  causing this error:
    from tf2_resnets import models as resnet_models
  File "/home/anargh/anaconda3/envs/max-thesis/lib/python3.7/site-packages/tf2_resnets/models.py", line 1, in <module>
    from tf2_resnets.resnet import ResNet
  File "/home/anargh/anaconda3/envs/max-thesis/lib/python3.7/site-packages/tf2_resnets/resnet.py", line 298, in <module>
    ret=imagenet_utils.PREPROCESS_INPUT_RET_DOC_TORCH)
KeyError: 'error'

Code location:
https://github.com/RichardXiao13/TensorFlow-ResNets/blob/a77281032e2942ad30319200e2819962a4a5e600/tf2_resnets/resnet.py#L296
I think it is not a good idea to reuse docstrings as they can not be considered a stable API. So I would favor the removal of those lines

Sure, I will get on that.

So to use it with tensorflow 2.4.1 is there any workaround at this moment?

You can delete the docstring as a workaround.
https://github.com/mjohenneken/TensorFlow-ResNets/commit/a1879d8626ae03ca1b0a5d41654fddd2eb3cffb2

This issue is solved, please close it.


[Tab] indicator transition property is now broken

Since latest beta, the generated transition property for tabs indicator is   transition: all 300ms cubic-bezier(0.4, false, false.2, false) 0ms which is invalid

[x] I have searched the issues of this repository and believe that this is not a duplicate.

Expected Behavior

The default transitions property should be properly generated.
Current Behavior

The default transitions property is wrong.
Your Environment

Tech
Version

Material-UI
beta 28

React
16

browser
Chrome 63

etc
TS 2.6.2

I am using material-UI 4.2.0, and still getting this error. Is this fixed? Thanks.

@sunilb0575 I would only expect an issue in the devtool. The logic should be working in the browsers. Do you want to give #14763 a try :)?


Review and enhance tracing support for email service (Ruby)

The following is a list of requirements that we need to evaluate before declaring v1 for trace telemetry. These requirements are across the entire application; Not all services will meet all requirements. Determine the relevant features for this service.

[x] Automatic Instrumentation is being used where appropriate.
[x] Library instrumentation is being used if automatic instrumentation is unavailable.
[x] Services extend automatic instrumentation.

[x] New attributes/events attached to existing spans.
[x] New spans are being created from existing spans.

[ ] Automatic and Manual Context Propagation are being demonstrated.
[x] Telemetry is not being sampled upfront.
[x] Telemetry is not being filtered upfront.
[ ] Baggage is being set and read appropriately (i.e., baggage must be explicitly set as attributes on spans)
[ ] Update the centralized tracing doc

Referencing: https://github.com/open-telemetry/opentelemetry-demo-webstore/issues/42
Service: https://github.com/open-telemetry/opentelemetry-demo-webstore/blob/main/src/emailservice/README.md
Ruby dependent on this item: https://github.com/open-telemetry/opentelemetry-demo-webstore/issues/37

@cartersocha since I can't check off the boxes myself, here's what could be updated right away:

✅ Automatic Instrumentation is being used where appropriate.

The demo installs automatic instrumentation for Sinatra in the configuration block. Because the ruby app is so simple, there's really nothing else to instrument automatically. 😄

✅ Library instrumentation is being used if automatic instrumentation is unavailable.

We have automatic instrumentation available, so we should satisfy this condition. I'm not sure what "Library instrumentation" is in this case anyways.

❓ Telemetry is not being sampled upfront.

Can you clarify what you mean by "sampled"? In the ruby SIG, we use the term "sampled" the way the SDK spec uses it - to mean "this span will be exported."
The way that the SDK uses this term in combination with this TODO item makes me think the requirement is that the ruby app should not be exporting spans by default?
That's a little confusing to me, and so I'm curious what you mean. On the other hand, if you actually mean "the ruby app should be exporting all spans," then I think we're fine.
The ruby SDK installs a "parent-based, always-on" sampler by default. This sampler will always choose to record the span, unless the parent span says otherwise. Since we aren't manually changing the sampler anywhere, that behavior will be what the ruby app exhibits.
This is also the default behavior according to the SDK spec.

❓ Telemetry is not being filtered upfront.

Unclear exactly what you mean by this, but we aren't doing any kind of additional filtering that I am aware of in the ruby SDK.

❓ Automatic and Manual Context Propagation are being demonstrated.

We aren't calling out to any other service from the ruby app (it's a "leaf" in the tree), so I'm not sure how best to demonstrate additional context propagation.
Do you have any advice on where we should demonstrate that?

The remaining items are:

[ ] baggage
[ ] updating the centralized tracing doc

Since there's not much remaining that I don't have questions on, I'll wait to get a little clarification on the rest of the TODO items before I begin work. If you want to assign the issue to me, please do so! 😄

It occurs to me that one interesting way to make the ruby service exhibit some of the desired behaviors would be to over-complicate it with a job queue. 🤣
...I think that actually might be a good idea. Are there any concerns with that? Conceptually it would be over-complicated, but the actual code to implement it would be rather simple and straightforward. We could more easily demonstrate context and baggage propagation in that way.
Curious what you think.

Good questions. I'd suggest posting the queue question in the slack channel to get a broader discussion. Sounds interesting to me

@austinlparker, @puckpuck, what are yalls thoughts on our context propagation requirement. Should we simplify to just creating a new span based off an existing span? (inside service context prop). The instrumentation libraries already have automatic context propagation covered in my mind & connect the services

Tracing is code complete


Update exif.h

trying to fix #2100

Thanks for the suggested change, @a-shvedov! Could you follow the instructions above from the google-cla bot?!

Thanks for the suggested change, @a-shvedov! Could you follow the instructions above from the google-cla bot?!

Yes, sure!


Vulcanize integration

Hi,
Is it possible to combine gwt-polymer and vulcanize ?
Imports are made in java code (and so imported in javascript), vulcanize is blind...

gwt-polymer-elements does not include any utility to run vulcanize, but, since <IP_ADDRESS>  it supports that you import all the dependencies by your own, not doing automatic imports.
Basically, you have to manually create an auxiliary.html file with all the dependencies (web components) your app uses and run vulcanize on that file. Install the output of vulcanize in your public folder and include if in your index.html.
Disclaimer: I have not tested it yet, so I maintain this issue opened until we test this procedure and document it.

Hi there,
where to find the version <IP_ADDRESS> ?
I need to vulcanize imports because polymer is very slow on mobile.
using vulcanize and crosswalk project can help.

Vulcanizing with polymer.html excluded (and imported manually) seems to work.
Unfortunately I've struggled with custom styles (see this question on stackoverflow). Seems I have to refactor my project first...

Update: the combination of Widgets and UIBinder causes LazyDomElement exceptions ("element not attached to the DOM") in some cases.
Perhaps the strategy of UIBinder to attach Widgets logically first and the Webcomponents lifecycle does not fit together?
With elements it is working. Seems widgets are dead before GWT 3.0...

Hi @muelbr can you share some sample achieving vulcanize using gwt polymer elements?

Hm, there is nothing special.

Develop your app with gwt polymer elements as recommended here:
GWT & Polymer / Elements UI

In onModuleLoad replace Polymer.importHref() just with Polymer.whenReady()

In your host file index.html add the vulcanized components file statically
<link rel="import" href="vulcanized.html">

@muelbr Thanks for sharing your approach. Do you mind to contribute writing a .md file so as we can officially publish here ?
Note: in Vaadin, we are not yet going to develop any utility to automatically vulcanize (for instance when compiling gwtc) until we don't know more about the future of the compiler. Generators could do the work but are going to be deprecated. A process developed in java could be tough. Vulcanize is still being developed and optimised. PWA + HTTP2.0 would use push instead of Vulcanize, etc.

Hi @manolo, would it be possible to make it so that we can use the vulcanized polymer files and still use Polymer-GWT Widgets?
As far as I can see, only thing that impedes this option is that every PolymerWidget calls Polymer.ensureCustomElement (getElement(), src);.
If there were some property, say static private boolean ENSURE_CUSTOM_ELEMENTS_LOADED = true; that would control whether the loading of the element source is called, only thing that should be needed is to import the vulcanized file the same way with elements using Polymer.whenReady()
Am I right?
If yes, would you be so kind as to enable this option? We have an application coded around PolymerWidgets and we struggle with loading speeds and the fact that not all element definitions are loaded before the components are displayed. We hope that vulcanizing would solve our problems.

Fixed in <IP_ADDRESS>.
Import your elements or vulcanized bundle in your host file, and widgets will take care of not importing if the component is already registered.

It's working like a charm, just an issue (only in chrome) using the paper-card in the vulcanized file.

Here is a deeper explanation of problems when using Polymer-Widgets and static imports...
PaperDropdownMenu fails after initial import


ability to deal with server side defaults/initializers

Is your feature request related to a problem?
Ability to specify fields initialized by service post create. Following are the patterns observed:

Generated fields. Example:

roleARN field in ScalableTarget resource is generated by service. See https://github.com/aws-controllers-k8s/applicationautoscaling-controller/pull/12#discussion_r640097037 (service - Applicationautoscaling)
staticHyperparameter field in Hyperparameter tuning resource. Slice has some extra fields returned from the service which should be ignored for comparison. See customSetDefaults in HPJob (service - SageMaker)

Default value for fields

Boolean fields returned as False.

Scalable target struct in ScalableTarget resource - SuspendedState.<*> (all fields) returned as False by default. See setDefaults in autoscaling (service - Applicationautoscaling)

String fields return default values. e.g. Model resource - PrimaryContainer.Mode retuned as  SingleMode by default (service - SageMaker)

Describe the solution you'd like
1.a, 2.a, 2.b - if DesiredResource.Spec.field = nil set these values from latestResource
1.b -  a hook to write custom algorithm for comparison and remove service generated fields, ignore setting values from latestResource to desired
Note: In 2.a,

whole desired.spec.SuspendedState == nil and service still returns false values for all fields in the structs (DynamicScalingInSuspended, DynamicScalingInSuspended, ScheduledScalingSuspended)
desired.spec.SuspendedState != nil but desired.spec.SuspendedState.DynamicScalingInSuspended == nil and service still returns False for desired.spec.SuspendedState.DynamicScalingInSuspended

Describe alternatives you've considered
1.b - a hook to write custom logic to insert only server generated <key, value> pairs into the slice only if diff is False
1.b - a hook to write custom algorithm for comparison and insert service generated fields, set values from latestResource into desired only for comparison

1.b - a hook to write custom algorithm for comparison and remove service generated fields, ignore setting values from latestResource to desired

@surajkota do you mean a hook that would give service teams the ability to control whether or not the diff for a certain field should be added to the delta? i.e. essentially overwriting any one of these blocks: https://github.com/aws-controllers-k8s/elasticache-controller/blob/main/pkg/resource/user/delta.go#L35-L41
I definitely agree if so, but please let me know if I misinterpreted the suggestion.

1.a, 2.a, 2.b - if DesiredResource.Spec.field = nil set these values from latestResource
Also, could you elaborate on this suggestion? Do you mean directly populating desired.Spec with the server-returned default values?

In general, the generated code for adding a diff to the delta makes the assumption that a nil difference either direction is sufficient grounds for invoking an Update.
However, this is a case in which desired.Spec.field = nil and latest.Spec.field = non-nil, but we don't want to invoke an Update (because by not specifying the field, the user has signaled that they are okay with the default configuration for that field). Even if we did, we would be populating field with the equivalent of nil in the Update payload, which doesn't really make sense.
So some possible solutions would be:

Changing ackcompare.HasNilDifference to only check for the nil difference in one direction, rather than both, on every Spec field. In other words, only add the diff if desired.Spec.field = non-nil and latest.Spec.field = nil but not the other way around.
Adding support in the generator config to enable the above behavior, but on a per-field basis

@vijtrip2 do you have any thoughts on this, and has work already been done on this that I've missed or am not aware of?

essentially overwriting any one of these blocks:

yes

Do you mean directly populating desired.Spec with the server-returned default values?

yes

Finished core implementation. Upcoming PR for back-off delay work.

The current implementation does not handle all types of fields. Would like to keep this issue opened unless there is another one

/lifecycle frozen


About the margin-left

Hi,
Finally got time to try out Neat 2.0 :)
Just try out some simple grids and realise that the columns are using margin-left to position themselves. Is this suppose to be the case, or am I using it wrongly?
So technically my container is not center on the screen since the first column have the margin-left and at the last column, there's no margin-right being compensated.

cross posting an image from [https://github.com/thoughtbot/neat/pull/496]. this is how the grid lays out in a container. columns use margin left to create their gutters. they are centered because of the width/calc operation makes it so the right gutter is just not used and remains empty.
You can also check out an example page by cloning the repo with neat-2.0.0-beta.1 checked out and then run gulp to preview the contib page. more on that here.

Ah, I get it. Sorry my bad. I was just trying to use display:flex since this was another exciting advantage of Neat 2.0.
Using display flex on the container will determine its child-items width itself and since the last column doesn't have margins, it will be shifted wrongly.
Maybe the title should be renamed to using in conjunction with flexbox.

I gave up on this grid - it seems to be complicated - one has to do a collapsed grid in order to have the grid to start at the left - I don't understand why neat is not using halve gutter left and right and compensate on the parent like many other grids out there - foundation and bootstrap are just 2 of them. calculations of the grid would be easier as well as it would be just calc($column-width - $gutter).

Hi @mmjaeger, your comment is not directly related to this issue. Please feel to create a new issue if you have a specific concern. You have previously stated that you don't like this project but posting unsolicited negative comments on other peoples issues is not productive or helpful.

sorry if you think that my comment was negative - moreover, I never said
that I don't like the project - have been using bourbon - neat for years.
On Tue, Aug 23, 2016 at 12:40 PM, Will H McMahan<EMAIL_ADDRESS>wrote:

Hi @mmjaeger https://github.com/mmjaeger, your comment is not directly
related to this issue. Please feel to create a new issue if you have a
specific concern. You have previously stated
https://github.com/thoughtbot/neat/issues/483 that you don't like this
project but posting unsolicited negative comments on other peoples issues
is not productive or helpful.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/thoughtbot/neat/issues/497#issuecomment-241852059,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABe4lBOkgGmrPu6j_3jenFwcYwDIrX8gks5qi0yqgaJpZM4Jq7c_
.

--
Marco M. Jaeger
http://net4visions.com

@nickngqs Here is a screen shot of using flexbox on the example page container (.grid, the one with the outline). Here I had to add flex-wrap: wrap; since everything was in one big-ol container to make sure the columns would reflow on to the next line. if the container was only 1 'row' of content, this wouldn't be necessary.

What method would you suggest to achieve this kind of layout

As you see, sometimes design wise, they don't really always take up a space of 12 columns for each row (full width)

Should i be adding a extra container to function as a row?

Should i be using grid-push to push Something Text column to center it?

Should i be overriding that single Something Text column with margin: 0 auto; so that it centers, as well as removing the margin-left.

If I have to add an extra container, it seems to unnecessarily add additional markup. I believe that cleaner and more concise markup seems to be one of the features of using a Neat Grid.

If i grid-push to center my column. And if the second column Others happen to able to be small enough to take up the space, it will go up to join the first column. So should I be pushing the second column as well? Seems to add complications, especially when it comes to doing responsive.

I guess this is the best current method, but it does requires additional css again on every column.

@nickngqs Cool! so here we go. If I were to put this together it would be something like the following:
body
  .container
    .something-text
    .others
    .empty

.container {
  @include grid-container;
}

.something-text,
.others {
  @include grid-column(4);
  @include grid-push(4);
  clear: left
}

.empty {
  @include grid-column(6);
  @include grid-push(3);
  clear: left
}

In this example you don't really need the container since all it does is add a clearfix. you were on the right track with using push to center. all that's needed is clear: left;  to finish things off.
hope that helps 👍

Closing for now. Reopen if the issue persists.


Prepare Wasmi bytecode, translator and executor for tail-call based instruction dispatch

This is a rather large working item that will span across multiple PRs.
Currently Wasmi uses an enum based Instruction to represent all of its Wasmi bytecode instructions.
Pros & Cons

The advantage of this approach is that it is a very simple and rusty representation, easy to work with and allows to avoid unsafe Rust as much as possible.
The downside of this approach is that it is extremely dependent on the Rust and LLVM compiler and optimization pipeline to produce great machine code for the executor in the end as past experiences and experiments showed time and time on. The underlying issue is that this representation is too abstract and does not allow for the fine grained control over the layout and dispatch mechanism.

Major Flaw
Furthermore the enum based Instruction representation with its associated data fields cannot be easily combined with explicit tail calls once they are available in Rust (if ever).
The makepad-stitch Wasm interpreter has clearly showed that tail call based instruction dispatch is clearly superior to the enum based approach that Wasmi is using, especially on Apple Silicon and modern CPUs.
Threaded Code
The dispatching technique that we want to use is called threaded code dispatch and is made possible by explicit tail calls amongst others. Fundamentally there are two different kinds of threaded code:

Direct threading: This stores pointers to instruction handlers directly in the bytecode and simply invokes them once the instruction pointer points to them. This is fast and has the least amount of indirections during execution but not very space efficient because a pointer on a 64-bit machine takes up a whopping 8 bytes.
Indirect Threading: This stores simple C-like enum instruction op-codes instead of instruction handler function pointers directly in the bytecode and thus requires an additional indirection to resolve from the C-like enum variant to the associated instruction handler during execution. Some papers suggest that the overhead compared to direct threaded code is insignificant but we should probably test this ourselves. This form is easier to debug and simpler to deal with since the op-codes are still simple integers and not raw pointers.

Due to the above research related question of direct or indirect threaded code it is important to implement the new architecture to easily exchange the used technique in the end.
In case we want to support both styles of threaded code it should be easily possible to design the architecture so that conditional compilation could enable to disable either dispatching mode via a new Wasmi crate feature.
Link to GitHub Gist where I tried out some potential structure on the compiler explorer website to see what assembly is generated: https://gist.github.com/Robbepop/0b254c86dccabc48ae0eb41e89567ea0
Fallback & Performance
The new Wasmi bytecode can be dispatched using the match+loop approach used so far as a fall back when explicit tail calls are not available for an architecture or until they are not available in Rust at all. The execution performance difference should ideally be insignificant from the performance of today's Wasmi bytecode executor.
Tasks

[ ] Redefine Wasmi bytecode in terms of a new InstructionEncoder that provides a large API surface to encode all the required Wasmi bytecodes. We should stay as close as possible to the current set while also take into account for the new possibilities of the more relaxed bytecode representation. The new bytecode representation is more relaxed because it will no longer be organized on an 8 byte "instruction word" buffer but on a raw byte buffer. This means the encoding of an instruction is less restricted with respect to the amount of bytes it may use. We want this InstructionEncoder to be usable from both the Wasmi translator as well as the Wasmi translation test suite.
[ ] Rewrite the Wasmi translator in terms of the new InstructionEncoder.
[ ] Adjust all the translation tests to use the new InstructionEncoder and the new Wasmi translator.
[ ] Implement an InstructionDecoder that can be used to decode instructions encoded by the InstructionEncoder. This InstructionDecoder shall offer an efficient API for the Wasmi bytecode executor.
[ ] Implement the new Wasmi bytecode executor based on the new InstructionDecoder.

With #1152 merged we now have a useful baseline Wasmi bytecode representation with which we could (in theory) test this out once explicit tail calls have been merged into Rust proper.

This has effectively been implemented in https://github.com/wasmi-labs/wasmi/pull/1152.
With the new Wasmi bytecode and the macro sourced definitions we will be able to more or less simply move to tail call based dispatch once that is available.


Update the module soon

Merge it and give latest YT app support

But they are already here :P
I am just making copy of those commits on my fork - they are the same here and there :P


security: update latest containerd to 1.5.11-1 from 1.5.9-3 (for k8s >= 1.22)

What type of PR is this?
/kind bug

What this PR does / why we need it:
Updating containerd to 1.5.11+azure-1, from 1.5.9+azure-3, with fix for recent CVE-2022-24769 (https://nvd.nist.gov/vuln/detail/CVE-2022-24769).
Which issue(s) this PR fixes:

Fixes #
Requirements:

[ ] uses conventional commit messages

[ ] includes documentation
[ ] adds unit tests
[ ] tested upgrade from previous version

Special notes for your reviewer:
Release note:
This contains containerd CVE fix for CVE-2022-24769, for kubernetes versions >= 1.22 (which used containerd 1.5.9).

@anujmaheshwari1 seeing a couple issues with e2e, I think garbage collection bit us again? didn't peek at logs here but saw it in another PR


Threshold setting

Please provide threshold parameter/input (on the snScrollSpy directive?), as if answering the question: "what is the threshold value to consider an element as active?"
In my use case I would see two types of units (for both height and width) that I would like to express the threshold:

vh/vw (how much of vieport dimension does it have to occupy)
% (how much of its own clientWidth/clientHeight does the element have to reveale to consider it active)

There should probably also be a parameter to specify whether the calculation should include/exclude margins/padding.
In the below scenario (we see a fixed, semi-transparent navbar above the green line)

the content is scrolled to the very bottom (ie. bottom of last section, Section 4), and as you see, the link for that section is still not activated. With the proposed feature, if we specified that 60% of the height is enough to consider an element active, the link would get highlighted, which would correspond with the intuitive perception.

Closing as it's a dup of #1


Add parches for hardknott yocto release

Fixed comments. Also I have to close this MR and apply a new one - because source branch should be changed.


Add Morecon as genesis validator

Please can you open a new PR if you still want to join the testnet?


AppMeasurement plugins

Provide support in the Web SDK for appMeasurement plugins

Stephen Pelhan in the consulting org is leading this effort


Update latest Scala 2 and Scala 3 versions

Description
Updates to latest minor version of Scala 2 or patch version of Scala 3:

2.12.19 -> 2.12.20
2.13.14 -> 2.13.15
3.3.3    -> 3.3.4
3.5.0    -> 3.5.2

For reviewers:
The first commit contains most of important changes: changes to versions and SHA's
The next commits are fixing rest of the build to ensure usage of the latest version in tests

Thanks, @WojciechMazur!


koord-scheduler: inplace update quota min and max

Ⅰ. Describe what this PR does
inplace update quota min and max

Ⅱ. Does this pull request fix one issue?

Ⅲ. Describe how to verify it
Ⅳ. Special notes for reviews
V. Checklist

[ ] I have written necessary docs and comments
[ ] I have added necessary unit tests and integration tests
[ ] All checks passed in make test

/lgtm


WiFi example does not work

I cloned mruby-esp32, and tried the examples.
While simplest and system worked, wifi_example did not work.
When just editing the ssid and password, it crashes with a stack overflow
https://gist.github.com/ndac-todoroki/a01a35f956bb5159b49b9f3d6bce2cd4
and commenting out the loop do ... end results in a error somewhere around the garbage collection.
https://gist.github.com/ndac-todoroki/1465c52f97f195b864767849dbdcacc9
I'm compiling and uploading the file from Ubuntu 17.04 with gcc 7.2.0 .
and the latest idf tools.
Does this still work on your mac OS?

Hi! Is this an issue that might closed with just an ammeded README.md, or would you rather correct the mruby_main.c file? I have a Pull Request ready for merging (PR #12) with the adjusted README.md, and would be glad to perform the simple alteration to the aforementioned file. I tested the other examples and they worked fine.


[Feature] llama model size selection

Thank you so much for your open-source project.
Have you considered using llama models of larger sizes?
After completing several finetuning tasks, I found that the current llama model, even after finetuning training, has difficulty learning the difference between synonyms in different contexts.
For example, the word '滚'  in the Chinese blessing '财运滚滚' was read like a swearing.
Perhaps this is the limitation of small-size llama. Or I may need more complex training data to finetune. (currently using 20 hours)
Maybe offering different sizes like openai-whisper would be a good solution.

hello，you seem to have successfully trained ,can you help me for this ,when i run train.py ,i meet this bug.
https://github.com/fishaudio/fish-speech/issues/59

hello,have you meet the bug : nstantiationException: Error locating target 'fish_speech.models.vqgan.VQGAN', can you help me?

Currently, we are working on a larger and more diversed dataset, hope this can help mitigating synonyms issue.


Handling failures correctly and adding timeout status

Instead reporting any result that's not OK as a FAIL, it will keep the error message (TIMEOUT, FAIL, ERROR, etc)
Fixes this issue in the upload result script as well

test this please
HOST_OSES=ALL
BUILD_TOOLCHAINS=GCC_ARM
TARGETS=NRF51822,NUCLEO_L053R8


RFC - Simplified @connection directive

Is your feature request related to a problem? Please describe.
Using @connection with keyName is complex and using it with name requires extra naming work and its deletion behavior can be ambiguous. I've thought of a different approach that may be simpler, not sure.
Describe the solution you'd like
Maybe something like:
@connection(reverseProperty: String!, propagateDeletion: Boolean!)
And possibly a convenience directive:
@connection(propagateDeletion: Boolean!)
The term reverseProperty could also be inverseProperty, but either would make sense I think.
With this approach you don't need to think about:

Keys
Foreign key placement
Ownership semantics like "belongs to" vs "has one"
Naming connections or keys

Multiplicity and optionality are specified in the normal way ([], !), and many-to-many relationships can either be explicit (using [] at both ends) or implicit (using two one-to-many relationships like can be done now).
I think it might make the model easier to reason about too. I haven't worked through it thoroughly or anything, just curious what people think.
Example:
type PhoneCall @model {
  id: ID!
  caller: Person! @connection(reverseProperty: "callerOf", propagateDeletion: false)
  callee: Person! @connection(reverseProperty: "calleeOf", propagateDeletion: false)
}

type Person @model {
  id: ID!
  name: String!
  callerOf: [PhoneCall!] @connection(reverseProperty: "caller", propagateDeletion: true)
  calleeOf: [PhoneCall!] @connection(reverseProperty: "callee", propagateDeletion: true)
}

Closing this out as we've tried to vastly simplify the relationship directives and separated them out by use cases "hasOne", "hasMany", "belongsTo", and "manyToMany" https://docs.amplify.aws/cli/graphql/data-modeling/#setup-relationships-between-models


Section headings no longer have anchors

You used to be able to send people links to a specific section, such as http://docs.deis.io/en/latest/troubleshooting_deis/#troubleshooting-etcd

I know @krancour has been getting around this by using raw HTML in the docs. For example ## Logger: fluentd, logger at https://github.com/deis/workflow/blob/master/src/understanding-workflow/components.md

Ya... Markdown allows you to drop to HTML for certain things... explicitly adding anchors to section headings is one of them.  But I've done that in cases where I needed to link from one place in the document to another... i.e. in cases where I needed to know (or control) what the anchor's name was.
I think in the general case, however, that we used to get anchors automatically generated for all section headings.  We could mouseover a section heading to make a little icon appear that you could click to take you directly to that section... which meant you could then copy the URL, including the anchor, from the address bar.  This isn't happening anymore... and I would presume it's a function of the change in how docs are generated.  i.e. Maybe mkdocs doesn't do this for us?

I think in the general case, however, that we used to get anchors automatically generated for all section headings.

:+1: I'd love to see this as an automatic thing, as it once was.

Maybe mkdocs doesn't do this for us?

Pretty sure we were using mkdocs in v1, too, no?

@carmstrong we were still using sphinx in v1: https://github.com/deis/deis/blob/master/docs/conf.py

we were still using sphinx in v1

Ah, okay.

@bacongobbler I don't know mkdocs well.  Do you know if this is something it can do if enabled/configured correctly?

This is still currently something that is yet to be implemented. See https://github.com/mkdocs/mkdocs/issues/248

Actually, if we just enable the HeaderID extension we should get exactly what we're looking for. This can be easily plugged in. See https://github.com/deis/workflow/blob/master/mkdocs.yml#L68 and https://github.com/deis/workflow/blob/master/requirements.txt#L2 for an example on how we add custom extensions to mkdocs.


Replicator syncing data very slowly

Hello. I am trying to run our own  Hub and replicator on Google Cloud. My end goal is to sync Farcaster data into our Postgres database hosted also on Google.
Db configuration is 96 vCPU, 768 Memory.
Hubble is running on a VM with 80 vCPU. Syncing the Hub was pretty quick, 15-20 mins.
The replicator is running on the same VM, connected to our hub, concurrency set to 800.
My issue is that even though the machines seem to be using maximum 80% of their resources syncing into postgres is still super slow.
I am not receiving any errors, the logs say that backfilling the data should take approx 2 hours, but it is showing this since 10 hours.

I have also added some extra indexes to the db to improve select queries used by the replicator.
Is it possible to further speed up the syncing process, or due to the size of the data this is the best there is?
Thanks for the help!

+1
I am trying to replicate to Postgres too but on a much smaller hardware, it's been running for 5 days already and only got halfway so far. My guess is there are lots of index in the database that slow down the inserts.

@randombishop How much data do you have already in PG? For us replicator synced more than 450 GB, which seems odd to me as our hub uses only 130 GB.

Hmm, 800 is a very high degree of concurrency, possible it's causing db contention. Could you try lowering it? Also how does db health look like? Are you seeing any deadlocks or timeouts?

Cloud sql allowed 1K connections, cpu usage was under 60% (with 96 vCPU).
Deadlocks only appeared for a 3 minute period for the past week since I am trying to replicate the data.
Lowering concurrency only increases backfill time.
I have seen a few of these Info messages after starting the replicator, but no errors.
Job 73816705 took too long and has stalled.

@randombishop Have you managed to sync the data into PG?

@flyingrabbit-lab Not yet, progress seems to be 80%:
Backfilled events for 479559 of 595301 FIDs. Estimated time remaining: 1 week, 2 days, 7 hours, 19 minutes, 14 seconds

Hey all, we've officially deprecated the Replicator due to issues such as this one. We recommend you have a look at the Shuttle package if you want to build a service that ingests messages from hubs into another data source. Thanks!


Project lock and zone status not handled on network timeout in landing_zone_move

(Updated from original ticket)
We have some cases where connections from celery to iRODS time out for some reason within landing_zone_move. This will not free the project lock or update the landing zone status.
Example:
sodar-celeryd-default_1  | [2023-11-30 10:37:41,753: INFO/MainProcess] Task taskflowbackend.tasks_celery.submit_flow_task[5393f54b-0f52-440c-8453-abe434e06f18] received
sodar-celeryd-default_1  | [2023-11-30 10:37:41,967: INFO/ForkPoolWorker-15] Lock OK for project xxxxx
(...)
sodar-celeryd-default_1  | [2023-11-30 10:40:31,426: CRITICAL/ForkPoolWorker-15] The read operation timed out
sodar-celeryd-default_1  | [2023-11-30 10:40:31,438: ERROR/ForkPoolWorker-15] Could not receive server response

A bit later we get:
sodar-celeryd-default_1  | [2023-11-30 10:42:31,647: WARNING/ForkPoolWorker-15] Exception ignored in:
sodar-celeryd-default_1  | [2023-11-30 10:42:31,648: WARNING/ForkPoolWorker-15] <function Connection.__del__ at 0x7f4e7ba76670>
sodar-celeryd-default_1  | [2023-11-30 10:42:31,648: WARNING/ForkPoolWorker-15] Traceback (most recent call last):
sodar-celeryd-default_1  | [2023-11-30 10:42:31,648: WARNING/ForkPoolWorker-15]   File "/usr/local/lib/python3.8/site-packages/irods/connection.py", line 90, in __del__
sodar-celeryd-default_1  | [2023-11-30 10:42:31,649: WARNING/ForkPoolWorker-15]
sodar-celeryd-default_1  | [2023-11-30 10:42:31,649: WARNING/ForkPoolWorker-15] self.disconnect()
sodar-celeryd-default_1  | [2023-11-30 10:42:31,649: WARNING/ForkPoolWorker-15]   File "/usr/local/lib/python3.8/site-packages/irods/connection.py", line 306, in disconnect
sodar-celeryd-default_1  | [2023-11-30 10:42:31,649: WARNING/ForkPoolWorker-15]
sodar-celeryd-default_1  | [2023-11-30 10:42:31,650: WARNING/ForkPoolWorker-15] self.socket = self.socket.unwrap()
sodar-celeryd-default_1  | [2023-11-30 10:42:31,650: WARNING/ForkPoolWorker-15]   File "/usr/local/lib/python3.8/ssl.py", line 1285, in unwrap
sodar-celeryd-default_1  | [2023-11-30 10:42:31,651: WARNING/ForkPoolWorker-15]
sodar-celeryd-default_1  | [2023-11-30 10:42:31,651: WARNING/ForkPoolWorker-15] s = self._sslobj.shutdown()
sodar-celeryd-default_1  | [2023-11-30 10:42:31,651: WARNING/ForkPoolWorker-15] socket
sodar-celeryd-default_1  | [2023-11-30 10:42:31,651: WARNING/ForkPoolWorker-15] .
sodar-celeryd-default_1  | [2023-11-30 10:42:31,651: WARNING/ForkPoolWorker-15] timeout
sodar-celeryd-default_1  | [2023-11-30 10:42:31,652: WARNING/ForkPoolWorker-15] :
sodar-celeryd-default_1  | [2023-11-30 10:42:31,652: WARNING/ForkPoolWorker-15] The read operation timed out

This should be caught and, at the very least, the project lock should be freed.

Closing, see #1535 and #1537.

I closed this yesterday but since reconsidered. I should still look into updating the status if we enter a crash like this.

It turned out my most recent sighting of this was a misunderstanding. A taskflow job was slowed down by external issues, so it appeared to be hanging, while actually still running. Meanwhile, another operation timed out for "natural causes", leading me to confuse the two for eachother. Mental note: learn to read celery logs :)


Execution speed up

load waves into data folders.
Improve calculation efficiencies.

waves into data folders done https://github.com/quantixed/CellMigration/commit/0aa09feba34c2ef66d9749acd9e36528cc3994a3

no calc efficiencies that wouldn't break something


Lost a sign in redditHot

You're not getting into account a sign you've got in this line.
So the result will be same for both:
const date = new Date();
const hotness = redditHot();

hotness(100, 3, date);
// 1.9867717342662448

hotness(3, 100, date);
// 1.9867717342662448

Wow. Yeah that is clearly wrong. Thanks for reporting this.
Appears I just forgot to take the sign into the final account.
I'll add a fix in a bit.


Un-break CatchedException and derived exceptions

Exceptions must inherit from BaseException (in rare cases) or Exception (the regular case).
This fixes #338. Ideally, the test suite should actually try and use the subclassed exceptions, however.

Coverage remained the same at 99.739% when pulling 274097d3145d6c28f4054f28a10b109fc8c7ad6b on mjpieters:patch-1 into ad4c5402ca6f5cfa257ce54dfd46ea6cc1abdd41 on pydoit:master.


x/build: have darwin/amd64 builders on macOS 13 (Ventura)

This is the tracking issue like #55354, but for GOARCH=amd64.
CC @golang/release, @golang/darwin.

cc @prattmic

Is there anything left to do in order to close this issue?


[Bug]: Download as Excel feature for table widget

Is there an existing issue for this?

[X] I have searched the existing issues

Description

When I use the auto repair feature in Excel to open the file with above error, I get this message and the file's still broken:
Replaced Part: /xl/worksheets/sheet1.xml part with XML error.  Illegal xml character. Line 252, column 3636.
I wasn't able to replicate the issue on the dev environment of ours, the problem only occurs on our client's prod environment.
I feel this might have something to do with specific texts in the data of the table widget.
Steps To Reproduce

Populate a table widget with some data
Use table widget's download feature to download as excel
Open the excel to find the error message

Public Sample App
No response
Environment
Production
Issue video log
No response
Version
Self hosted v1.9.43

@BorenTsaiAMD it's a bit difficult to debug this issue without an example of the data where it fails. Could you help us with that data so we can debug it?


Internal Server Error

can request /genesis
but request other  error
{
"status_code": 500,
"error": "Internal Server Error",
"message": "An unexpected response was received from the backend."
}

Hi @Wbsba!
What is the network value inside your config file?
Could you please paste logs from the backend itself? It should contain the reason behind the error.

I think my configuration is wrong ,tnx


Return type for _Attrib.get() and .pop()

Currently the lxml.etree._Attrib stub is like this:
    # Signature is pop(self, key, *default), yet followed by runtime
    # check and raise exception if multiple default argument is supplied
    @overload
    def pop(self, key: _AttrName) -> str | None: ...
    @overload
    def pop(self, key: _AttrName, default: _T) -> str | _T: ...
[...]
    def __getitem__(self, __k: _AttrName) -> str: ...
[...]
    @overload
    def get(self, key: _AttrName) -> str | None: ...
    @overload
    def get(self, key: _AttrName, default: _T) -> str | _T: ...

I'm looking at the current lxml code and I think all these methods never return None, raising KeyError instead, so it makes sense to me to change both pop() and get() to match __getitem__() and return str when the default is not provided. What do you think?

@wRAR Sorry for late reply. It seems the story is a bit more complex than either you or me have thought.
In [15]: import lxml
In [16]: lxml.__version__
Out[16]: '4.9.1'

In [17]: import lxml.html
In [18]: root = lxml.html.fragment_fromstring('<input type="checkbox" checked>')
In [19]: root.attrib
Out[19]: {'type': 'checkbox', 'checked': 'checked'}

In [20]: y = root.attrib.get('foobar')
In [21]: type(y)
Out[21]: <class 'NoneType'>

In [22]: y = root.attrib.pop('xxx')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "src\lxml\etree.pyx", line 2466, in lxml.etree._Attrib.pop
KeyError: 'xxx'

'xxx'

I haven't checked earlier versions of lxml, but for simplicity I'll follow the behavior of newest version for now -- you're correct that _Attrib.pop() needs fixing.

Indeed, I've looked at the code again and now I see that get() can return None. Maybe I've confused it with __getitem__(), which throws.
Thanks!


Reporting in deeper modules

Currently, the report_metrics method is added by monkeypatching the model given to the Trainer class:
https://github.com/zipzou/hf-multitask-trainer/blob/d53f9d6b118d7e299e065e408833590f25f868dd/hf_mtask_trainer/trainer.py#L62-L68
However, what if you want to report values in deeper modules of the model? If model has two submodules model.child1 and model.child2, neither of them can use self.report_metrics. The only way to work around this would be to pass all the values you want to track from the children in their forward() call so that they can be logged by their parent, but clearly that doesn't scale well (if you want to track one value 10 modules deep, you have to alter the forward() method for all 10 modules).
Could this method be added recursively to all submodules? I'm pretty sure nn.Module.apply() is made for this purpose.

Currently, the report_metrics method is added by monkeypatching the model given to the Trainer class:
https://github.com/zipzou/hf-multitask-trainer/blob/d53f9d6b118d7e299e065e408833590f25f868dd/hf_mtask_trainer/trainer.py#L62-L68
However, what if you want to report values in deeper modules of the model? If model has two submodules model.child1 and model.child2, neither of them can use self.report_metrics. The only way to work around this would be to pass all the values you want to track from the children in their forward() call so that they can be logged by their parent, but clearly that doesn't scale well (if you want to track one value 10 modules deep, you have to alter the forward() method for all 10 modules).
Could this method be added recursively to all submodules? I'm pretty sure nn.Module.apply() is made for this purpose.

Thanks for your feedback, I think what you mentioned is reasonable, and I will update this implementation in the next future release, so that the report_metrics method can be called in submodules.


Assets everywhere ... and a futures fix.

The original point of this branch was to fix a bug in how we calculate a position's cost basis for futures (it wasn't taking into account contract size).
This led me into a refactoring that I had been meaning to do for a while, in which all mentions of sid were removed in favor of asset (the sid properties were almost always Asset objects already).

Coverage increased (+0.01%) to 87.544% when pulling a3a1ff0275e9d48ef814ad3a2a680050843be17a on futures-commissions into 4c334c6c381611705ad3c9664ffd507b3343d60b on master.

Coverage increased (+0.01%) to 87.546% when pulling 0cc81a53cc32053f56d8de011d097aa37858d40a on futures-commissions into 4c334c6c381611705ad3c9664ffd507b3343d60b on master.

Coverage increased (+0.01%) to 87.546% when pulling 0cc81a53cc32053f56d8de011d097aa37858d40a on futures-commissions into 4c334c6c381611705ad3c9664ffd507b3343d60b on master.

cc @ssanderson , the refactoring we have been discussing.

Coverage increased (+0.009%) to 87.541% when pulling aabb9142d33c68c7d18379620f80cf086de2b460 on futures-commissions into 4c334c6c381611705ad3c9664ffd507b3343d60b on master.

Coverage decreased (-0.4%) to 87.171% when pulling 1d0a2e367582515edf530795b27aba62017cc40d on futures-commissions into 4c334c6c381611705ad3c9664ffd507b3343d60b on master.

Coverage decreased (-0.05%) to 87.487% when pulling 735d6f4d11d39946a9ad58097dea1ac1c3745fbb on futures-commissions into 4c334c6c381611705ad3c9664ffd507b3343d60b on master.

Coverage increased (+0.08%) to 87.612% when pulling 6b477bcf949237f18b6f6a8fe10ead2a0381dad8 on futures-commissions into 35c3cf0eb72ce51a865045032a460b6644f72285 on master.

Coverage increased (+0.03%) to 87.557% when pulling f4bccf02edf3653fb8cdbd9e1017628e94d066f1 on futures-commissions into 35c3cf0eb72ce51a865045032a460b6644f72285 on master.

Coverage increased (+0.08%) to 87.611% when pulling f4bccf02edf3653fb8cdbd9e1017628e94d066f1 on futures-commissions into 35c3cf0eb72ce51a865045032a460b6644f72285 on master.

@ssanderson thanks for the review.  You were completely correct about the places that were now doing redundant lookups, and I've since removed assetfinder and multiplier caching from PositionTracker and the assetfinder from  PerformancePeriod.

@jbredeche finished a second pass.  Had a few small comments, but mostly looks good to me.

Coverage increased (+0.07%) to 87.578% when pulling 15d8dc93a3bfd46e96d1f5f55311fde01472b480 on futures-commissions into 123398d0e47f2087f545ca155aa5ee01a0c64c09 on master.

Coverage decreased (-0.3%) to 87.208% when pulling 15d8dc93a3bfd46e96d1f5f55311fde01472b480 on futures-commissions into 123398d0e47f2087f545ca155aa5ee01a0c64c09 on master.


The phlow seems to be broken

Somehow master seems to be hardcoded!

....the integration branch for this repo is gh-pages

@lakruzz Well this is totally expected behavior! From our requirements we wanted phlow to work with a config file, and return to a default behaviour! Which is origin/master if no phlow files are specified.. Hence the break in backwards compatibility. I refer to the documentation located here https://github.com/Praqma/git-phlow/tree/master/docs

Create the .phlow INI config file with the right configuration and try again! If that does not work there is a bug!


Update Pipelines to support Android 29 and JDK 11

@eman1986 - This PR is Implementing changes to the pipeline to reflect require changes for issue 359
This does the following

Sets up Android
Installs Android 29
Sets up JDK 11

#359

Looks like an easy fix.


[Bug] Problem compiling llama model

🐛 Bug

To Reproduce
Steps to reproduce the behavior:

Follow the guide at https://llm.mlc.ai/docs/deploy/android.html#create-android-project-using-compiled-models
When running command

# 2. compile: compile model library with specification in mlc-chat-config.json
mlc_chat compile ./dist/${MODEL_NAME}-${QUANTIZATION}-MLC/mlc-chat-config.json \
    --device android -o ./dist/${MODEL_NAME}-${QUANTIZATION}-MLC/${MODEL_NAME}-${QUANTIZATION}-android.tar

Expected behavior

Environment

Platform (e.g. WebGPU/Vulkan/IOS/Android/CUDA):
Operating system (e.g. Ubuntu/Windows/MacOS/...): Windows
Device (e.g. iPhone 12 Pro, PC+RTX 3090, ...)
How you installed MLC-LLM (conda, source): prebuilt
How you installed TVM-Unity (pip, source): prebuilt
Python version (e.g. 3.10): 3.11
GPU driver version (if applicable):
CUDA/cuDNN version (if applicable):
TVM Unity Hash Tag (python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))", applicable if you compile models):
Any other relevant information:

Additional context

[2024-03-03 15:48:36] INFO auto_config.py:69: Found model configuration: Llama-2-7b-chat-hf-q4f16_1-MLC\mlc-chat-config.json
[2024-03-03 15:48:36] INFO auto_config.py:153: Found model type: llama. Use `--model-type` to override.
[2024-03-03 15:48:36] WARNING auto_target.py:304: --system-lib-prefix is automatically picked from the filename, llama_q4f16_1_, this allows us to use the filename as the model_lib in android/iOS builds. Please avoid renaming the .tar file when uploading the prebuilt.
Compiling with arguments:
  --config          LlamaConfig(hidden_size=4096, intermediate_size=11008, num_attention_heads=32, num_hidden_layers=32, rms_norm_eps=1e-05, vocab_size=32000, position_embedding_base=10000, context_window_size=768, prefill_chunk_size=768, num_key_value_heads=32, head_dim=128, tensor_parallel_shards=1, max_batch_size=80, kwargs={})
  --quantization    GroupQuantize(name='q4f16_1', kind='group-quant', group_size=32, quantize_dtype='int4', storage_dtype='uint32', model_dtype='float16', linear_weight_layout='NK', num_elem_per_storage=8, num_storage_per_group=4, max_int_value=7)
  --model-type      llama
  --target          {"thread_warp_size": 1, "host": {"kind": "llvm", "tag": "", "keys": ["arm_cpu", "cpu"], "mtriple": "aarch64-linux-android"}, "texture_spatial_limit": 16384, "max_threads_per_block": 256, "max_function_args": 128, "max_num_threads": 256, "kind": "opencl", "max_shared_memory_per_block": 16384, "tag": "", "keys": ["opencl", "gpu"]}
  --opt             flashinfer=0;cublas_gemm=0;cudagraph=0
  --system-lib-prefix "llama_q4f16_1_"
  --output          Llama-2-7b-chat-hf-q4f16_1-MLC\Llama-2-7b-chat-hf-q4f16_1-android.tar
  --overrides       context_window_size=None;sliding_window_size=None;prefill_chunk_size=None;attention_sink_size=None;max_batch_size=None;tensor_parallel_shards=None
[2024-03-03 15:48:36] INFO compile.py:135: Creating model from: LlamaConfig(hidden_size=4096, intermediate_size=11008, num_attention_heads=32, num_hidden_layers=32, rms_norm_eps=1e-05, vocab_size=32000, position_embedding_base=10000, context_window_size=768, prefill_chunk_size=768, num_key_value_heads=32, head_dim=128, tensor_parallel_shards=1, max_batch_size=80, kwargs={})
[2024-03-03 15:48:37] INFO compile.py:154: Exporting the model to TVM Unity compiler
[2024-03-03 15:48:39] INFO compile.py:160: Running optimizations using TVM Unity
[2024-03-03 15:48:39] INFO compile.py:173: Registering metadata: {'model_type': 'llama', 'quantization': 'q4f16_1', 'context_window_size': 768, 'sliding_window_size': -1, 'attention_sink_size': -1, 'prefill_chunk_size': 768, 'tensor_parallel_shards': 1, 'kv_cache_bytes': 0}
[2024-03-03 15:48:39] INFO pipeline.py:42: Running TVM Relax graph-level optimizations
[2024-03-03 15:49:37] INFO pipeline.py:42: Lowering to TVM TIR kernels
[2024-03-03 15:49:42] INFO pipeline.py:42: Running TVM TIR-level optimizations
[2024-03-03 15:49:53] INFO pipeline.py:42: Running TVM Dlight low-level optimizations
[15:49:53] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:53] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:53] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:53] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:53] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:53] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:53] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:53] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:54] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:54] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:54] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:54] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:54] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:54] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:54] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:54] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:54] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:54] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:54] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:54] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[2024-03-03 15:49:54] INFO pipeline.py:42: Lowering to VM bytecode
[15:49:55] D:\a\package\package\tvm\src\relax\ir\expr.cc:39: Check failed: (!op->struct_info_.defined() || op->struct_info_->IsInstance<FuncStructInfoNode>()) is false: ValueError: Call expects its operator to have FuncStructInfo, but operator I.GlobalVar("fuse_add_norm_prefill"), which was called with arguments [lv1, input_embeds, model_layers_0_post_attention_layernorm_weight5, alloc4, alloc5], has struct info R.Object
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[15:49:55] D:\a\package\package\tvm\src\relax\ir\block_builder.cc:65: Warning: BlockBuilder destroyed with remaining blocks!
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Scripts\mlc_chat.exe\__main__.py", line 7, in <module>
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\mlc_chat\__main__.py", line 24, in main
    cli.main(sys.argv[2:])
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\mlc_chat\cli\compile.py", line 131, in main
    compile(
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\mlc_chat\interface\compile.py", line 226, in compile
    _compile(args, model_config)
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\mlc_chat\interface\compile.py", line 175, in _compile
    args.build_func(
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\mlc_chat\support\auto_target.py", line 175, in build
    relax.build(
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\tvm\relax\vm_build.py", line 335, in build
    mod = pipeline(mod)
          ^^^^^^^^^^^^^
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\tvm\ir\transform.py", line 238, in __call__
    return _ffi_transform_api.RunPass(self, mod)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\tvm\_ffi\_ctypes\packed_func.py", line 239, in __call__
    raise_last_ffi_error()
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\tvm\_ffi\base.py", line 481, in raise_last_ffi_error
    raise py_err
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\tvm\_ffi\_ctypes\packed_func.py", line 82, in cfun
    rv = local_pyfunc(*pyargs)
         ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\mlc_chat\compiler_pass\pipeline.py", line 145, in _pipeline
    mod = seq(mod)
          ^^^^^^^^
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\tvm\ir\transform.py", line 238, in __call__
    return _ffi_transform_api.RunPass(self, mod)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\tvm\_ffi\_ctypes\packed_func.py", line 239, in __call__
    raise_last_ffi_error()
  File "C:\Users\yeoph\AppData\Local\Programs\Python\Python311\Lib\site-packages\tvm\_ffi\base.py", line 481, in raise_last_ffi_error
    raise py_err
tvm._ffi.base.TVMError: Traceback (most recent call last):
  File "D:\a\package\package\tvm\src\relax\ir\expr.cc", line 39
ValueError: Check failed: (!op->struct_info_.defined() || op->struct_info_->IsInstance<FuncStructInfoNode>()) is false: Call expects its operator to have FuncStructInfo, but operator I.GlobalVar("fuse_add_norm_prefill"), which was called with arguments [lv1, input_embeds, model_layers_0_post_attention_layernorm_weight5, alloc4, alloc5], has struct info R.Object

Hey @sherelynyap thanks for reporting. I feel that this issue might have been fixed by this commit https://github.com/mlc-ai/mlc-llm/commit/9037aae1bbda663f1ba41487b2ef073bc908ad50. Could you help confirm if your mlc_chat package is the latest? Or please let me know if the issue persists.

I was sing the latest package, and still has the same error. "/concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time."

hmm i think i was following the old workflow using mlc_chat instead of mlc_llm. It works with mlc_llm

It seems to compile successfully? But it still prints the error so i'll leave this issue open for now
[14:49:27] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:28] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:28] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:28] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:28] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:28] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:28] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:28] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:28] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:28] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:28] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:28] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:29] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:29] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:29] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:29] D:\a\package\package\tvm\src\tir\schedule\./concrete_schedule.h:287: ValueError: The block no longer exists in the IRModule
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:29] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:29] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:29] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:29] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:29] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:29] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:29] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[14:49:29] D:\a\package\package\tvm\src\tir\ir\stmt.cc:122: InternalError: Check failed: (e.dtype().bits() <= loop_var.dtype().bits()) is false:  Loop variable's dtype (int32) is narrower than that of `min` or `extent` (int64)
Stack trace not available when DMLC_LOG_STACK_TRACE is disabled at compile time.

[2024-03-16 14:49:29] INFO pipeline.py:48: Lowering to VM bytecode
[2024-03-16 14:49:31] INFO estimate_memory_usage.py:55: [Memory usage] Function `alloc_embedding_tensor`: 6.00 MB
[2024-03-16 14:49:31] INFO estimate_memory_usage.py:55: [Memory usage] Function `batch_decode`: 9.02 MB
[2024-03-16 14:49:31] INFO estimate_memory_usage.py:55: [Memory usage] Function `batch_prefill`: 86.62 MB
[2024-03-16 14:49:31] INFO estimate_memory_usage.py:55: [Memory usage] Function `batch_verify`: 86.62 MB
[2024-03-16 14:49:31] INFO estimate_memory_usage.py:55: [Memory usage] Function `create_tir_paged_kv_cache`: 0.00 MB
[2024-03-16 14:49:32] INFO estimate_memory_usage.py:55: [Memory usage] Function `decode`: 0.11 MB
[2024-03-16 14:49:32] INFO estimate_memory_usage.py:55: [Memory usage] Function `embed`: 6.00 MB
[2024-03-16 14:49:32] INFO estimate_memory_usage.py:55: [Memory usage] Function `prefill`: 86.63 MB
[2024-03-16 14:49:32] INFO estimate_memory_usage.py:55: [Memory usage] Function `softmax_with_temperature`: 0.00 MB
[2024-03-16 14:49:33] INFO pipeline.py:48: Compiling external modules
[2024-03-16 14:49:33] INFO pipeline.py:48: Compilation complete! Exporting to disk
[2024-03-16 14:49:39] INFO model_metadata.py:96: Total memory usage: 3701.77 MB (Parameters: 3615.13 MB. KVCache: 0.00 MB. Temporary buffer: 86.63 MB)
[2024-03-16 14:49:39] INFO model_metadata.py:105: To reduce memory usage, tweak `prefill_chunk_size`, `context_window_size` and `sliding_window_size`
[2024-03-16 14:49:39] INFO compile.py:194: Generated: dist\finllama_chat-q4f16_1-MLC\finllama_chat-q4f16_1-android.tar

Hi @sherelynyap, from that msg I cannot directly tell if the compilation succeeded or not, since I didn't see the “error msgs” there before. Do you mind letting us know the followup? Like how the deployment on Android goes.

the latest stage i got to was https://github.com/mlc-ai/mlc-llm/issues/1923#issuecomment-2012515615


uv.lock is out of sync after running a release

Question
How can I keep my uv.lock file in sync with my other files like pyproject.toml?
I reviewed https://github.com/python-semantic-release/python-semantic-release/issues/1097 but my situation is different in two important ways:

I'm not building using psr; I have my own custom logic for that which is handled after the release is cut.
We use the GitHub Action and my uv uses private indexes so I would need to find a way to pass numerous secrets / specially formatted env vars into the runtime using the prior approach.

Suggestion
When I update my uv.lock, the change looks something like this:
 [[package]]
 name = "example"
-version = "0.7.0"
+version = "0.10.0"

There's no need for any other complex locking behavior (hash pinning, etc.).
It seems like we could be able to specify this version update with a version_toml configuration or similar. However the problem is that we need to match on the [[package]] where name = "example" and then update the corresponding version field. I considered version_variables for this, which shows the potential to work but feels a bit hacky.
Configuration

Semantic Release Configuration
[tool.semantic_release]
major_on_zero = false
version_toml = [
    "pyproject.toml:project.version",
]
version_variables = [
    "src/example/__init__.py:__version__",
]
changelog_file = "CHANGELOG.md"
commit_parser = "angular"
commit_author = "Example<EMAIL_ADDRESS>commit_message = "chore(release): example {version}"
tag_format = "example=={version}"

[tool.semantic_release.branches.release]
match = "main"

[tool.semantic_release.publish]
dist_glob_patterns = ["path_relative_to_root/example/dist/*"]
upload_to_vcs_release = true

[tool.semantic_release.remote]
name = "origin"
type = "github"
ignore_token_for_push = false
insecure = false

I reviewed https://github.com/python-semantic-release/python-semantic-release/issues/1097 but my situation is different in two important ways:

I'm not building using psr; I have my own custom logic for that which is handled after the release is cut.

We use the GitHub Action and my uv uses private indexes so I would need to find a way to pass numerous secrets / specially formatted env vars into the runtime using the prior approach.

The GitHub Action is only a convenience wrapper. It is not designed to fit all use cases. It the case it's not convenient then just install PSR as a Python package into your CI environment and call it directly from the shell. If you decide to use the build command then we provide build_command_env as a specification of the env variable names you want preserved into the build command shell environment.
Since you are not wanting to use the PSR build command, I would just run your uv lock as a separate step. In this case you would probably need to run the following:
# PSR stamps version into files (nothing else)
# building changelog is a matter of preference here (I wouldn't)
semantic-release -v version --skip-build --no-commit --no-tag [--no-changelog]

# run UV lock as pyproject.toml is updated with the next version
uv lock

# stage the lock file to ensure it is included in the PSR commit
git add uv.lock

# run PSR fully to create release
semantic-release -v version --skip-build

It seems like we could be able to specify this version update with a version_toml configuration or similar. However the problem is that we need to match on the [[package]] where name = "example" and then update the corresponding version field. I considered version_variables for this, which shows the potential to work but feels a bit hacky.

We don't have an advanced toml parser to consider multiple keys to point to the correct value. Maybe we could if we had a clear way of defining it in the configuration but I don't know how that would work best.
version_variables is only a complex Regular Expression replace and likely will have undesirable effects on your uv.lock file as there are multiple version = specifications which will all be replaced regardless of which package it is related to.
Neither of these current implementations will work for your use case. The best recommendation I have is the double PSR call above.

Makes sense. I will move off the GitHub action for now.
As far as a more complex solution I may be willing to contribute something if we can agree on a design that makes sense. My first thought was some sort of per line predicate for the toml config? We could make it optional and invoked with a second : perhaps? Not sure exactly what it would look like and if you'd even be open to that. Or if people may already have a second : in those lines which would make it require a different approach

Regarding the regex my thought was to handle multiple lines to cover that issue about hitting other versions. I do still believe it'd be fragile and not sure if multiline would work; I only briefly looked at the implementation

@codejedi365 thanks; I confirmed this all works e2e. Closing this for now; you're welcome to reopen if you want to pursue the enhancement


any ways to reduce model file size?

can we reduce the generated model file size? it takes too much time on loading from a web server.

@bourdakos1 thanks for the reply
Could u give me  a clue on how to quantize the model for mobile web？ Do you have plan to make quantization a part of calic tool ? thanks.

hi @bourdakos1,
any updates on this. my current converted web_model is of about 23MB in size.
Request you to please add quantize functionality for model.
23MB downloads took too much time.

To be honest, quantized model support isn’t on my immediate road map, I’ve been very busy working on the cloud annotations GUI lately. However, I’m happy to merge any pull requests that add the feature. The TensorFlow object detection api has support so it shouldn’t be too difficult to add

In the mean time you can follow this to create a quantized ssd model instead of faster rcnn

see issue #49


Allow custom loggers in the VSTestSettings

VSTestSettings has a property called "Logger" with two possible values: None or Trx.
However, we need to be able to specify custom values. For example, when running on AppVeyor we can specify "/logger:AppVeyor".
As a workaround, I was able to specify the AppVeyor logger using ArgumentCustomization like so:
Task("Run-Code-Coverage")
	.IsDependentOn("Build")
	.Does(() =>
{
	var testAssemblyPath = string.Format("./MyLibraryUnitTests/bin/{0}/MyLibrary.UnitTests.dll", configuration);
	var vsTestSettings = new VSTestSettings();
	if (AppVeyor.IsRunningOnAppVeyor) vsTestSettings.ArgumentCustomization = args => args.Append("/logger:Appveyor");

	OpenCover(
		tool => { tool.VSTest(testAssemblyPath, vsTestSettings); },
		new FilePath("./CodeCoverageData/coverage.xml"),
		new OpenCoverSettings()
			.ExcludeByAttribute("*.ExcludeFromCodeCoverage*")
			.WithFilter("+[MyLibrary]*")
			.WithFilter("-[MyLibrary]MyLibrary.Properties.*")
	);
});

A few ideas were suggested:

Change the Logger property to a string and create some nice extension methods to accomplish the same thing. Like .WithAppVeyorLogger() or similar (this would probably break existing scripts though).
Perhaps a CustomLogger string?

I propose the following:

Mark the 'Logger' property as obsolete (I don't know what you policy is regarding obsolete properties but I imagine you want to keep it around for a release or two before removing it).
Also mark the VSTestLogger enum as obsolete. It wil no longer be necessary when the 'Logger' property is removed.
Add a new string property called 'LoggerName'
Add a few extension methods such as:
.WithTrxLogger()
.WithAppVeyorLogger()
.WithCustomLogger(string loggerName)
.WithoutAnyLogger()
As much as possible, keep the obsolete 'Logger' property and the new property in sync (for example: set LoggerName to "trx" if Logger is set to VSTestLogger.Trx; set LoggerName to a blank string if Logger is set to VSTestLogger.None; etc) and throw an exception if, for some reason, the two properties can't be kept in sync.
Add a new value called 'Custom' to the VSTestLogger enum. This will make it a little bit easier to keep the two properties in sync

I would be happy to submit a PR with these changes.

@Jericho in general, this seems like a valid approach to me. Question though... Aren't 2 and 6 slightly contradictory?

They are not contradictory. #6 is necessary to keep the to properties in sync. For example if I invoke .WhitCustomLogger("mylogger") I will set the LoggerName to "mylogger" and I also need to be able to set Logger to VSTestLogger.Custom
To be clear, #6 is temporary, until we remove the obsolete property and obsolete enum.


[Cherry-pick] Update changes from GoogleCA

Cherry pick from #17152. Bot failed to resolve rebase conflict
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[X] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

@pitlv2109: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

codecov_istio_release-1.3
ffa35ba074322551887866eb0b6c24d72b37f913
link
/test codecov

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test codecov


PR: Don't print DeprecationWarning's that come from comm handlers (IPython console)

Description of Changes

Those warnings don't provide valuable information and only pollute the console.
Depends on spyder-ide/spyder-kernels#350

Issue(s) Resolved

Fixes #14928.
Affirmation
By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct: @ccordoba12

Hello @ccordoba12! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file external-deps/spyder-kernels/spyder_kernels/console/tests/test_console_kernel.py:

Line 963:5: E303 too many blank lines (2)
Line 966:1: E302 expected 2 blank lines, found 1


Negation Ability

I want to be able to give all but one file in a certain directory the same owners and none to the one special file.
For example, for the following directory foo:
foo/
    - bar.py
    - foobar.py
    - no-owners.py

I want bar.py and foobar.py to inherit owners from foo's parent directory OWNERS file but I don't want no-owners.py to inherit from the parent directory.
The only way I can do this right now is to add an OWNERS file under foo that looks like this:
set noparent
per-file bar.py=file://owners
per-file foobar.py=file://owners

or I can put no-owners.py under it's own directory and use the set noparent function under that directory.
This first method is unmaintainable as every time I want to add a new file or change a file name I have to touch the OWNERS file. The second method is difficult because some things may rely on the path of no-owners.py being under foo/
If there is a current ability to negate please send me to the docs, otherwise it would be great if we could add an ability to negate. Thanks!

Looking for the same feature. Let me know if you found it in docs anywhere.


It‘s too slow to show the selected chinese word in fuzzy finder

just with chinese word

I'll gladly accept help with this one as I have no idea about chinese input.

thanks


Unexpected indentation in Root.py

Impact of the bug
DBS migrate monitoring was impacted.
Describe the bug
A clear and concise description of what the bug is.
Please see https://github.com/dmwm/DBS/issues/644
How to reproduce it
Please see https://github.com/dmwm/DBS/issues/644
Expected behavior
no "Unexpected indentation" error
Additional context and error message
DBS currently is using WMCore 1.3.2. If the latest version is compatible or no major change affect to DBS , we can move to the current version. Otherwise, please provide new tag that are build from 1.3.2.

This issue has a proposed fix, but I just don't manage to get those unit tests fixed...
Even though it could be set aside, the proposed fix enabled around 10 unit tests on the WebTools package, which might be handy for the python3 validation. So I'm keeping it under the Q2 ToDo items for now.