Adds Gizmo line styles

Objective

Adds line styles to bevy gizmos, suggestion of #9400
Currently solid and dotted lines are implemented but this can easily be extended to support dashed lines as well if that is wanted.

Solution

Adds the enum GizmoLineStyle and uses it in each GizmoConfig to configure the style of the line.
Each "dot" in a dotted line has the same width and height as the line_width of the corresponding line.

Changelog

Added GizmoLineStyle to bevy_gizmos
Added a line_style: GizmoLineStyle  attribute to GizmoConfig
Updated the lines.wgsl shader and the pipelines accordingly.

Migration Guide

Any manually created GizmoConfig must now include the line_style attribute

Additional information
Some pretty pictures :)
This is the 3d_gizmos example with/without line_perspective:

And the 2d example:

At this point I think we should break gizmos 2d_example and 3d_example in a few parts, the examples are getting pretty cramped

At this point I think we should break gizmos 2d_example and 3d_example in a few parts, the examples are getting pretty cramped

I agree but I think this should wait until both #12252 and this are merged so we can have one "fancy_gizmos" example.

Thank you to everyone involved with the authoring or reviewing of this PR! This work is relatively important and needs release notes! Head over to https://github.com/bevyengine/bevy-website/issues/1290 if you'd like to help out.


Update APISnoop to consume test metadata of multiple release versions in conformance.yaml

The requirements for Conformance Test Comment Metadata when updating test in later releases:
" Release: indicates the Kubernetes release that the test was added to the conformance test suite. If the test was modified in subsequent releases then those releases should be included as well (comma separated) "
Please update APISnoop to consume the comma-separated subsequent release, always using the latest release.
It is noted that this will move the endpoints hit in the prior release to the latest release in the APISnoop data.
This anomaly would be reported in the Roadmap files tracking the conformance progress.

Done.
Working well. Thank you @zachmandeville

Done.
Working well. Thank you @zachmandeville


testing

in https://github.com/ploomber/core/pull/75 I introduced a change to reduce latency when capturing telemetry calls. we had a function that would check internet access by hitting google.com, if it failed, it didn't perform the telemetry call to posthog.
turns out, this was severely affecting performance so I tested what would happen if I removed the call and just make the posthog call regardless of internet status. turns out, nothing happens (calls to posthog don't break if there is no internet connection)
@neelasha23 just to double check, can you perform some testing? try calling the telemetry (log_api) module without internet connection and let me know if you find anything weird
ensure you set the debug flag so the testing events go to the debugging posthog project

Sure, working on this now!

I tired some calls telemetry.log_api(...) with internet on/off multiple times. The calls would have issue_76 mentioned in the action/ metadata. I didn't get any errors. But I dont see any posthog files for 15.8. How to verify ? @edublancas

shouldn't I be able to access the posthog logs of the calls made by running the posthog-analysis.ipynb file?


⚠ ClamAV has degraded performance

In 8fe0968, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 562 ms

Resolved: ClamAV performance has improved in 48c4d4d.


Add Apple silicon (M1) support

Address #397
This needs to be tested.
I had a hard time building/including lldb and I must doing something wrong as I got a 3GB extension 😅
Any help on lldb is appreciated (I followed this guide but it seems to be somewhat outdated)

Thanks for the PR!
Unfortunately, this is a bit more complicated:  LLDB you've just built is going to depend on system Python, which is fine for you, but I need it to work on different versions of OSX, so the extension includes Python as well.   I am using Python packages from  https://github.com/indygreg/python-build-standalone project, and as of now there is no arm64 version for OSX.

@vadimcn the system Python versions wont diverge for some time, no? it might be a good idea to merge this and then python-build-standalone gets support for m1, use that.

A major objective of mine is that the extension should "just work" when installed, - with zero prerequisites, across a wide range of OS versions currently in use.
To this end I maintain a fork of LLDB with minimized dependencies (and also added-in Rust support): libraries that aren't needed for IDE use were removed (libcurses, libedit), what cannot be removed, I link statically (libxml, libstdc, libgcc).
(Due to long build times, LLDB is not built as a part of CodeLLDB, but as a separate workflow, whose outputs CodeLLDB then  consumes).
CodeLLDB currently supports 5 platforms (x64, arm32, arm64 Linux, OSX and Windows), soon to become 6, or even 7 (if arm Windows becomes popular), so building manually all of that is not feasible, automated builds in cloud CI are a must.   Both LLDB and CodeLLDB are currently built in Azure Pipelines CI.  I am not sure what is their status of arm OSX support, I haven't seen anything mentioned explicitly yet.
Then, there's Python with a host of its dependencies.   Building a portable Python is a significant amount of work on its own.  Fortunately, I was able to find the Python Standalone Builds project, which shares the same portability objectives.   There are no arm64 builds for OSX yet, though, I am sure, this will come eventually.   Using system Python for this one platform is definitely a possibility, however it isn't installed by default, which breaks the "just works" objective.

... so given these complications, I am wondering whether getting the x64 version working via Rosetta would be easier.

Do you know how to set it up with Rosetta?

It seems to not work well after I compile according to this pr, he can go into debug, but errors occur when viewing data on the heap
configuration: {
  type: 'lldb',
  request: 'launch',
  name: 'run hello-rs',
  program: '${workspaceFolder}/target/debug/hello-rs',
  args: [],
  cwd: '${workspaceFolder}',
  sourceMap: {},
  sourceLanguages: [ 'rust' ],
  env: {
    RUST_BACKTRACE: 'short',
    ELECTRON_RUN_AS_NODE: '1',
    USER: 'lizhaofa',
    __CFBundleIdentifier: 'com.microsoft.VSCodeInsiders',
    COMMAND_MODE: 'unix2003',
    LOGNAME: 'lizhaofa',
    PATH: '/Users/lizhaofa/.jenv/shims:/Users/lizhaofa/.jenv/bin:/Users/lizhaofa/.cargo/bin:/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin',
    SSH_AUTH_SOCK: '/private/tmp/com.apple.launchd.iRbWh8iG3P/Listeners',
    SHELL: '/bin/zsh',
    HOME: '/Users/lizhaofa',
    __CF_USER_TEXT_ENCODING: '0x1F5:0x19:0x34',
    TMPDIR: '/var/folders/hy/gkr7_sm52wb575hrqymskz6h0000gn/T/',
    XPC_SERVICE_NAME: 'application.com.microsoft.VSCodeInsiders.192119.192125',
    XPC_FLAGS: '0x0',
    ORIGINAL_XDG_CURRENT_DESKTOP: 'undefined',
    VSCODE_NLS_CONFIG: '{"locale":"zh-cn","availableLanguages":{},"_languagePackSupport":true}',
    VSCODE_NODE_CACHED_DATA_DIR: '/Users/lizhaofa/Library/Application Support/Code - Insiders/CachedData/4a875e23d20b64504a818834f3fa4c40adb8d480',
    VSCODE_IPC_HOOK: '/Users/lizhaofa/Library/Application Support/Code - Insiders/1.53.0-insider-main.sock',
    VSCODE_PID: '1197',
    SHLVL: '0',
    PWD: '/',
    OLDPWD: '/',
    ZSH: '/Users/lizhaofa/.oh-my-zsh',
    PAGER: 'less',
    LESS: '-R',
    LSCOLORS: 'Gxfxcxdxbxegedabagacad',
    EDITOR: 'vim',
    JENV_SHELL: 'zsh',
    JENV_LOADED: '1',
    _: '/Applications/Visual Studio Code - Insiders.app/Contents/MacOS/Electron',
    AMD_ENTRYPOINT: 'vs/workbench/services/extensions/node/extensionHostProcess',
    PIPE_LOGGING: 'true',
    VERBOSE_LOGGING: 'true',
    VSCODE_LOG_NATIVE: 'false',
    VSCODE_IPC_HOOK_EXTHOST: '/var/folders/hy/gkr7_sm52wb575hrqymskz6h0000gn/T/vscode-ipc-a06d2737-9841-49be-9364-e7611fc8dc84.sock',
    VSCODE_HANDLES_UNCAUGHT_ERRORS: 'true',
    VSCODE_LOG_STACK: 'true',
    APPLICATION_INSIGHTS_NO_DIAGNOSTIC_CHANNEL: 'true'
  },
  relativePathBase: '/Users/lizhaofa/Documents/Projects/hello-rs'
}
Listening on port 54062
[2020-12-28T02:15:42Z ERROR codelldb::debug_session] Initialize Python interpreter: Error:  Fail
Error Message:
error: module importing failed: This script interpreter does not support importing modules.

Assertion failed: (Prior->FD->hasAttr<NoUniqueAddressAttr>() && "should not have reused this field's tail padding"), function clipTailPadding, file /Users/lizhaofa/Documents/Projects/llvm-project/clang/lib/CodeGen/CGRecordLayoutBuilder.cpp, line 743.
Received signal: SIGABRT
   0: backtrace::backtrace::trace_unsynchronized
   1: backtrace::backtrace::trace
   2: backtrace::capture::Backtrace::create
   3: backtrace::capture::Backtrace::new
   4: codelldb::hook_crashes::handler
   5: __platform_memmove
   6: __pthread_atfork_parent
   7: <unknown>
   8: <unknown>

Debug adapter exit code=255, signal=null.

... so given these complications, I am wondering whether getting the x64 version working via Rosetta would be easier.

I believe it is a this stage. Eventually, Rosetta will be going away so it has to be native! ;)

It seems to not work well after I compile according to this pr, he can go into debug, but errors occur when viewing data on the heap

Assertion failed: (Prior->FD->hasAttr<NoUniqueAddressAttr>() && "should not have reused this field's tail padding"), function clipTailPadding, file /Users/lizhaofa/Documents/Projects/llvm-project/clang/lib/CodeGen/CGRecordLayoutBuilder.cpp, line 743.
Thanks Zoain-it  look like a problem with llvm, not vscode-lldb itself. Or maybe an integration issue...

Maybe we should try the llvm maintained by rust

python-build-standalone already supports OSX arm64, can we start providing support for OSX arm64

Apple M1 is supported as of v1.6.2


Jets does not support multiple value response headers

Checklist

[ x] Upgrade Jets: Are you using the latest version of Jets? This allows Jets to fix issues fast. There's a jets upgrade command that makes this a simple task. There's also an Upgrading Guide: http://rubyonjets.com/docs/upgrading/
[x ] Reproducibility: Are you reporting a bug others will be able to reproduce and not asking a question. If you're unsure or want to ask a question, do so on https://community.rubyonjets.com
[ ] Code sample: Have you put together a code sample to reproduce the issue and make it available?  Code samples help speed up fixes dramatically. If it's an easily reproducible issue, then code samples are not needed. If you're unsure, please include a code sample.

My Environment

Software
Version

Operating System
OSX 10.15.7

Jets
3.0.5

Ruby
2.7.2

Expected Behaviour
Setting response headers with multiple values should create a seperate response header for each value.
Current Behavior
Currently, settings a header to multiple values results in a single response header containing all values.
Step-by-step reproduction instructions
Create a multi-value header and deploy to staging.
Note: This issue does not occur locally using  puma
set_header('Set-Cookie', ['cookie_a=a', 'cookie_b=b'])
Solution Suggestion
Use API gateways"multiValueHeaders" field when rendering a response.

I will have a PR for this issue shortly.

Fixed in #556


security updatescripture-utilities

also use USJ consts in editor extension

[!NOTE]
This PR was draft because there was still one copy of @biblionexus-foundation/scripture-utilities at v0.0.4. This is now fixed. Note there is a way to override npm to fix this (see commit 2 which will be squashed on merge) but it is properly fixed in<EMAIL_ADDRESS>

This change is

extensions/src/legacy-comment-manager/package.json line 37 at r2 (raw file):
Previously, tjcouch-sil (TJ Couch) wrote…
Does the tilde ~ mean only update to newer patch versions? Why choose this option if so?

I put a a fuller explanation in our DM in Discord.


Quota message incorrect

Following is the over quota message that I get when I push to go over quota (as a test).
Note that the contact email address should be<EMAIL_ADDRESS> The help page mentioned should be https://help.hydroshare.org/about-hydroshare/policies/quota/ and should be a link.  There should be space between numbers and units.  This message appears when I log in, so the words "Your action to add content ..." do not make sense.  Please refer to earlier guidance on these messages to get these right.

Note that I also did not get this message in email (yet).  I understood that over quota email messages were supposed to be sent.

@hyi Can you look into the nightly quota emails on prod?

@aphelionz The quota email content is retrieved from quota field setup from admin panel, so I think if you go to admin and edit the quota content, the email should reflect the updated quota content.

Closing.  I think we are OK for now


Generic Linux install Desktop Client instructions don't work

The 'generic Linux' install instructions for the Mattermost Desktop Client don't work.
A [sic] AppImage distribution (Beta) of a compressed tarball is available. Automatic app updates are supported and enabled. When a new version of the desktop app is released, your app updates automatically.
What is it? An AppImage or a binary in a compressed tarball? Confusing wording. I expected an AppImage, but get a tarball, not an AppImage. I think the sentence should read: "An AppImage distribution AND a compressed tabball of a binary are available." Both are downloadeble here: https://github.com/mattermost/desktop/releases/tag/v5.8.1 but in the docs the AppImage gets and awkward mention convoluted with the tarball and then only the tarball is linked as a download.
Following the instructions:
2. Extract the archive to a convenient location, then execute mattermost-desktop located inside the extracted directory.
$ ./mattermost-desktop
[19564:0623/180528.261713:FATAL:setuid_sandbox_host.cc(158)] The SUID sandbox helper binary was found, but is not configured correctly. Rather than run without sandboxing I'm aborting now. You need to make sure that /home/mxa/Applications/mattermost-desktop-5.8.1-linux-x64/chrome-sandbox is owned by root and has mode 4755.
Trace/breakpoint trap (core dumped)

Thanks for reporting! We have included those resolution steps in the Desktop app's tarball installation instructions, as on Ubuntu Noble it seems the only way to give chrome-sandbox the right ownership and permissions after extracting the tarball as a non-root user.

Btw, I appreciate the speedy support and resolution here. That is a good user experience.


Increase code coverage in testing

Is your feature request related to a problem? Please describe.
Through the use of coverage current code coverage in test suite is 33%. Coverage of stencils should optimally be at ~75%
Output from coverage:
Name                                                       Stmts   Miss Branch BrPart  Cover
--------------------------------------------------------------------------------------------
pyFV3/_config.py                                             281     42     84      2    80%
pyFV3/dycore_state.py                                        120     39     52      1    64%
pyFV3/initialization/analytic_init.py                         20     11      6      0    35%
pyFV3/initialization/init_utils.py                           139     92     12      0    31%
pyFV3/initialization/test_cases/initialize_baroclinic.py      89     67      4      0    24%
pyFV3/initialization/test_cases/initialize_tc.py             164    164      2      0     0%
pyFV3/stencils/a2b_ord4.py                                   252    125     98     12    47%
pyFV3/stencils/c_sw.py                                       174    120     68      3    24%
pyFV3/stencils/d2a2c_vect.py                                 252    136    112      8    39%
pyFV3/stencils/d_sw.py                                       323    132    140     18    51%
pyFV3/stencils/del2cubed.py                                   90     45     40      0    36%
pyFV3/stencils/delnflux.py                                   252    129    155      8    37%
pyFV3/stencils/divergence_damping.py                         201     68     74      7    57%
pyFV3/stencils/dyn_core.py                                   277    122    114     11    46%
pyFV3/stencils/fillz.py                                       78     57     50      0    21%
pyFV3/stencils/fv_dynamics.py                                188    145     80      0    18%
pyFV3/stencils/fv_subgridz.py                                377    331     90      0    14%
pyFV3/stencils/fvtp2d.py                                      83     16     24      2    74%
pyFV3/stencils/fxadv.py                                      141     81     66      3    30%
pyFV3/stencils/map_single.py                                  59     21     16      0    59%
pyFV3/stencils/mapn_tracer.py                                 24      1      8      2    91%
pyFV3/stencils/moist_cv.py                                    52     33     20      0    43%
pyFV3/stencils/neg_adj3.py                                   188    169    106      0     8%
pyFV3/stencils/nh_p_grad.py                                   51     22     10      0    48%
pyFV3/stencils/pe_halo.py                                     11      8     10      0    14%
pyFV3/stencils/pk3_halo.py                                    24     10     10      0    41%
pyFV3/stencils/ppm.py                                         40     26     20      0    30%
pyFV3/stencils/ray_fast.py                                    94     67     84      0    21%
pyFV3/stencils/remap_profile.py                              318    276    220      2    11%
pyFV3/stencils/remapping.py                                  173     77     88      8    41%
pyFV3/stencils/riem_solver3.py                                74     45     30      1    31%
pyFV3/stencils/riem_solver_c.py                               57     31     28      0    33%
pyFV3/stencils/saturation_adjustment.py                      525    400    200      1    30%
pyFV3/stencils/sim1_solver.py                                 77     64     58      0    11%
pyFV3/stencils/temperature_adjust.py                          18     13     10      0    18%
pyFV3/stencils/tracer_2d_1l.py                                94     71     36      0    21%
pyFV3/stencils/updatedzc.py                                   72     36     28      1    45%
pyFV3/stencils/updatedzd.py                                   84     31     24      1    54%
pyFV3/stencils/xppm.py                                       169    122     78      3    32%
pyFV3/stencils/xtp_u.py                                       35     27     16      0    24%
pyFV3/stencils/yppm.py                                       170    122     78      3    33%
pyFV3/stencils/ytp_v.py                                       35     27     16      0    24%
pyFV3/testing/map_single.py                                   15     15      2      0     0%
pyFV3/testing/translate_dyncore.py                            55      3     16      1    94%
pyFV3/testing/translate_fvdynamics.py                         81     52     24      0    30%
pyFV3/testing/validation.py                                   89     62     20      0    27%
pyFV3/utils/functional_validation.py                          31     25      4      0    17%
pyFV3/version.py                                               2      0      0      0   100%
pyFV3/wrappers/geos_wrapper.py                               275    275     32      0     0%
--------------------------------------------------------------------------------------------
TOTAL                                                       6493   4053   2563     98    33%

Describe the solution you'd like
Additional tests to increase coverage

Those results are surprising, considering that they were generated with Acoustics which touches upon most of the code. The next steps are:

Run FVDynamics which is exactly all the dycore
Dive in some of the more central test and understand why the discrepancy (could be configuration, lack of data, etc.)

Ultimately, we should tie a coverage to a dataset so we now how to reach global coverage.

Thinking more about this, it actually makes sense.
The way the code works we never "execute" the stencil code declared under gt4py frontend. Instead we code generate then we jump over to the generate code in the .gt_cache_X folders. This means the coverage will always see those code as "uncovered".
Partial fix is to include in the coverage the cache directory. Partial solution, because we still want the coverage to look at what code is being touched overall in-between stencils. Also __INLINED and other code culling system within the stack will chop away code and will remove use from being able to have coverage look at them. Maybe this can only be fixed by custom implementation of coverage tool from with gt4py and/or orchestration.
This issue will be worse under orchestration.


feat: [CDE]: added instructions for ssh

Thanks for contributing to the Harness Developer Hub! Our code owners will review your submission.
Description

Please describe your changes: __________________________________
Jira/GitHub Issue numbers (if any): ______________________________
Preview links/images (Internal contributors only): __________________

PR lifecycle
We aim to merge PRs within one week or less, but delays happen sometimes.
If your PR is open longer than two weeks without any human activity, please tag a code owner in a comment.
PRs must meet these requirements to be merged:

[ ] Successful preview build.
[ ] Code owner review.
[ ] No merge conflicts.
[ ] Release notes/new features docs: Feature/version released to at least one prod environment.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://66f52c8e161dc4b211a5fc36--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://66ffa83d1c356fdf3363ad30--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://67060898da9221ca2166b54d--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://670f7187d8f8e603464c48ce--harness-developer.netlify.app


Alt-tabbing out of a fullscreen client asks for any size in the configure event

The compositor keeps the client in fullscreen, but stops asking for the exact output size, which can lead to the client giving a non-optimal buffer.
Here is the problematic event:
zxdg_toplevel_v6@15.configure(0, 0, array(0x02, 0x00, 0x00, 0x00))

Looks like https://github.com/swaywm/wlroots/pull/680 also fixes this one.

Is it fixed? Can we close this?


fix pydantic 2.6. incompatibility

This fixes https://github.com/dapper91/pydantic-xml/issues/167.

@floscha I went through pydantic-core 2.15.0 changes. Unfortunately, they not just removed TuplePositionalSchema schema but removed tuple-variable and tuple-positional types at all.

Closed in favor to https://github.com/dapper91/pydantic-xml/pull/169


Overwhelming issues on Mac M1 with newest TF 2.5

Several issues after installing the TensorFlow metal plugin for Mac M1 according to the provided documentation:

Model crashes when using Adam optimizer (logs will follow)
Even on SGD optimizer, though it does not crash, the loss is as high as possible and does not change
More issues, but since these were so fundamentally flawed, I think I'll just leave it at that

Here's a basic script, taken from a short demo notebook:
import tensorflow as tf

mnist = tf.keras.datasets.mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10)
])

predictions = model(x_train[:1]).numpy()
tf.nn.softmax(predictions).numpy()

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

loss_fn(y_train[:1], predictions).numpy()

model.compile(optimizer = 'sgd', loss = loss_fn)
model.fit(x_train, y_train, epochs=100)

Here's what one would expect the logs to look like (run on a Colab runtime):
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
11493376/11490434 [==============================] - 0s 0us/step
Epoch 1/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.7249
Epoch 2/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.3882
Epoch 3/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.3217
Epoch 4/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.2853
Epoch 5/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.2566

And here's what they look like when run on the Mac M1:
Init Plugin
Init Graph Optimizer
Init Kernel
Metal device set to: Apple M1
2021-06-10 08:05:36.921503: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2021-06-10 08:05:36.921605: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
2021-06-10 08:05:37.062193: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
2021-06-10 08:05:37.062396: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
Epoch 1/5
2021-06-10 08:05:37.140640: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
1875/1875 [==============================] - 5s 3ms/step - loss: 60000.0000
Epoch 2/5
1875/1875 [==============================] - 5s 3ms/step - loss: 60000.0000
Epoch 3/5
1875/1875 [==============================] - 5s 3ms/step - loss: 60000.0000
Epoch 4/5
1875/1875 [==============================] - 5s 3ms/step - loss: 60000.0000
Epoch 5/5
1875/1875 [==============================] - 5s 3ms/step - loss: 60000.0000

Here's what happens when you try to supplement that with the 'adam' optimizer:
Init Plugin
Init Graph Optimizer
Init Kernel
Metal device set to: Apple M1
2021-06-10 08:07:02.300603: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2021-06-10 08:07:02.300696: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
2021-06-10 08:07:02.375056: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
2021-06-10 08:07:02.375226: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
Epoch 1/5
2021-06-10 08:07:02.458202: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-06-10 08:07:02.487 python3[42618:4085236] -[MPSNDArrayIdentity encodeToCommandEncoder:commandBuffer:sourceArrays:resultState:destinationArray:kernelDAGObject:]: unrecognized selector sent to instance 0x1695be290
2021-06-10 08:07:02.490 python3[42618:4085236] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[MPSNDArrayIdentity encodeToCommandEncoder:commandBuffer:sourceArrays:resultState:destinationArray:kernelDAGObject:]: unrecognized selector sent to instance 0x1695be290'
*** First throw call stack:
(
	0   CoreFoundation                      0x0000000199a4a320 __exceptionPreprocess + 240
	1   libobjc.A.dylib                     0x0000000199778c04 objc_exception_throw + 60
	2   CoreFoundation                      0x0000000199ad9020 -[NSObject(NSObject) __retain_OA] + 0
	3   CoreFoundation                      0x00000001999ac184 ___forwarding___ + 1444
	4   CoreFoundation                      0x00000001999abb30 _CF_forwarding_prep_0 + 96
	5   libmetal_plugin.dylib               0x0000000134a55db4 ___ZN12metal_plugin14MPSApplyAdamOpIfE7ComputeEPNS_15OpKernelContextE_block_invoke.115 + 92
	6   libdispatch.dylib                   0x0000000199725420 _dispatch_client_callout + 20
	7   libdispatch.dylib                   0x0000000199733a98 _dispatch_lane_barrier_sync_invoke_and_complete + 60
	8   libmetal_plugin.dylib               0x0000000134a533c0 _ZN12metal_plugin14MPSApplyAdamOpIfE7ComputeEPNS_15OpKernelContextE + 4884
	9   libmetal_plugin.dylib               0x0000000134a51e78 _ZN12metal_pluginL15ComputeOpKernelINS_14MPSApplyAdamOpIfEEEEvPvP18TF_OpKernelContext + 44
	10  _pywrap_tensorflow_internal.so      0x0000000148076460 _ZN10tensorflow15PluggableDevice7ComputeEPNS_8OpKernelEPNS_15OpKernelContextE + 148
	11  libtensorflow_framework.2.dylib     0x0000000115d78664 _ZN10tensorflow12_GLOBAL__N_113ExecutorStateINS_15PropagatorStateEE7ProcessENS2_10TaggedNodeEx + 3080
	12  libtensorflow_framework.2.dylib     0x0000000115d79dbc _ZNSt3__110__function6__funcIZN10tensorflow12_GLOBAL__N_113ExecutorStateINS2_15PropagatorStateEE7RunTaskIZNS6_13ScheduleReadyEPN4absl14lts_2020_09_2313InlinedVectorINS5_10TaggedNodeELm8ENS_9allocatorISB_EEEEPNS5_20TaggedNodeReadyQueueEEUlvE0_EEvOT_EUlvE_NSC_ISL_EEFvvEEclEv + 56
	13  _pywrap_tensorflow_internal.so      0x00000001487aa230 _ZN5Eigen15ThreadPoolTemplIN10tensorflow6thread16EigenEnvironmentEE10WorkerLoopEi + 1508
	14  _pywrap_tensorflow_internal.so      0x00000001487a9b28 _ZZN10tensorflow6thread16EigenEnvironment12CreateThreadENSt3__18functionIFvvEEEENKUlvE_clEv + 80
	15  libtensorflow_framework.2.dylib     0x0000000116314b08 _ZN10tensorflow12_GLOBAL__N_17PThread8ThreadFnEPv + 120
	16  libsystem_pthread.dylib             0x00000001998d206c _pthread_start + 320
	17  libsystem_pthread.dylib             0x00000001998ccda0 thread_start + 8
)
libc++abi.dylib: terminating with uncaught exception of type NSException
[1]    42618 abort      python3 tf-demo.py

Run on both Python 3.8 and Python 3.9, the same errors occur.

Same error on my end. M1 Mac mini, 16 GB RAM. I can't get anything to work since updating to tf2.5 from the Apple fork that they just archived.

FYR. on my MBP M1, I created a new python 3.8 venv,  then pip install tensorflow-macos and pip install tensorflow-metal. So far didn't meet any problem. Your little test python script worked with either sgd or adam.

FYR. on my MBP M1, I created a new python 3.8 venv,  then pip install tensorflow-macos and pip install tensorflow-metal. So far didn't meet any problem. Your little test python script worked with either sgd or adam.

Did you do the two steps before this in the instructions here https://developer.apple.com/metal/tensorflow-plugin/? Installing latest miniforge3 and installing the apple tensorflow-deps?

Nope, I am an old-school unix user. I use macports to build python and some dependency, e.g., BLAS for numpy, and use pip to build pip wheels needed by tensorflow-macos.

FYR. on my MBP M1, I created a new python 3.8 venv,  then pip install tensorflow-macos and pip install tensorflow-metal. So far didn't meet any problem. Your little test python script worked with either sgd or adam.

Did you do the two steps before this in the instructions here https://developer.apple.com/metal/tensorflow-plugin/? Installing latest miniforge3 and installing the apple tensorflow-deps?

Yes, I did this.

FYR. on my MBP M1, I created a new python 3.8 venv,  then pip install tensorflow-macos and pip install tensorflow-metal. So far didn't meet any problem. Your little test python script worked with either sgd or adam.

Did you do the two steps before this in the instructions here https://developer.apple.com/metal/tensorflow-plugin/? Installing latest miniforge3 and installing the apple tensorflow-deps?

Yes, I did this and can exactly replicate the issue.

@aseembehl, I did just that and did not get the same results. Here is exactly what I run:
conda create -n tf python=3.8
conda activate tf
conda install -c apple tensorflow-deps
python -m pip install tensorflow-macos
python -m pip install tensorflow-metal

Then, I run the demo script and get exactly the original output I uploaded. Not sure why this differs between you and I.

Is it possible that I'd have to reinstall miniforge3? When I try to do this (using the -u (upgrade) flag), I get this error towards the end:
Encountered problems while solving:
  - nothing provides requested ca-certificates 2020.12.5 h4653dfc_0
  - nothing provides requested tzdata 2021a he74cb21_0
  - nothing provides requested idna 2.10 pyh9f0ad1d_0
  - nothing provides requested six 1.16.0 pyh6c4a22f_0
  - nothing provides requested wheel 0.36.2 pyhd3deb0d_0
  - nothing provides requested certifi 2020.12.5 py39h2804cbe_1
  - nothing provides requested pyopenssl 20.0.1 pyhd8ed1ab_0
  - nothing provides requested urllib3 1.26.4 pyhd8ed1ab_0
  - nothing provides requested requests 2.25.1 pyhd3deb0d_0

ERROR   Could not solve for environment specs

OH, did not see that MacOS Monterey is the requirement. I'm still on Big Sur but will update to the developer beta soon. That is almost certainly the problem here.

@aseembehl, I did just that and did not get the same results. Here is exactly what I run:
conda create -n tf python=3.8
conda activate tf
conda install -c apple tensorflow-deps
python -m pip install tensorflow-macos
python -m pip install tensorflow-metal

Then, I run the demo script and get exactly the original output I uploaded. Not sure why this differs between you and I.

Maybe you misread my comment. TF with macOS doesn't work for me as well. I ran into the same problems as you did, NSexception with Adam and loss blowups with SGD.

OH, did not see that MacOS Monterey is the requirement. I'm still on Big Sur but will update to the developer beta soon. That is almost certainly the problem here.

Where do you see the Monterey requirement? Are you referring to the mention on Metal homepage?

@aseembehl, it says it here

OS Requirements

macOS 12.0+

Just updated to MacOS Monterey and this fixed the issue.

Updating to 12.0 beta worked for me as well! Perhaps that should be displayed in bold on the requirements page or be stressed that it is a beta OS. I mistakenly assumed it was the current stable release Big Sur (11.x). Thanks!!!

I am having same issue , apple M1, python3.8 . I have tried the fix and similar things but nothing has worked
here is the error stack  (I don't understand anything here )
Epoch 1/100
2021-11-10 15:09:53.313171: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-11-10 15:09:53.336 python[77750:5229041] -[MPSGraph adamUpdateWithLearningRateTensor:beta1Tensor:beta2Tensor:epsilonTensor:beta1PowerTensor:beta2PowerTensor:valuesTensor:momentumTensor:velocityTensor:maximumVelocityTensor:gradientTensor:name:]: unrecognized selector sent to instance 0x15e61f2c0
2021-11-10 15:09:53.354 python[77750:5229041] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[MPSGraph adamUpdateWithLearningRateTensor:beta1Tensor:beta2Tensor:epsilonTensor:beta1PowerTensor:beta2PowerTensor:valuesTensor:momentumTensor:velocityTensor:maximumVelocityTensor:gradientTensor:name:]: unrecognized selector sent to instance 0x15e61f2c0'
*** First throw call stack:
(
0   CoreFoundation                      0x0000000187e53838 __exceptionPreprocess + 240
1   libobjc.A.dylib                     0x0000000187b7d0a8 objc_exception_throw + 60
2   CoreFoundation                      0x0000000187ee4694 -[NSObject(NSObject) __retain_OA] + 0
3   CoreFoundation                      0x0000000187db4cd4 forwarding + 1444
4   CoreFoundation                      0x0000000187db4670 _CF_forwarding_prep_0 + 96
5   libmetal_plugin.dylib               0x000000013ecae290 _ZN12metal_plugin14MPSApplyAdamOpIfEC2EPNS_20OpKernelConstructionE + 656
6   libmetal_plugin.dylib               0x000000013ecadebc _ZN12metal_pluginL14CreateOpKernelINS_14MPSApplyAdamOpIfEEEEPvP23TF_OpKernelConstruction + 52
7   libtensorflow_framework.2.dylib     0x0000000133d985d4 _ZN10tensorflow12_GLOBAL__N_120KernelBuilderFactory6CreateEPNS_20OpKernelConstructionE + 88
8   libtensorflow_framework.2.dylib     0x0000000133e1a158 _ZN10tensorflow14CreateOpKernelENS_10DeviceTypeEPNS_10DeviceBaseEPNS_9AllocatorEPNS_22FunctionLibraryRuntimeEPNS_11ResourceMgrERKNSt3__110shared_ptrIKNS_14NodePropertiesEEEiPPNS_8OpKernelE + 784
9   libtensorflow_framework.2.dylib     0x0000000133ff52b8 _ZN10tensorflow21CreateNonCachedKernelEPNS_6DeviceEPNS_22FunctionLibraryRuntimeERKNSt3__110shared_ptrIKNS_14NodePropertiesEEEiPPNS_8OpKernelE + 272
10  libtensorflow_framework.2.dylib     0x0000000133f9fc20 _ZN10tensorflow26FunctionLibraryRuntimeImpl12CreateKernelERKNSt3__110shared_ptrIKNS_14NodePropertiesEEEPNS_22FunctionLibraryRuntimeEPPNS_8OpKernelE + 600
11  libtensorflow_framework.2.dylib     0x0000000133e1dd80 _ZN10tensorflow9OpSegment12FindOrCreateERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_PPNS_8OpKernelENS1_8functionIFNS_6StatusESC_EEE + 272
12  _pywrap_tensorflow_internal.so      0x000000011efc1258 _ZNSt3__110__function6__funcIZN10tensorflow13DirectSession15CreateExecutorsERKNS2_15CallableOptionsEPNS_10unique_ptrINS3_16ExecutorsAndKeysENS_14default_deleteIS8_EEEEPNS7_INS3_12FunctionInfoENS9_ISD_EEEEPNS3_12RunStateArgsEE4$11NS_9allocatorISJ_EEFNS2_6StatusERKNS_10shared_ptrIKNS2_14NodePropertiesEEEPPNS2_8OpKernelEEEclESS_OSV + 140
13  libtensorflow_framework.2.dylib     0x000000013400a430 _ZN10tensorflow22ImmutableExecutorState10InitializeERKNS_5GraphE + 1192
14  libtensorflow_framework.2.dylib     0x0000000133ff5064 _ZN10tensorflow16NewLocalExecutorERKNS_19LocalExecutorParamsERKNS_5GraphEPPNS_8ExecutorE + 304
15  libtensorflow_framework.2.dylib     0x0000000134002e6c _ZN10tensorflow12_GLOBAL__N_124DefaultExecutorRegistrar7Factory11NewExecutorERKNS_19LocalExecutorParamsERKNS_5GraphEPNSt3__110unique_ptrINS_8ExecutorENS9_14default_deleteISB_EEEE + 48
16  libtensorflow_framework.2.dylib     0x00000001340037e8 _ZN10tensorflow11NewExecutorERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEERKNS_19LocalExecutorParamsERKNS_5GraphEPNS0_10unique_ptrINS_8ExecutorENS0_14default_deleteISG_EEEE + 92
17  _pywrap_tensorflow_internal.so      0x000000011efbb1e4 _ZN10tensorflow13DirectSession15CreateExecutorsERKNS_15CallableOptionsEPNSt3__110unique_ptrINS0_16ExecutorsAndKeysENS4_14default_deleteIS6_EEEEPNS5_INS0_12FunctionInfoENS7_ISB_EEEEPNS0_12RunStateArgsE + 3160
18  _pywrap_tensorflow_internal.so      0x000000011efbdd80 _ZN10tensorflow13DirectSession12MakeCallableERKNS_15CallableOptionsEPx + 276
19  _pywrap_tensorflow_internal.so      0x000000011bb744c4 _ZN10tensorflow10SessionRef12MakeCallableERKNS_15CallableOptionsEPx + 192
20  _pywrap_tensorflow_internal.so      0x000000011bb6e7e8 _ZN10tensorflow12_GLOBAL__N_118MakeCallableHelperEPNS_7SessionEPK9TF_BufferPxP9TF_Status + 116
21  _pywrap_tf_session.so               0x0000000135755eec _ZZN8pybind1112cpp_function10initializeIZL32pybind11_init__pywrap_tf_sessionRNS_7module_EE3$0xJP10TF_SessionPK9TF_BufferEJNS_4nameENS_5scopeENS_7siblingEEEEvOT_PFT0_DpT1_EDpRKT2_ENUlRNS_6detail13function_callEE_8__invokeESQ + 212
22  _pywrap_tf_session.so               0x0000000135741934 ZN8pybind1112cpp_function10dispatcherEP7_objectS2_S2 + 3216
23  python                              0x00000001005cd8cc cfunction_call_varargs + 292
24  python                              0x00000001005ccf50 _PyObject_MakeTpCall + 640
25  python                              0x00000001006e23e0 call_function + 680
26  python                              0x00000001006deebc _PyEval_EvalFrameDefault + 29472
27  python                              0x00000001005cdd90 function_code_fastcall + 128
28  python                              0x00000001005ccc1c _PyObject_FastCallDict + 284
29  python                              0x00000001005cecc4 _PyObject_Call_Prepend + 156
30  python                              0x0000000100645044 slot_tp_init + 288
31  python                              0x0000000100652f28 type_call + 308
32  python                              0x00000001005ccf50 _PyObject_MakeTpCall + 640
33  python                              0x00000001006e23e0 call_function + 680
34  python                              0x00000001006deebc _PyEval_EvalFrameDefault + 29472
35  python                              0x00000001005cdd90 function_code_fastcall + 128
36  python                              0x00000001006e2348 call_function + 528
37  python                              0x00000001006deea0 _PyEval_EvalFrameDefault + 29444
38  python                              0x00000001005cdd90 function_code_fastcall + 128
39  python                              0x00000001006e2348 call_function + 528
40  python                              0x00000001006deea0 _PyEval_EvalFrameDefault + 29444
41  python                              0x00000001005cdd90 function_code_fastcall + 128
42  python                              0x00000001005ccc1c _PyObject_FastCallDict + 284
43  python                              0x00000001005cecc4 _PyObject_Call_Prepend + 156
44  python                              0x0000000100642774 slot_tp_call + 296
45  python                              0x00000001005ccf50 _PyObject_MakeTpCall + 640
46  python                              0x00000001006e23e0 call_function + 680
47  python                              0x00000001006def34 _PyEval_EvalFrameDefault + 29592
48  python                              0x00000001006d7584 _PyEval_EvalCodeWithName + 3340
49  python                              0x00000001005cdf10 _PyFunction_Vectorcall + 236
50  python                              0x00000001005ccb84 _PyObject_FastCallDict + 132
51  python                              0x0000000100793d0c partial_call + 444
52  python                              0x00000001005ccf50 _PyObject_MakeTpCall + 640
53  python                              0x00000001006e23e0 call_function + 680
54  python                              0x00000001006defac _PyEval_EvalFrameDefault + 29712
55  python                              0x00000001006d7584 _PyEval_EvalCodeWithName + 3340
56  python                              0x00000001005cdf10 _PyFunction_Vectorcall + 236
57  python                              0x00000001005d191c method_vectorcall + 156
58  python                              0x00000001006e2348 call_function + 528
59  python                              0x00000001006defac _PyEval_EvalFrameDefault + 29712
60  python                              0x00000001006d7584 _PyEval_EvalCodeWithName + 3340
61  python                              0x00000001005cdf10 _PyFunction_Vectorcall + 236
62  python                              0x00000001005d191c method_vectorcall + 156
63  python                              0x00000001006e2348 call_function + 528
64  python                              0x00000001006defac _PyEval_EvalFrameDefault + 29712
65  python                              0x00000001005cdd90 function_code_fastcall + 128
66  python                              0x00000001006e2348 call_function + 528
67  python                              0x00000001006def34 _PyEval_EvalFrameDefault + 29592
68  python                              0x00000001006d7584 _PyEval_EvalCodeWithName + 3340
69  python                              0x00000001007349d8 PyRun_StringFlags + 292
70  python                              0x00000001006d20a0 builtin_eval + 748
71  python                              0x000000010061f970 cfunction_vectorcall_FASTCALL + 284
72  python                              0x00000001006e2348 call_function + 528
73  python                              0x00000001006def34 _PyEval_EvalFrameDefault + 29592
74  python                              0x00000001005cdd90 function_code_fastcall + 128
75  python                              0x00000001006e2348 call_function + 528
76  python                              0x00000001006def34 _PyEval_EvalFrameDefault + 29592
77  python                              0x00000001006d7584 _PyEval_EvalCodeWithName + 3340
78  python                              0x00000001005cdf10 _PyFunction_Vectorcall + 236
79  python                              0x00000001006e2348 call_function + 528
80  python                              0x00000001006def34 _PyEval_EvalFrameDefault + 29592
81  python                              0x00000001006d7584 _PyEval_EvalCodeWithName + 3340
82  python                              0x0000000100730e3c PyRun_SimpleFileExFlags + 1052
83  python                              0x0000000100758eb0 Py_RunMain + 2980
84  python                              0x000000010075a0a8 pymain_main + 1244
85  python                              0x00000001005a41e0 main + 56
86  libdyld.dylib                       0x0000000187cf5430 start + 4
)
libc++abi: terminating with uncaught exception of type NSException
[1]    77750 abort

I need help as I cannot get tensorflow installed in M1 using macOS Monterey 12.0.1.
I have managed to create conda virtual env and managed to install tensorflow-deps for apple.
However, I cannot install tensroflow-macos for some reason.
(tf) [~/Downloads/TensorFlow]
kelvin > python -m pip install tensorflow-macos
ERROR: Could not find a version that satisfies the requirement tensorflow-macos (from versions: none)
ERROR: No matching distribution found for tensorflow-macos
Does anyone how to fix it?
Thanks
Kel

To be safe its better to get MiniForge [https://github.com/conda-forge/miniforge]
Then you should create a virtual env
conda create -n mienv python=3.8
Then activate that env
source activate mien
Last step :
conda install -c apple tensorflow-deps
pip install tensorflow-macos
pip install tensorflow-metal
After that all will work fine.
regards
Aryan
On 18 Nov 2021, at 00:07<EMAIL_ADDRESS>wrote:
EXTERNAL EMAIL: This email originated from outside of the University of Limerick. Do not click on links or open attachments unless you recognize the sender's email address and know the content is safe.
I need help as I cannot get tensorflow installed in M1 using macOS Monterey 12.0.1.
I have managed to create conda virtual env and managed to install tensorflow-deps for apple.
However, I cannot install tensroflow-macos for some reason.
(tf) [~/Downloads/TensorFlow]
kelvin > python -m pip install tensorflow-macos
ERROR: Could not find a version that satisfies the requirement tensorflow-macos (from versions: none)
ERROR: No matching distribution found for tensorflow-macos
Does anyone how to fix it?
Thanks
Kel
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/tensorflow/tensorflow/issues/50196#issuecomment-972349964, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AVTLUNXYYYIJ6EZUWNB7253UMQ7STANCNFSM46OMMXVA.

To be safe its better to get MiniForge [https://github.com/conda-forge/miniforge]
Then you should create a virtual env
conda create -n mienv python=3.8
Then activate that env
source activate mien
Last step :
conda install -c apple tensorflow-deps
pip install tensorflow-macos
pip install tensorflow-metal

try it many times, find the followings version  can work for 'adam optimizer'.
tensorflow-estimator          2.5.0
tensorflow-macos              2.5.0
tensorflow-metal              0.1.1

Mac OS 11，M1

Hi @ryanrudes @sxzhou1937 !
I can not replicate this exact behaviour . But I am getting below error with 2.9 and 2.11 api (Tested with metal plugin instruction)
Traceback (most recent call last):
  File "/Users/mohantym/git_50196.py", line 1, in <module>
    import tensorflow as tf
  File "/Users/mohantym/miniforge3/envs/tf-gpu/lib/python3.10/site-packages/tensorflow/__init__.py", line 439, in <module>
    _ll.load_library(_plugin_dir)
  File "/Users/mohantym/miniforge3/envs/tf-gpu/lib/python3.10/site-packages/tensorflow/python/framework/load_library.py", line 151, in load_library
    py_tf.TF_LoadLibrary(lib)
tensorflow.python.framework.errors_impl.NotFoundError: dlopen(/Users/mohantym/miniforge3/envs/tf-gpu/lib/python3.10/site-packages/tensorflow-plugins/libmetal_plugin.dylib, 0x0006): symbol not found in flat namespace '__ZN3tsl8internal10LogMessage16VmoduleActivatedEPKci'

Re-opening as requested.
Thank you!

I don't think this issue is specific to the optimizer. Please post the answer if anyone can fix this.

Hi @ryanrudes ,
I have tested the code on Mac M1 with tensorflow-macos==2.11 and its working fine. Please note older optimizers moved to legacy namespace hence we need to use tf.keras.optimizers.legacy.Adam() in place of 'adam' and it works as intended.
Please refer the attached logs below.
(tf-metal) suryanarayanay-macbookpro:~ suryanarayanay$ python 50196.py
2.11.0
Metal device set to: Apple M1 Pro

systemMemory: 16.00 GB
maxCacheSize: 5.33 GB

2023-03-30 13:41:43.720528: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:306] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2023-03-30 13:41:43.720935: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:272] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
2023-03-30 13:41:44.060864: W tensorflow/tsl/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
Epoch 1/10
2023-03-30 13:41:44.213540: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.
1875/1875 [==============================] - 7s 4ms/step - loss: 0.2868
Epoch 2/10
1875/1875 [==============================] - 7s 4ms/step - loss: 0.1357
Epoch 3/10
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0997
Epoch 4/10
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0815
Epoch 5/10
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0700
Epoch 6/10
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0583
Epoch 7/10
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0512
Epoch 8/10
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0479
Epoch 9/10
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0415
Epoch 10/10
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0384
(tf-metal) suryanarayanay-macbookpro:~ suryanarayanay$ cat 50196.py
import tensorflow as tf
print(tf.__version__)
mnist = tf.keras.datasets.mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10)
])

predictions = model(x_train[:1]).numpy()
tf.nn.softmax(predictions).numpy()

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

loss_fn(y_train[:1], predictions).numpy()

model.compile(optimizer = tf.keras.optimizers.legacy.Adam(), loss = loss_fn)
model.fit(x_train, y_train, epochs=10)
(tf-metal) suryanarayanay-macbookpro:~ suryanarayanay$

If still anyone have issues please let us know.Thanks!


Suggestions: Add a "Remove Filter" button that removes all applied filters.

#1 - Sorry for the English, I'm not very familiar with the language.
#2 - During my use, I needed to implement a button that removes all filters applied to the table. So I thought it would be a good idea to suggest this implementation.
#3 - I would like to help in some way, but I'm not that good with VueJs. But if you want, I can try to help in some way.

I'm new to this package but also like the ides - was one of the first things I saw which is missing but required with a lot of filter set.

Fixed in v2


[Question] For what purpose, should I use the project?

I configured kcg. It's very convenient to generate a config file and the Cli interaction is cool!
But I am curious when should I use kcg.
I normally deploy a Kubernetes cluster in a Kubernetes cloud provider, like Alibaba Cloud, or by tool like kind or rke. In either way, a Kube-config file will be automatically generated.
If I'd like to use kcg, I need to check the cluster in Alibaba Cloud Console, or check Kube config itself, so in which situation is kcg targeted?
Thanks.

Thanks for the nice words.
Long answer : https://sudheer.dev/posts/2020/08/generate-kubernetes-config-files/
Short answer: if K8 cluster is hosting isolated tenants separated by namespaces and if you need to provide these tenants with credentials to manage their namespace.
also it is another way not to rely on your manged K8 provider tools like alibaba, awscli , etc..


Ghost Teleporting

Closes #1904
Adds the ability for ghosts to teleport to people and places.
If people have the same name the server should still be able to teleport you to them without error.
It will also not add the local player (You) to the list... because that's just useless

Maybe include their job in the text too, so you can tell duplicates easier

Seems you commited a submodule update with your PR.
Please revert it, and make sure not to commit the submodule again in the future!

Seems you commited a submodule update with your PR.
Please revert it, and make sure not to commit the submodule again in the future!
Ahh heck where do I fix that?

Seems it still conflicts :cold_sweat:

Hey! Only 4 files are edited now, that's  good! Now all im getting are nullable errors, iv tried adding #nullable enable but that generates a good bit of errors.

am I winning?


Add a batch_norm_inference op to VMLA

The test is located at: https://github.com/google/iree/blob/master/test/e2e/xla/batch_norm_inference.mlir

Note that there is a conversion pass which expands to the fundamental ops. If you just add that, it should work (versus implementing a dedicated op).


Add distiller id to scan page

@ercius Is this what you mean? We already have your "scan id" on the scan page.

Yes. It would be good to see both IDs on that page
On Mon, Mar 7, 2022 at 11:06 AM Chris Harris @.***>
wrote:

@ercius https://github.com/ercius Is this what you mean? We already
have your "scan id" on the scan page.
—
Reply to this email directly, view it on GitHub
https://github.com/OpenChemistry/distiller/issues/277#issuecomment-1061030524,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADXORFD7MC3R7MFMNYGRB2TU6ZHSZANCNFSM5QEFVUBA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>

@ercius I think it would be good come up with consistent names for these two ids, what about:
scan id generated by detector => Detector Scan ID
scan id generated by distiller => Distiller Scan ID
I am open to suggestions?

@cjh1 I think that makes sense. Lets go with those!

Fixed in #313, deployed.


llama3.2-1B-CC

I would like to ask which vision model corresponds to Llama3.2-1B-CC?

We will release tomorrow

We will release tomorrow

thank you

The corresponding vision is right here.

i got it, thanks


Cannot run on iOS

Hi,
maybe there is something I'm missing but it seems that I can't change the provisionning profile used by Patrol.
I'm getting this error which I can't fix.
Am I missing something?

Device is iPhone X running on iOS 16

Hi,
When running on physical iOS you need to manually sign Patrol's AutomatorServer. Open the Xcode project at ~/.cache/patrol/ios-0.10.4 and set up code signing.

Worked like a charm.
Thank you.
Could be amzing to add on doc under iOS setup. (Even if that's temporary, this is a bit frustrating 😅)


[3.1] Use fully doc block path for Excel facade

Requirements
Please take note of our contributing guidelines: https://docs.laravel-excel.com/3.1/getting-started/contributing.html
Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion.
Mark the following tasks as done:

[x] Checked the codebase to ensure that your feature doesn't already exist.
[x] Checked the pull requests to ensure that another person hasn't already submitted the feature or fix.
[ ] Adjusted the Documentation.
[ ] Updated CHANGELOG.md
[ ] Added tests to ensure against regression.

Description of the Change
Added fully path for Doc Block comment.
Why Should This Be Added?
Because ide-helper generate types by comments and right now the comments incorrect.

Benefits
Correct highlight in the IDE.

I'm not a big fan of having the FQN's in docblocks. Why doesn't ide-helper get the FQN from the namespace import? Sounds like a problem of that library, not this one.

Me too. But it will be better for facades as all packages do.

The classes you edited are not facades

Ohh, yes. Can I change facade comments ?

I've checked and I made correct changes

If I use this import then highlight is correct: use Maatwebsite\Excel\Facades\Excel;. But Ide helper allow use facades as use Excel; and in that case the highlight incorrect. Because plugin read real comment from Excel class which mapped to Excel facade.

I would accept changes to the Excel facades doc blocks, but not to the interfaces

it doesn't make any sense because if import use Maatwebsite\Excel\Facades\Excel; then highlight is correct.
But alias Excel incorrect because IDE Helper plugin read comments from real class for each methods.

I would personally always import the FQN of the facade though. I'll think about this, still not a big fan of FQN's in docblocks.

Ok, these changes will affect only if user going to use alias as me)


Approximation-based Designs for Binary Data

Currently Ambrosia supports only simulation-based power calculations for experiments with binary outcomes (see design_binary_size ultimately referencing  __helper_calc_empirical_power).
One could rely on approximations to arrive at an analytical expression for power. First, consider variance-stabilising transformation of the proportions in the control ( $p_1$ ) and the treated group ( $p_2$ ) and express power of a two-sided two-sample test for proportions as:
(1-\beta) = \Phi \left( \Phi^{-1}\left( \frac{\alpha}{2} \right) - 2 \left( \arcsin \sqrt{p_1} - \arcsin \sqrt{p_2}  \right) \sqrt{\frac{n}{2}}\right) + \left(1 - \Phi \left( \Phi^{-1}\left( \frac{\alpha}{2} \right) - 2 \left( \arcsin \sqrt{p_1} - \arcsin \sqrt{p_2}  \right) \sqrt{\frac{n}{2}}\right)\right)

and search for either of $<\beta,p_1,p_2,\alpha,n>$, holding the other four fixed, such that the function reaches zero.
Second, when $n$ is large enough one could rely on Normal approximations of the binomial distribution and express power of the two-sided test as
(1-\beta) = \Phi \left( \frac{ \sqrt{n} \left| p_1 - p_2 \right| + \Phi^{-1} \left( \frac{\alpha}{2} \right) \sqrt{ \left( p_1 + p_2 \right) \left( 1 - \left( p_1 + p_2 \right) \right)} } { \sqrt{p_1 \left( 1 - p_1 \right) + p_2 \left( 1 - p_2 \right)} }\right)

and perform the same search.
Let us analytically solve a problem in your 4_usage_example_binary_design.ipynb: find $n$ such that we are able to detect a 5% increase in experimental group proportion vis-à-vis the control group proportion of 5% with type-I error of 5% and type-II error of 20%. In R parlance the solution is:

effect <- 1.05

p1 <- 0.05
p2 <- 0.05*effect
sig.level <- 0.05
power <- 0.8
tol <- .Machine$double.eps^0.25

# Variance-stabilising transformation
h <- 2 * asin(sqrt(p1)) - 2 * asin(sqrt(p2))

p.asin <- quote({pnorm(qnorm(sig.level/2, lower = F) - h * sqrt(n/2), lower = F) + pnorm(qnorm(sig.level/2, lower = T) - h * sqrt(n/2), lower = T)})

# Normal approximation of the binomial distribution
p.normal <- quote(pnorm((sqrt(n) * abs(p1 - p2) - (qnorm(sig.level/2, lower.tail = F) * sqrt((p1 + p2) * (1 - (p1 + p2)/2))))/sqrt(p1 * (1 - p1) + p2 * (1 - p2))))

# Solve for n
n.asin <- stats::uniroot(function(n) eval(p.asin) - power, c(2 + 1e-10, 1e+09))$root

n.normal <- stats::uniroot(function(n) eval(p.normal) - power, c(2 + 1e-10, 1e+09))$root

# What is n to achieve the MDE of interest under two approximations?
n.asin # 122106.8
n.normal # 122123.5

This is a self-contained solution that could be easily translated into Python. It is taken from the existing routines:
# Variance stabilising transformation-based
pwr::pwr.2p.test(h = ES.h(0.05, 0.05*effect), power = 0.8, sig.level = 0.05)
#     Difference of proportion power calculation for binomial distribution (arcsine transformation)
#
#              h = 0.01133831
#              n = 122106.8
#      sig.level = 0.05
#          power = 0.8
#    alternative = two.sided
#
#NOTE: same sample sizes

# Normal approximation-based
stats::power.prop.test(n = NULL, p1 = 0.05, p2 = 0.05*effect, power = 0.8, sig.level = 0.05)
#     Two-sample comparison of proportions power calculation
#
#              n = 122123.5
#             p1 = 0.05
#             p2 = 0.0525
#      sig.level = 0.05
#          power = 0.8
#    alternative = two.sided
#
#NOTE: n is number in *each* group

I think offering analytical methods in binary designs using the above approximations could be a valuable alternative to your simulation-based power calculations since the former are commonplace in statistics.

Yes, it's a good alternative for empiric evaluations. So we will try to insert this approach to the source code. Thanks for the issue.

Analytical methods for binary metrics added in #30
Now theoretical design supports asin and normal approximations.


Broken link in the docs

Hello,
the link to source code index on https://socketry.github.io/console/ points to http://localhost:37695/source/
Thank you!

This is now fixed, sorry for the late reply!


Validate user supplied TARGET

This has a tiny side effect, the error gets triggered even during make clean, but otherwise it works as intended.

About the Travis failures:
-The ARM builds have timed out, no idea why, maybe the Travis servers are too busy
-There is one legitimate failure. I do not really understand how the Travis builds work, but the build parameters seem to be conflicting. First there is

export BTYPE="BINARY=64 NO_AFFINITY=1 USE_OPENMP=0 NO_LAPACK=0 TARGET=core2"

After that,

$ COMMON_FLAGS="DYNAMIC_ARCH=1 TARGET=NEHALEM NUM_THREADS=32"

Is this supposed to to be a CORE2 or a NEHALEM build? Also, why set DYNAMIC_ARCH=1, if you want a specific target?

ARM tests are always timing out as the CI job runs on virtual hardware (qemu) - the idea is to use them for now to catch any glaring compile bugs, and set up automatic integration with a dedicated system later. (#1867).
The failing build is a good demonstration, setting TARGET in addition to DYNAMIC_ARCH can be useful when the build host is not autodetected, or to ensure that the common parts of the code (BLAS driver routines and the entire LAPACK) do not rely on "modern" cpu features present in the build system.


Source links

The source link on this page - https://ml5js.org/docs/simple-image-classification-example.html -isn't right. @cvalenzuela I'm happy to fix some of these things but not 100% sure best way as #37 is still waiting to be merged?
(Thanks @AlcaDesign for noting this!)

There's a lot of text that says "deeplearn.js" instead of "tenserflow.js," and missing periods at the end of sentences. Other than that, here's some more broken links, (minor) automatically redirected links, specific grammar mistakes, missing links, etc.
Documentation

On the "Getting Started" docs page:

(Minor) the "latest version" link is redirected from https://github.com/ITPNYU/ml5 to https://github.com/ml5js/ml5-library automatically.

(Minor) the "ml5.min.js" link is redirected from https://github.com/ITPNYU/ml5/tree/master/dist to https://github.com/ml5js/ml5-library/tree/master/dist automatically.

On the "Image Classifier" docs page:

the "SqueezeNet," "MobileNet," and "original SqueezNet model" links are broken.

On the "KNN Image Classifier" docs page:

the "ImageClassifier" link is broken.

(Minor) the "deeplearn.js KNN image classifier model" link is redirected from https://github.com/PAIR-code/deeplearnjs/tree/master/models/knn_image_classifier to https://github.com/tensorflow/tfjs-core/tree/master/models/knn_image_classifier automatically.

the "Source" link is broken.

On the "LSTM" docs page:

the "this set of pretrained models" link is broken.

a grammar error in that same sentence: "used" instead of "use."

the "Source" link is broken.

On the "Style Transfer" docs page:

the "this tutorial" link is self-referential, basically broken.

(Minor) the "this demo" link is redirected from https://github.com/PAIR-code/deeplearnjs/tree/0608feadbd897bca6ec7abf3340515fe5f2de1c2/demos/fast-style-transfer to https://github.com/tensorflow/tfjs-core/tree/0608feadbd897bca6ec7abf3340515fe5f2de1c2/demos/fast-style-transfer automatically.

On the "Word2Vec" docs page:

the "this tutorial" link is self-referential, basically broken.

(Minor) the "Source" link is redirected from https://github.com/ITPNYU/ml5/blob/master/src/Word2vec/index.js to https://github.com/ml5js/ml5-library/blob/master/src/Word2vec/index.js automatically.

Examples

All other examples except "Multiple Image Classification"

On the "Train a LSTM" example page:

the "this tutorial" link is self-referential, basically broken.

On the "LSTM Interactive" example page:

the "this tutorial" link is self-referential, basically broken.

On the "Style Transfer" example page:

the "this tutorial" link is self-referential, basically broken.

the "Udnie (Young American Girl, The Dance), 1913 - Francis Picabia" link is the same as the previous link, which is incorrect.

On the "Style Transfer with Webcam" example page:

the "this tutorial" link is self-referential, basically broken.

the "Udnie (Young American Girl, The Dance), 1913 - Francis Picabia" text is not linked.

Datasets

On the "Making Your Own Datasets" example page:

(Minor) the "OpenCV" link can be https but isn't.

Training Models

On the "Style Transfer" example page:

the "deeplearn.js" and "ml5.js" links is missing its href value, basically broken.

On the "Train a LSTM" training models page:

(Minor) the "deeplearn.js" link is redirected from https://github.com/PAIR-code/deeplearnjs to https://github.com/tensorflow/tfjs-core automatically.

(Minor) the "ml5.js" text is not linked.

the "More detailed instructions here" link is directed to the homepage instead of the "Python Setup" page.

All solved! See https://github.com/ml5js/ml5-website/pull/47


namespace-cli: init at 0.0.295

Description of changes
https://github.com/namespacelabs/foundation
https://namespace.so/
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 261267 run on x86_64-linux 1

1 package built:

namespace-cli

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/1173

Result of nixpkgs-review pr 261267 run on x86_64-linux 1

1 package built:

namespace-cli

[x] package path fits guidelines
[x] package name fits guidelines
[x] package version fits guidelines
[x] package build on x86_64-linux
[x] executables tested on x86_64-linux
[x] meta.description is set and fits guidelines
[x] meta.license fits upstream license
[x] meta.platforms is set (via buildGoModule)
[x] meta.maintainers is set
[x] build time only dependencies are declared in nativeBuildInputs
[x] source is fetched using the appropriate function
[x] the list of phases is not overridden
[x] when a phase (like installPhase) is overridden it starts with runHook preInstall and ends with runHook postInstall.
[x] patches have a comment describing either the upstream URL or a reason why the patch wasn't upstreamed
[x] patches that are remotely available are fetched rather than vendored


Contribute updated GitLab tap back to the Singer community

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/653
Originally created by @dmor on 2019-05-24 14:19:30

Our tap is much more feature-rich, and we need to contribute it back

View 1 previous comment from the original issue on GitLab


Don't know how to open the website ( Windows )

Like, i don't what's the mail and password, i can't login on the website
Query : select * from accounts where id is null and accounts.deleted_at is null limit 1

Run php artisan migrate --seed
Login with the following credentials:

Email<EMAIL_ADDRESS>Password: 12345678

Read Documentation for more details.

Still not working, when i want to login in the website just refresh the page and the url's going from localhost/public/login to localhost/public/login?, i got an error in console :
login:449        GET http://localhost/livewire/livewire.js?id=99a05389 net::ERR_ABORTED 404 (Not Found)


Toggle logging on/off

Is this possible? I find that my logs are saturated with messages about WifiSignalStrengthChanged.

Thanks for asking the question.
Currently, you can turn off wifi scanning. Please, take a look here: https://github.com/pwittchen/NetworkEvents#initialize-objects
If you compile the app in release mode, it should not produce any logs.
Toggling logs is good idea, so I can add it to the next version of the library.

I've turned off wifi scanning and that's solved it for the time being. Having the toggling function would be great, thanks!

If you don't need to perform any action when Wifi signal strength is changing, you should have Wifi scanning turned off. Nevertheless, I'm going to add possibility to turn off logs.

Now, we can disable logging by passing EmptyLogger implementation as follows:
new NetworkEvents(context, bus, new EmptyLogger());

I know, it's not the best solution, but in current version of the library implementation of the logger have to be passed during object creation. We can think of improving it in the future.
This functionality is not available on Maven Central yet. It will be available in the next release.


Use a single connection for data and metadata management

As management of data and metadata using deltalake was developed in parallel, is code and configuration for connecting to, reading from, and writing to object stores of different types duplicated across the modelardb_common/src/storage.rs and  modelardb_common/src/metadata modules. This seems unnecessary and a single type for connecting to and performing operations on the object store in ModelarDB should be created.

When combining these into a single type we should also reconsider whether we want to add unit tests for the DeltaLake struct directly or continue to rely on other components testing the functionality indirectly.


Estimating uncertainty on fitted parameters

First, let me say thanks for this great package!
I am using the Bayesian optimization gp_minimize to find the minimum of an hard-to-compute function.
At the end of the optimization I obtain "best-fit" parameters. I'd like to estimate the uncertainty on each of them. What is the recommended approach in scikit-optimize?
The function evaluations performed during the optimization are used to compute an Gaussian-process approximation of the loss function. I was thinking that, if there was a way to extract this analytical approximation, I could compute the second derivative with respect to each parameter as an indication of "uncertainty". Is this possible? Are there other better ways?
Thanks for any suggestion.

Hi, we do not have any functionality that evaluates uncertainty on specific parameters. Maybe what you suggest is the way to go.
What do you expect the uncertainty value for a particular value of parameter to tell you? The likelihood that a given point is closer than some epsilon to the global optmum? Or would it be something else? Maybe if you could provide a use case where you would use the uncertainty values we could give some suggestions.

Hi @iaroslav-ai,
I am using scikit-optimize to fit parameters of a single-molecule dynamics simulators to experimental results. The loss function is basically a metric of how much the simulation differs from the experiment.
Parameters are physical properties of the molecule. I would like to provide an estimation of the uncertainty of the fit. For example, for a given noise level with variance sigma^2, I could estimate how much to vary a parameter from the minimum to increase the loss function more than 1 sigma. Smaller variations of the parameter are not statistical significant, meaning that with the current information we cannot tell where the true value lies within the range.
Does this make sense?

Hi,
sorry for a delayed response!
Firstly, that is a cool use - case :D Yes, that makes sense to me.
Given the description of what you want, I think the way that you suggested is the right way. What you look for is the .models property of optimization function result, which contain the list of fitted gaussian processes for every iteration of the algorithm. You can take the -1st, which would be the most recent / accurate one. Then, you can try to vary different parameters, and see what the estimated objective would be. Note that the model is an estimator, similar to sklearn estimators. Alternatively, you can also try estimating the curvature of Gaussian process.
Beyond that I cannot suggest anything. Maybe if you find something let us know here! :)

Feel free to reopen if your issue still stands.


Ensure imported polyfills are minified

Seems some of the polyfills that Simpla depends on aren't being minified (eg: promise-polyfill/Promise.js). Ensure we're pulling minified versions, or minify it ourselves in a buildstep.

Should be fixed with #37

Also worth noting that that particular polyfill is being required by iron-ajax and potentially a few other Polymer components, which should be resolved once we move to the SDK rather than making AJAX calls from within Simpla components.

Effective dupe of #49 - will be using babel transforms for minor polyfills, and not bundling wc polyfill at all. Closing in favour of that issue.


ARROW-16752: [R] Rework Linux binary installation

See the jira for the main behavior changes here. Other changes of note:

There are more brief messages printed to the installation log, even in the default "quiet" mode, that indicate which branch of the logic in nixlibs.R you've gone through. They're factual and generally connected to the tests that are being run, but they are worded somewhat ambiguously or coded, so as not to run afoul of censors should they appear in the wrong context. This should help us in the triaging of installation failures, even in circumstances where we can't enable greater verbosity.
There is a start of a test suite for nixlibs.R, run separately from the package tests. It has been wired up to run in ci/scripts/r_test.sh.

@github-actions crossbow submit -g r

@github-actions crossbow submit test-ubuntu-18.04-r-sanitizer test-r-offline-minimal r-binary-packages

@github-actions crossbow submit test-ubuntu-18.04-r-sanitizer test-r-offline-minimal r-binary-packages

@github-actions crossbow submit test-ubuntu-18.04-r-sanitizer test-r-offline-minimal r-binary-packages

Indications that this is working as expected:

In the r-binary-packages workflow, I've edited the test-linux-binary job to test both the automatic binary downloading behavior for some images (LIBARROW_BINARY unset) and the opt-in by setting LIBARROW_BINARY=true on OSes that aren't in the allowlist.

For CentOS 7, we see the "Some features are not available with gcc 4" message, and then success using the centos-7 binary https://github.com/ursacomputing/crossbow/runs/7158493032?check_suite_focus=true#step:12:1170
rocker/r-ver:3.6.3 is Debian, so we set LIBARROW_BINARY=true and it detects libcurl and openssl 1, and uses ubuntu-18.04 https://github.com/ursacomputing/crossbow/runs/7158493078?check_suite_focus=true#step:12:490
rocker/r-ubuntu:22.04 finds openssl3 so uses the ubuntu-22.04 binary: https://github.com/ursacomputing/crossbow/runs/7158493225?check_suite_focus=true#step:12:489
etc.

In the as-cran workflow, all jobs say "Building libarrow from source", and none of them show any of the messages from the code that tries to find system requirements to determine a binary to download. (Even if they did try to download, the messages it prints are generic enough not to broadcast it, and we would be fine as long as the build succeeded.)
We can spot-check the other builds to see which binary they would have used if it existed--only the r-binary-packages workflow has the binary repo set up, but unless we set LIBARROW_BINARY=false, the other jobs will still try to find one.

Remaining to-dos:

I'd like to add a few more docker images to the test-linux-binary job in the packages workflow, just to confirm that the binaries are working across a broader spectrum of system configurations.
Installation vignette needs revision

@github-actions crossbow submit r-binary-packages

@github-actions crossbow submit r-binary-packages

@github-actions crossbow submit r-binary-packages

@github-actions crossbow submit r-binary-packages

@github-actions crossbow submit r-binary-packages

@github-actions crossbow submit r-binary-packages test-r-linux-as-cran

Here's the confirmation that we're detecting libc++ in the fedora-clang-devel build, if we tell it to look for a binary (which it won't by default because fedora): https://github.com/ursacomputing/crossbow/runs/7201387872?check_suite_focus=true#step:13:492

@github-actions crossbow submit r-binary-packages


Updated to launcher v0.5

Resolve integration issue with launcher v0.5

This change is

r? @ustulation
(maidsafe_highfive has picked a reviewer for you, use r? to override)


Segmentation fault reading from systemd

I'm seeing continuous segmentation faults on STDERR, seemingly when trying to initialize the journal:
/var/lib/gems/2.3.0/gems/systemd-journal-1.3.3/lib/systemd/journal.rb:230: [BUG] Segmentation fault at 0x00000000050b20
ruby 2.3.1p112 (2016-04-26) [x86_64-linux-gnu]

-- Control frame information -----------------------------------------------
c:0013 p:---- s:0056 e:000055 CFUNC  :sd_journal_open_directory
c:0012 p:0102 s:0050 e:000049 METHOD /var/lib/gems/2.3.0/gems/systemd-journal-1.3.3/lib/systemd/journal.rb:230
c:0011 p:0046 s:0042 e:000041 METHOD /var/lib/gems/2.3.0/gems/systemd-journal-1.3.3/lib/systemd/journal.rb:51 [FINISH]
c:0010 p:---- s:0034 e:000033 CFUNC  :new
c:0009 p:0027 s:0030 e:000029 METHOD /var/lib/gems/2.3.0/gems/fluent-plugin-systemd-1.0.1/lib/fluent/plugin/in_systemd.rb:68
c:0008 p:0009 s:0026 e:000025 METHOD /var/lib/gems/2.3.0/gems/fluent-plugin-systemd-1.0.1/lib/fluent/plugin/in_systemd.rb:107 [FINISH]
c:0007 p:---- s:0023 e:000022 IFUNC
c:0006 p:0014 s:0021 e:000020 METHOD /var/lib/gems/2.3.0/gems/fluentd-1.2.5/lib/fluent/plugin_helper/timer.rb:80 [FINISH]
c:0005 p:---- s:0017 e:000016 CFUNC  :run_once
c:0004 p:0042 s:0013 e:000012 METHOD /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88
c:0003 p:0033 s:0009 e:000008 BLOCK  /var/lib/gems/2.3.0/gems/fluentd-1.2.5/lib/fluent/plugin_helper/event_loop.rb:93
c:0002 p:0081 s:0006 e:000005 BLOCK  /var/lib/gems/2.3.0/gems/fluentd-1.2.5/lib/fluent/plugin_helper/thread.rb:78 [FINISH]
c:0001 p:---- s:0002 e:000001 (none) [FINISH]

-- Ruby level backtrace information ----------------------------------------
/var/lib/gems/2.3.0/gems/fluentd-1.2.5/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
/var/lib/gems/2.3.0/gems/fluentd-1.2.5/lib/fluent/plugin_helper/event_loop.rb:93:in `block in start'
/var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run'
/var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run_once'
/var/lib/gems/2.3.0/gems/fluentd-1.2.5/lib/fluent/plugin_helper/timer.rb:80:in `on_timer'
/var/lib/gems/2.3.0/gems/fluent-plugin-systemd-1.0.1/lib/fluent/plugin/in_systemd.rb:107:in `run'
/var/lib/gems/2.3.0/gems/fluent-plugin-systemd-1.0.1/lib/fluent/plugin/in_systemd.rb:68:in `init_journal'
/var/lib/gems/2.3.0/gems/fluent-plugin-systemd-1.0.1/lib/fluent/plugin/in_systemd.rb:68:in `new'
/var/lib/gems/2.3.0/gems/systemd-journal-1.3.3/lib/systemd/journal.rb:51:in `initialize'
/var/lib/gems/2.3.0/gems/systemd-journal-1.3.3/lib/systemd/journal.rb:230:in `open_journal'
/var/lib/gems/2.3.0/gems/systemd-journal-1.3.3/lib/systemd/journal.rb:230:in `sd_journal_open_directory'

While STDOUT is showing the following repeatedly (with pid changing):
2018-11-01 00:18:00 +0000 [info]: fluent/log.rb:322:info: Worker 0 finished unexpectedly with signal SIGSEGV
2018-11-01 00:18:00 +0000 [info]: fluent/log.rb:322:info: gem 'fluent-plugin-systemd' version '1.0.1'
2018-11-01 00:18:00 +0000 [info]: fluent/log.rb:322:info: gem 'fluentd' version '1.2.5'
2018-11-01 00:18:00 +0000 [info]: fluent/log.rb:322:info: adding source type="systemd"
2018-11-01 00:18:00 +0000 [info]: #0 fluent/log.rb:322:info: starting fluentd worker pid=1304 ppid=25 worker=0

Source config:
<source>
  @type systemd
  filters [{ "_SYSTEMD_UNIT": "docker.service" }]
  <storage>
      @type local
      persistent true
      path /var/log/docker.log.pos
  </storage>
  tag docker
</source>

Let me know if there are any other details I can provide or experiments I can run.

Are you running fluentd on the same host as the journal is being created on or in a docker container?
If you are in a container could you share the version of systemd on the host and in your container?

Closing since there isn't much we can do without knowing specific versions!


Test: Intellisense in Code Actions on Save

[x] anyOS @aiday-mar
[ ] anyOS @cwebster-99
[x] anyOS @andreamah

Complexity: 1
Related: https://github.com/microsoft/vscode/issues/161284
Create Issue

Allows contributed code actions (source actions) from extensions to be shown by default. Will list all available code actions on save via intellisense.
Testing

Make sure you have a few extensions installed (try Ruff, Black, Eslint, etc)
in settings.json, check editor.codeActionsOnSave
You should see a few more code actions on save via Intellisense.
Make sure these work for explicit, always, and never

Things to note:

currently does not work for notebook code actions (notebook.codeActionsOnSave), but is on our backlog.

Works well for me 👍


infiniteQueryOptions can't be used with both useInfiniteQuery and useSuspenseInfiniteQuery

Describe the bug
When upgrading from 5.59.9 to 5.59.14 the types from basic infiniteQueryOptions changed so they don't work with both  useInfiniteQuery and useSuspenseInfiniteQuery when you set
 "compilerOptions": {
    "strict": true,
    "exactOptionalPropertyTypes": true
  }

Your minimal, reproducible example
https://stackblitz.com/edit/vitejs-vite-xfkpkc?file=src%2FApp.tsx
Steps to reproduce

Go to App.tsx
Observe Type error on line 41 (infiniteQueryResult)
Downgrade to "@tanstack/react-query": "5.59.9"
Go back to App.tsx and observe that the Type error is gone

Expected behavior
Not having a Type error
How often does this bug happen?
Every time
Screenshots or Videos
No response
Platform

Typescript "5.6.2"

Tanstack Query adapter
react-query
TanStack Query version
5.59.14
TypeScript version
5.6.2
Additional context
No response

Do you think adding | undefined here would fix it?
https://github.com/TanStack/query/blob/a085c92a5324b049ac4fc43d58c075815a6380e4/packages/react-query/src/infiniteQueryOptions.ts#L50-L62

Do you think adding | undefined here would fix it?

(same team) It doesn't fix it because infiniteQueryOptions still matches UnusedSkipTokenInfiniteOptions, which isn't supported by any of the useInifiniteQuery overloads. Previously, our infiniteQueryOptions used the UndefinedInitialDataInfiniteOptions type.

I have reviewed the example project you provided, but I couldn’t reproduce the bug you mentioned. Could you please provide more detailed information about the issue?

I have reviewed the example project you provided, but I couldn’t reproduce the bug you mentioned. Could you please provide more detailed information about the issue?

Try adding "exactOptionalPropertyTypes": true to the tsconfig.app.json file. It seems like it it wasn't always enough to just have it in the tsconfig.json.
You should see an error like this:


Getting two console warnings

Everything is working as expected, but I'm wondering why I'm getting these two warnings in chrome dev tools' console:
Warning: Unknown prop `portalId` on <div> tag. Remove this prop from the element. For details, see https://fb.me/react-unknown-prop
    in div

Warning: Unknown prop `_radiumDidResolveStyles` on <div> tag. Remove this prop from the element. For details, see https://fb.me/react-unknown-prop
    in div (created by StyleRoot)
    in StyleRoot (created by ReactImageLightbox)
    in div

Apologies for the inconvenience. Since React 15.2.0, React has started to throw warnings whenever unknown properties were assigned to DOM elements. I'm currently working on a branch that will resolve the top warning, but the bottom warning is dependent on this issue for Radium being resolved.

This is fixed in 3.0.0. Please let me know if there are any degradations on your upgrade (see the release notes for details)


float16, half precision, or mixed precision training with Keras

Hi,
Looking for advices about mixed precision training or half precision training with Keras.
Does setting floatx:float16 work well for half precision training ? I do not observe any speedup on Volta architectures / tensorcores ?
Thanks you

I posted a similar issue on stack overflow:
https://stackoverflow.com/questions/49782579/float16-slower-than-float32-in-keras
Did you ever get any speed-up from half precision training?

You need a deep neural architecture like Resnet50 in training. When using a shallow architecture, I think that type casting is more time-consuming.


Request: Support writing models with multiple loras applied (at 4-bit)

Related to #60 , would it be feasible to add the ability to take a base model and a list of loras, and output a merged model with the loras "baked in"?  #60 refers to "multi adapter" which I think is closely related, and references the project that should enable this, however that ticket relates to switching the current lora on the fly (e.g. running multiple loras off the same base model in memory at a single time).
I'm currently trying to work around the memory limitation imposed by running loras at inference time.  My proposed solution requires being able to write out a safetensors or other format, at 4-bit, that can then be used for inference later.
This capability really expanded what was possible in the Stable Diffusion community and created a whole ecosystem of sharing - I would be quite excited to see it come to alpaca_lora_4bit first!
I may be able to scrape together a PR, but Python is not my first language.  If anyone smarter has a bit of time/energy to invest in whipping up a separate python file, or if anyone can provide guidance on what it would take specifically, please let me know!

I think to bake in lora, we need the original weight but not the 4-bit one, because 4-bit already have some decrease on its performance due to quantization, If you bake the lora into 4-bit model and quantize it again, maybe the performance would be even worse.
Peft already have the feature of baking in lora, seems like the peft.tuners.lora.Linear.eval() method, after that you can just save the whole checkpoint.
Also If you want to bake in lora with 4-bit, you can use quant_cuda.vecquant4recons_v2 to reconstruct 4 bit weight in fp16, and then bake in lora, then save, then quantize it again.

Fascinating.  I didn't realize that it would be necessary to "rehydrate" the model back to FP16; given that, I agree, it probably makes sense to bake the 4-bit LoRA into a 16 bit version and quantize again.
I'm also wondering if I can use state dicts for this, or if the adapter files (which are created at the end of the process) are somehow better.
I've been chatting with GPT3 a lot and learning a lot today, but there are a lot of concepts and information on the topics exists but is spread out.
The other thing I am wondering is low large the training data sets need to be; I had heard/read that 10MB of text was a good guideline.  I recently tried with less than a megabyte, and the training seems to have worked fine.  Is there an upper or lower limit on size of training files?  (I'm training in text mode.). Anyhoo, thanks for taking the time to reply.  What I've given me might be enough to get me unblocked.

Any size would work. You can start with small dataset just like the sample dataset in the repo. Try giving something new the the model and test if it learned it from training.

Thanks :) I've run three datasets now, from about 800K all the way up to 10MB.  At first I trained for 3 epochs, but found the training wasn't "strong" enough; now I'm running training for 12 epochs.  My loss on the 10MB dataset is down to 0.4 with about 5 hours to go (90% complete) - I suspect this will be drastically overfit, but I wanted to get a full-on positive result so I could then "bisect" to get back to a reasonable value.
Regarding the original intent of this ticket, I think it can be closed since you filled me in on a strategy to get this done.
Something else I'm wondering is if it's possible to use the periodic checkpoints for inference?  Since we can periodically save checkpoints to resume training, these could also be used to save progress periodically and then use those saved checkpoints to run a training data set and test perplexity, or just ad-hoc test response quality.  If this was possible, one could choose a high number of epochs, and then test against each epoch separately without having to run each training set over and over.  Do you happen to know if that's possible?  So far, from the research I've done, I haven't found a way, but I expect it's simple.

peft.tuners.lora.Linear.eval()
quant_cuda.vecquant4recons_v2

I will definitely be taking a look at these functions, thanks so much for the reply.


Replace outdated url of Cookiecuter.dockerfile

Clicked the URL & it was outdated due to this change: https://github.com/backstage/backstage/commit/ad9d74ff240c934ce46a474ae3f440d5f5fa493b
No update or verification of the actual documenation content

tack så mycket! @OrkoHunter always great to see your name here :)

Thank you for contributing during Hacktoberfest 🍂! You can claim the Backstage Hacktoberfest Holopin at https://bck.st/hacktoberfest-holopin 🙌🏻


[Fargate] Document Automatic injection

Specify the requirements and the possibility to do it manually if needed
Clearly point to the required permissions added when injecting the sidecar
Explain how to garbage collect resources
Explain that the solution when failing would be NOT a bloker for deployment
Explain in the FAQ the hybrid solution

Is this still needed?

yes!