Use in-house benchmarking framework instead of swift-benchmark

Hello!
I was unhappy with the the Google's swift-benchmark library:

It doesn't allow automatic comparison of benchmarks, and the results are not exposed within the benchmark app.
It uses DispatchTime.now to timestamp events, which can be unreliable according to Stack Overflow.
It looks semi-archived, with a lonely PR sitting for more than 6 months without any acknowledgement from the maintainers.

I've drafted this library today to try to fix these issues.
Firstly, this library is source compatible with swift-benchmark. In swift-parsing, one can remove the swift-benchmark dependency and create a Benchmark target with this code and it "just works" (there is only one change to make: annotate one of the internal function helper as mutating as BenchmarkSuite is a now a struct)
The library introduces an intermediate group called BenchmarkStudy which compares some alternative cases to a baseline one. The hierarchy is Suite>Study>Case. Each study is independent from another.
Each case is instantiated with a block receiving a (MeasureDelimiter) -> Void argument. You can use this function with MeasureDelimiter's cases .start or .stop to optionally delimit the portion of code that is measured. There is no setup or tear-down as the delimiters allow to execute setup, benchmarks and tear-down in the same block. The use of the delimiters is optional and the library uses the block boundaries if they're absent.
As an example, the updated code that take advantage of the studies for TCA is:
benchmarkSuite("Scoping") { suite in

  suite.addStudy("Deep scoping") { study in

    study.setBaseline("No scope") { measure in
      let store1 = Store(initialState: 0, reducer: counterReducer, environment: ())
      let viewStore = ViewStore(store1)
      measure(.start)
      viewStore.send(true)
    }

    study.addCase("Scope (1)") { measure in
      let store1 = Store(initialState: 0, reducer: counterReducer, environment: ())
      let store2 = store1.scope { $0 }

      let viewStore = ViewStore(store2)
      measure(.start)
      viewStore.send(true)
    }

    study.addCase("Scope (2)") { measure in
      let store1 = Store(initialState: 0, reducer: counterReducer, environment: ())
      let store2 = store1.scope { $0 }
      let store3 = store2.scope { $0 }

      let viewStore = ViewStore(store3)
      measure(.start)
      viewStore.send(true)
    }

    study.addCase("Scope (3)") { measure in
      let store1 = Store(initialState: 0, reducer: counterReducer, environment: ())
      let store2 = store1.scope { $0 }
      let store3 = store2.scope { $0 }
      let store4 = store3.scope { $0 }

      let viewStore = ViewStore(store4)
      measure(.start)
      viewStore.send(true)
    }
  }
}

Each benchmark case is run by default 1,000,000 times or during 5s, whichever comes first. Parameters are always shared in a study.
The library runs all the cases of all the studies of all the suites and present the results in a human friendly way. As an example, the current benchmark for TCA shows:
== Scoping ==================================================================
               Duration Difference Variation Performance Std.Error Iterations
-- Deep scoping -------------------------------------------------------------
No scope  (*) 12.738 µs                                    ±0.12 %     276000
Scope (1)     22.478 µs  +9.740 µs  +76.46 %       x0.57   ±0.11 %     159270
Scope (2)     31.876 µs +19.138 µs +150.24 %       x0.40   ±0.13 %     113743
Scope (3)     42.199 µs +29.461 µs +231.29 %       x0.30   ±0.15 %      86687
=============================================================================

The asterisk (*) denotes the fastest performer. The first row is empty for Difference, Variation (in %), and Performance (in "times faster") because it's the baseline. The time unit is automatically determined according to the data to be displayed (it is the same for all the cases of a study).
On the technical side, the library uses clock_gettime_nsec_np to retrieve a timestamp, which is the Apple's recommendation. It also uses Accelerate if available to compute the mean and standard deviation of the results.
More importantly, the library is already set to execute transversal benchmarks in the spirit swift-collections-benchmark. You can parametrize the execution block with a generic parameter (let say the number of pullbacks), and the library will execute the same sequence described above, but also varying the parameter according to your prescription. This is not really exposed right now as I don't now how to present the results in ASCII, but I guess it should be possible to draw graph images at some point. The currently exposed non-parametric API is hiding the generic parameter behind a typealias.
I guess this module should be extracted as an independent library (it also misses @inlinable annotations for now). If there is some interest, I can update swift-parsing benchmarks to take advantage of the studies. I've tested it at home and it gives similar results as swift-benchmark. In source compatibility mode, it runs all the benchmarks as baseline-only studies, but it can be way nicer if some proper studies are defined. For example, here is a partial result running Comma separated benchmarks in the Numeric suite, both as a proper study and in source compatibility:
== Numerics ============================================================================================
                                          Duration Difference Variation Performance Std.Error Iterations
-- Comma separated -------------------------------------------------------------------------------------
Double.parser                       (*)  13.772 ms                                    ±1.37 %        360
Scanner.scanDouble                      111.553 ms +97.781 ms +710.02 %       x0.12   ±0.71 %         45
String.split                             36.388 ms +22.616 ms +164.22 %       x0.38   ±0.92 %        137
--------------------------------------------------------------------------------------------------------
Comma separated: Double.parser           13.898 ms                                    ±1.39 %        357
Comma separated: Scanner.scanDouble     113.646 ms                                    ±0.57 %         44
Comma separated: String.split            36.877 ms                                    ±0.89 %        136
========================================================================================================

and running using swift-benchmark:
name                                         time             std        iterations
-----------------------------------------------------------------------------------
Numerics.Comma separated: Double.parser       12137402.000 ns ±  30.29 %        107
Numerics.Comma separated: Scanner.scanDouble 113536294.000 ns ±   4.45 %         13
Numerics.Comma separated: String.split        36109335.000 ns ±  10.55 %         39

Most importantly, I guess this library will provide a more flexible base to benchmark TCA. I have no issue with @mbrandonw  and @stephencelis taking ownership of the code. This is a very rough draft and I'm sure there is many ways to improve it.
Please let me know what you think!

Whoa! This is really impressive stuff, @tgrapperon! Thanks for sharing 😄
You bring up a lot of good points. swift-benchmark is definitely less-than-ideal in many ways, and it does appear to be a shelved project for now. You've certainly improved upon a lot of its deficiencies.
We share your hunch that it probably doesn't make sense to hide this code away in the TCA repo, as we benchmark several of our libraries with the same tool (CasePaths, Parsing), so it would make sense to have a single package that each library depends on.
After discussing with @mbrandonw I think we're not quite ready to take on the work associated with refining and maintaining our own benchmarking library. We may in the future, though, and we'd certainly return to this draft as a starting point! Even though swift-benchmark is far from perfect, I think it serves our needs in the meantime, but if another library surfaces, we will be open to making a switch when we have the bandwidth.

Thanks @stephencelis!
I'll think again about the topic and I'll open source a draft library on my side then. I don't plan to maintain a fully fledged oss benchmarking library neither, but I can shelter it until it reaches a point where it can be incorporated into your projects. I'll keep you posted.
I think one can abstract the kind of result of each benchmark case to something more general than elapsed time, and it would be quite powerful to measure arbitrary values (like the number of time a ViewStore publisher emits for example).
Do you want me to close the PR now, or do we wait for my hosted version to be published?

Since there's no plan to merge it at this time I think closing is fine. If you'd like more eyes on it, though, definitely feel free to open a GitHub discussion with a link here!

Hey @mbrandonw and @stephencelis! I've extracted the benchmarking library and just published swift-measure.
I completely refactored the code and the library can now benchmark other things than the time spent. Many things were abstracted and it should be ultimately possible to measure the number of time a store does something on device at some point, and present the result in the form of a SwiftUI sheet for example.
Benchmarks (or measures) have a chain based API. For example, the standard benchmark block that computes the time spent to execute something is built as:
measureTime(label, tag: tag, block) // Measure the time spent for one execution, outputs TimeInterval
  .repeat(iterations: iterations, duration: duration) // repeat the same operation, outputs [TimeInterval]
  .map { $0.extractMeasure() } // Extract the mean and standard error from [TimeInterval], outputs Measure<TimeInterval>? (the array can be empty
  .unwrap(default: 0.0) // Unwraps the optional measure, producing a final Measure<TimeInterval>

There are other higher-order blocks like forEach which repeat a measure over a sequence of inputs (this can be useful to generate reports like swift-collections-benchmark for example), and the user can build its own setups. Most of them are defined at https://github.com/tgrapperon/swift-measure/blob/main/Sources/Measure/Blocks/Block.swift
There is an example of a custom measure at https://github.com/tgrapperon/swift-measure/blob/92770892d1c794ccf04f289fdc1a9d00572d0353/Tests/BenchmarkTests/BenchmarkTests.swift#L44 and below.
I've also updated the benchmarks of swift-case-paths and  swift-parsing  to take advantage of the library. I'm not doing any PR right now, but you can have a look if you want.
Here is the kind of output generated by the library when it outputs strings to the console:
swift-case-paths
== Extraction =======================================================================
                          Value  Difference Variation  Performance  Error  Iterations
-- Success --------------------------------------------------------------------------
Manual                (*)  53 ns                                   ±0.20 %    1000000
Reflection                127 ns     +75 ns  +141.58 %       x0.41 ±0.09 %    1000000
Reflection (uncached)     703 ns    +650 ns +1233.12 %       x0.08 ±0.05 %    1000000
-- Failure --------------------------------------------------------------------------
Manual                (*)  57 ns                                   ±0.26 %    1000000
Reflection                 95 ns     +38 ns   +66.08 %       x0.60 ±0.14 %    1000000
Reflection (uncached)     658 ns    +600 ns +1048.15 %       x0.09 ±0.07 %    1000000
=====================================================================================

or
swift-parsing
== Numerics ===================================================================
                   Value    Difference Variation Performance  Error  Iterations
-- Int ------------------------------------------------------------------------
Int.init     (*)      73 ns                                  ±0.75 %    1000000
Parser                78 ns      +5 ns   +6.54 %       x0.94 ±0.26 %    1000000
-- Comma separated - Int ------------------------------------------------------
String.split     105.182 ms                                  ±0.46 %         48
Parser       (*)   5.278 ms -99.904 ms  -94.98 %      x19.93 ±0.16 %        937
Scanner           73.178 ms -32.004 ms  -30.43 %       x1.44 ±0.32 %         69
-- Double ---------------------------------------------------------------------
Double.init  (*)     101 ns                                  ±0.18 %    1000000
Parser               140 ns     +39 ns  +38.25 %       x0.72 ±0.15 %    1000000
Scanner              527 ns    +426 ns +422.26 %       x0.19 ±3.56 %    1000000
-- Comma separated - Double ---------------------------------------------------
String.split      29.038 ms                                  ±1.35 %        172
Parser       (*)   9.768 ms -19.270 ms  -66.36 %       x2.97 ±0.27 %        508
Scanner           88.261 ms +59.223 ms +203.95 %       x0.33 ±0.54 %         57
===============================================================================

== README Example =================================================================
                        Value   Difference Variation Performance  Error  Iterations
-- README Example -----------------------------------------------------------------
Adhoc                  7.498 µs                                  ±0.03 %     624168
Parser: Substring      3.831 µs  -3.667 µs  -48.91 %       x1.96 ±0.04 %    1000000
Parser: UTF8      (*)  1.484 µs  -6.014 µs  -80.20 %       x5.05 ±0.04 %    1000000
Scanner               18.708 µs +11.210 µs +149.50 %       x0.40 ±0.03 %     258062
===================================================================================

== String Abstractions ========================================================
                 Value     Difference  Variation Performance  Error  Iterations
-- String Abstractions --------------------------------------------------------
Substring     1194.212 µs                                    ±0.21 %       4185
UTF8      (*)  146.747 µs -1047.464 µs  -87.71 %       x8.14 ±0.05 %      33968
===============================================================================

If you have the time to have a look, please let me know what you think!
Thanks!

I've just updated the library to show how to build as study that count the number of time a comparison is realized when sorting an array using two different algorithms. The study shows the results for a range of array's sizes, and the time spent could have been measured as easily. Everything was setup from the client side, with the tools provided by the library.
It's quite astonishing what a functional architecture can achieve in a 4 days solo side project.
I think there's room for a leaner and more powerful API though.
The report generated is in the form of
== Sorting algorithms ========================================================================================
                   Value      Difference    Variation  Performance  Error  Iterations
-- Time elapsed ----------------------------------------------------------------------------------------------
Swift sort  (*)   573.868 µs                                       ±0.11 %       8683
Bubble sort     98905.897 µs +98332.029 µs +17134.96 %       x0.01 ±0.47 %         51
--------------------------------------------------------------------------------------------------------------
                Comparisons 10 100   250   500    1000   2500     5000    10000    12500     15000     20000
-- Number of comparisons -------------------------------------------------------------------------------------
Swift Sort  (*)      17.5×N 36 1419  4521  10123  20070   41262    88330   186401   259974    351550    392616
Bubble Sort        3038.6×N 45 4950 31125<PHONE_NUMBER>00 3123750 12497500 49995000 78118750 112492500 199990000
==============================================================================================================

and the code to set it up can be found at https://github.com/tgrapperon/swift-measure/blob/main/Tests/BenchmarkTests/BenchmarkTests.swift


Fix example export dbadbrole

Type of Change

[X] Documentation

Purpose
Update example documentation based on my automated example testing script
Someone might need to spend some quality time with the examples and their description, to make sure that they reflect reality...

Someone might need to spend some quality time with the examples and their description, to make sure that they reflect reality...

If you would like to fix the examples please do so in this PR and then we can accept it.


The file cannot be viewed in Google drive

I creates some .xlsx files by this module and upload to Google drive. It is no problem to open them in ms excel. However, I cannot view them in the drive. It pops up a dialog "Oops! There was a problem previewing this document". I would greatly appreciate it if you can fix this.

Does it work on pure MS Office?

Same here.


Flaky pure virtual function called in engine host tests

Linux Host Engine: https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket.appspot.com/8857910757129505712/+/steps/Host_Tests_for_host_profile/0/stdout?format=raw
Mac Host Engine: https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket.appspot.com/8857963650630585808/+/steps/Host_Tests_for_host_debug_unopt/0/stdout?format=raw
Earliest instance is on https://github.com/flutter/engine/commit/d2c4de0cd483df40f3c068bbc79988e845b2da0e but that doesn't seem related.

I originally suspected this was something to do with the mocking framework but there the failing test on Mac does not use mocking at all. There are tons of gmock warnings that I am not sure are related.
cc @chinmaygarde

I originally suspected this was something to do with the mocking framework but there the failing test on Mac does not use mocking at all. There are tons of gmock warnings that I am not sure are related.
cc @chinmaygarde

Hit this today https://ci.chromium.org/ui/p/flutter/builders/prod/Linux Host Engine/8615/overview

Hit this today https://ci.chromium.org/ui/p/flutter/builders/prod/Linux Host Engine/8615/overview

Caused tree to be closed here https://ci.chromium.org/ui/p/flutter/builders/prod/Mac Host Engine/8641/overview.
Bumping the priority one notch to P3.

Caused tree to be closed here https://ci.chromium.org/ui/p/flutter/builders/prod/Mac Host Engine/8641/overview.
Bumping the priority one notch to P3.

The message appears to be coming from here:
https://fuchsia.googlesource.com/third_party/libcxxabi/+/refs/heads/master/src/cxa_virtual.cpp#16
Which boils down to a call to abort() here:
https://fuchsia.googlesource.com/third_party/libcxxabi/+/refs/heads/master/src/abort_message.cpp#76
But I don't understand why abort() isn't causing a backtrace to be printed by the signal handlers here:
https://github.com/flutter/engine/blob/master/fml/backtrace.cc
installed by(?):
https://github.com/flutter/engine/blob/master/testing/run_all_unittests.cc#L39

The message appears to be coming from here:
https://fuchsia.googlesource.com/third_party/libcxxabi/+/refs/heads/master/src/cxa_virtual.cpp#16
Which boils down to a call to abort() here:
https://fuchsia.googlesource.com/third_party/libcxxabi/+/refs/heads/master/src/abort_message.cpp#76
But I don't understand why abort() isn't causing a backtrace to be printed by the signal handlers here:
https://github.com/flutter/engine/blob/master/fml/backtrace.cc
installed by(?):
https://github.com/flutter/engine/blob/master/testing/run_all_unittests.cc#L39

Looked into this and found a few issues:

The FML signal handler that prints backtraces may itself crash due to a null pointer dereference (see https://github.com/flutter/engine/pull/23970)
The backtrace handler uses dladdr to find symbol names.  dladdr is unable to locate most of the relevant symbols because they have hidden visibility.  Also, dladdr finds symbols exported by dynamic libraries and will not find symbols within the executable unless it is linked using --export-dynamic.
Dart initialization is replacing the SIGABRT handler set up by fml::InstallCrashHandler

Would it be possible for the LUCI environment to capture a core dump when a crash like this happens?
@godofredoc

Looked into this and found a few issues:

The FML signal handler that prints backtraces may itself crash due to a null pointer dereference (see https://github.com/flutter/engine/pull/23970)
The backtrace handler uses dladdr to find symbol names.  dladdr is unable to locate most of the relevant symbols because they have hidden visibility.  Also, dladdr finds symbols exported by dynamic libraries and will not find symbols within the executable unless it is linked using --export-dynamic.
Dart initialization is replacing the SIGABRT handler set up by fml::InstallCrashHandler

Would it be possible for the LUCI environment to capture a core dump when a crash like this happens?
@godofredoc

Yes, you'll need to modify the runner to collect a dumpstate and place it in the folder referenced by FLUTTER_LOGS_DIR  env variable and infra will automatically upload the logs to GCS.

Yes, you'll need to modify the runner to collect a dumpstate and place it in the folder referenced by FLUTTER_LOGS_DIR  env variable and infra will automatically upload the logs to GCS.

I can make the change to the recipe but will need your help on modifying the runner.

I can make the change to the recipe but will need your help on modifying the runner.

@jason-simmons Do you have a pointer to where Dart is doing that?

@jason-simmons Do you have a pointer to where Dart is doing that?

See the signal handling code in https://github.com/dart-lang/sdk/blob/master/runtime/bin/process_linux.cc
In particular, these tests will be creating and destroying Dart VM instances and going through cycles of Dart_Initialize/Dart_Cleanup (including an explicit call to dart::bin::CleanupDartIo).  CleanupDartIo calls Process::Cleanup, which calls ClearAllSignalHandlers.

See the signal handling code in https://github.com/dart-lang/sdk/blob/master/runtime/bin/process_linux.cc
In particular, these tests will be creating and destroying Dart VM instances and going through cycles of Dart_Initialize/Dart_Cleanup (including an explicit call to dart::bin::CleanupDartIo).  CleanupDartIo calls Process::Cleanup, which calls ClearAllSignalHandlers.

Good catch on the dladdr usage inaccuracy. export-dynamic on all Linux unit-test exectuables seems like a fine thing to do. The signal handling seems to be different Mac and Linux and the crash handler seems to work fine after I create and shutdown the VM in the process on Mac.

Good catch on the dladdr usage inaccuracy. export-dynamic on all Linux unit-test exectuables seems like a fine thing to do. The signal handling seems to be different Mac and Linux and the crash handler seems to work fine after I create and shutdown the VM in the process on Mac.

To clarify, this is happening much more frequently on the bots on Mac, so Linux-only fixes probably won't help too much.

To clarify, this is happening much more frequently on the bots on Mac, so Linux-only fixes probably won't help too much.

I was able to reproduce this on a Mac using the TypeConversionsTest.CanConvertEmptyList test.
There is a race between Dart VM shutdown and a deferred task posted to one of the test's threads.  When the TypeConversionsTest deletes its DartVMRef, Dart_Cleanup will delete the Timeline::recorder_ (which is a process-wide global).
However, there may be a pending call to MessageLoopImpl::FlushTasks on another thread.  FlushTasks logs a timeline event, and if the timeline recorder is partially deleted then this will fail with the pure virtual call error.
I think we only started seeing this recently because the timeline had been disabled during tests until https://github.com/flutter/engine/commit/f392053b1f2a10ed4022341620f341359b5f8401 landed.
@chinmaygarde What is the best way to make this safe?
The thread created by the TypeConversionsTest (which appears to be owned by the ThreadTest superclass) needs to be stopped before the TypeConversionsTest drops the DartVMRef.

I was able to reproduce this on a Mac using the TypeConversionsTest.CanConvertEmptyList test.
There is a race between Dart VM shutdown and a deferred task posted to one of the test's threads.  When the TypeConversionsTest deletes its DartVMRef, Dart_Cleanup will delete the Timeline::recorder_ (which is a process-wide global).
However, there may be a pending call to MessageLoopImpl::FlushTasks on another thread.  FlushTasks logs a timeline event, and if the timeline recorder is partially deleted then this will fail with the pure virtual call error.
I think we only started seeing this recently because the timeline had been disabled during tests until https://github.com/flutter/engine/commit/f392053b1f2a10ed4022341620f341359b5f8401 landed.
@chinmaygarde What is the best way to make this safe?
The thread created by the TypeConversionsTest (which appears to be owned by the ThreadTest superclass) needs to be stopped before the TypeConversionsTest drops the DartVMRef.

We guarantee that a DartVMRef can be created and destroyed on any thread. Creation, destruction and acquisition of new strong references to a VM already running in the process is done using a single global mutex. I suppose we could trace to the timeline with this mutex held. There is very little contention on this lock today as its done only when an engine is created and destroyed. However, I'd still use double-checked locking to access this mutex if we are using this for each timeline trace.

We guarantee that a DartVMRef can be created and destroyed on any thread. Creation, destruction and acquisition of new strong references to a VM already running in the process is done using a single global mutex. I suppose we could trace to the timeline with this mutex held. There is very little contention on this lock today as its done only when an engine is created and destroyed. However, I'd still use double-checked locking to access this mutex if we are using this for each timeline trace.

Would it work if DartVM sets a flag in the fml/trace_event library that controls whether trace_event actually calls Dart_TimelineEvent?
The flag would be set after calling Dart_Initialize and cleared before calling Dart_Cleanup.

Would it work if DartVM sets a flag in the fml/trace_event library that controls whether trace_event actually calls Dart_TimelineEvent?
The flag would be set after calling Dart_Initialize and cleared before calling Dart_Cleanup.

That wouldn't really work because of a TOCTOU race though. You'd need to hold a mutex while the trace is being written.

That wouldn't really work because of a TOCTOU race though. You'd need to hold a mutex while the trace is being written.


change track_as default in senegal.py

Brief description
Need to change track_as to 0 as default, while fixing the visible deprecation warning issue and possibly refactoring age-specific tracking code. Prerequisite to issue #124 but does not close it. Resolves issue #127
Type(s) of change

[x] Bugfix
[ ] Refactor
[ ] New feature
[ ] Other

Checklist

My code follows the style guide

[ ] Yes
[x] N/A

I've commented my code

[ ] Yes
[x] N/A

I've incremented the version number

[ ] Yes
[x] N/A

I've updated the changelog

[ ] Yes
[x] N/A

I've added tests and checked code coverage

[ ] Yes
[x] N/A

@MObrien-IDM Agree to change default of track_as to 0.   We can fix the warning by just adding the dtype=object line of code where it is throwing the warning in the library code.  Do you want to do that too with this PR?

@MObrien-IDM this should be good to merge now, my poor band-aid solution is removed :-)


Streamline steps for testing locally

bumps our version to 0.1.1 following our tech preview
adds a new make target run-local with only the steps needed to run the controller locally against your current openshift cluster

/retest

/retest


Types don't implement INotifyPropertyChanged?

Tried to use this.
Was able to generate the code, and create a test
This fails at runtime
 client.SaveChanges(SaveChangesOptions.PostOnlySetProperties | SaveChangesOptions.BatchWithSingleChangeset);

System.InvalidOperationException
An attempt to track an entity or complex type failed because the entity or complex type 'Microsoft.Dynamics.DataEntities.LedgerJournalLine' does not implement the INotifyPropertyChanged interface.
at Microsoft.OData.Client.DataServiceCollection`1.InsertItem(Int32 index, T item)

I didn't see any option to have types implement INotifyPropertyChanged
This option is available for example when using VS's add odata service reference

@dazinator thanks for the feedback!
It seems i missed enable-tracking command line option.
Can you manually add --enable-tracking to command from OData CLI UI tool window and check generation result?

@dazinator new version 1.2.0 provides --enable-tracking option
https://plugins.jetbrains.com/plugin/24117-odata-cli-ui/edit/versions/stable/591661
https://github.com/ellizio/odata-cli-ui/releases/tag/1.2.0-2024.2


Rearrange training link

Rearrange the training link according to the xls file

@morshedniaz - looks good! Your git history is kind of a mess, but we'll squash it into a single commit to keep the main branch clean.
Here's a preview of what the site will look like when it goes live: https://49-293151248-gh.circle-artifacts.com/0/intersect-training.github.io/training-links/index.html.
I like it and don't have any real concerns. A few of the categories now have just a single item in them...but I think that's ok. Other is now much smaller, which is good.

The changes look good to me. Hopefully we now have a structure that will allow us to do better sorting/filtering in the future.

Dear Professor,
Thank you so much for your comments. I have corrected the spelling and tried to make another pull request on the same branch but I can't. Should I create another branch for the spelling correction? please let me know.
Sincerely yours
Niaz
Md Niaz Morshed
Graduate Research Assistant
Department of Computer Science
University of Alabama, Tuscaloosa
USA

From: Ian Cosden @.>
Sent: Monday, January 10, 2022 2:25 PM
To: INTERSECT-training/intersect-training.github.io @.>
Cc: Md Niaz Morshed @.>; Mention @.>
Subject: [EXTERNAL] Re: [INTERSECT-training/intersect-training.github.io] Rearrange training link (PR #30)
@cosden requested changes on this pull request.
 Otherwise looks good to me

In _data/training.ymlhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FINTERSECT-training%2Fintersect-training.github.io%2Fpull%2F30%23discussion_r781522131&data=04|01|mmorshed%40crimson.ua.edu|3a3e6f5595f942b91c6808d9d4775757|2a00728ef0d040b4a4e8ce433f3fbca7|0|0|637774431293860613|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=ORuPXWkanj56Va%2Fu1DieBY7LjP%2BR3HdNCqfJXU9dVUQ%3D&reserved=0:

name: Python Packages

url: https://py-pkgs.org/
author: Tomas Beuzen & Tiffany Timbers
description: Python Packages is an open source textbook that describes modern and efficient workflows for creating Python packages.

tags: {Packaging, Book}

tags: {Packaging, Pyhton in Research}

⬇ Suggested change

tags: {Packaging, Pyhton in Research}

tags: {Packaging, Python in Research}

—
Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FINTERSECT-training%2Fintersect-training.github.io%2Fpull%2F30%23pullrequestreview-848302657&data=04|01|mmorshed%40crimson.ua.edu|3a3e6f5595f942b91c6808d9d4775757|2a00728ef0d040b4a4e8ce433f3fbca7|0|0|637774431293860613|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=33tFMURprAFBVPWKkRUT4%2FZb7balndM1%2BVZFOQkBZwQ%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAVQXILHCNRC3WODH3AIK4MTUVM6DPANCNFSM5LQ5WBVQ&data=04|01|mmorshed%40crimson.ua.edu|3a3e6f5595f942b91c6808d9d4775757|2a00728ef0d040b4a4e8ce433f3fbca7|0|0|637774431293860613|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=SfEbQhDiBWP18soUIyEmiSCon5RKeSIq%2FO%2FBgBvoLEE%3D&reserved=0.
Triage notifications on the go with GitHub Mobile for iOShttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04|01|mmorshed%40crimson.ua.edu|3a3e6f5595f942b91c6808d9d4775757|2a00728ef0d040b4a4e8ce433f3fbca7|0|0|637774431293860613|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=oE8eP22MAZI4f5WngIp43zNfw73DyW6PyLszSClf7vQ%3D&reserved=0 or Androidhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04|01|mmorshed%40crimson.ua.edu|3a3e6f5595f942b91c6808d9d4775757|2a00728ef0d040b4a4e8ce433f3fbca7|0|0|637774431293860613|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=78h9svZmXtofeDYL1%2BUiIr25uCyaLjg5iX67jmlFPN4%3D&reserved=0.
You are receiving this because you were mentioned.Message ID: @.***>

Your latest commit fixed it (https://github.com/INTERSECT-training/intersect-training.github.io/pull/30/commits/648a19cd70ad6bac3b4a5dbdc2da6bd1c7f061af).
Looks good. Thanks!


Xros Mini mode is reading incorrect data from toy

Perhaps the timings need adjusted.

Actually the 1 and 0 bits are switched.


Simsimi

Simsimi its no more working.
Any workround ?

You have to use a trial API key (here) and make requests to simsimi. If you have any troubles ask again.

can you show what part of code must change ?

You could for example add this library to your lib folder. Then add this code to your else statement:
else:
            from lib.simsimi import SimSimi
            from lib.language_codes import LC_ITALIANO
            from lib.simsimi import SimSimiException

            back = ''
            simSimi = SimSimi(conversation_language=LC_ITALIANO,
                              conversation_key='YOUR_API_KEY')

            try:
                from unicodedata import normalize
                text = normalize('NFKD', text).encode('ASCII', 'ignore')
                response = simSimi.getConversation(text)
                if not response['response']:
                    reply('You exceeded simsimi api daily limit!')
                back = response['response']
            except SimSimiException as e:
                    print e
            if not back:
                reply('Something went wrong..')
            elif 'I HAVE NO RESPONSE' in back:
                reply('you said something with no meaning')
            else:
                reply(back)


Fix typos and remove obsolete Service APIs reference

What type of PR is this?
/kind cleanup
/kind documentation
What this PR does / why we need it:
This fixes several typos, removes an obsolete Service APIs ref, and updates any remaining "Gateway APIs" refs to "Gateway API".
Does this PR introduce a user-facing change?:
NONE

/lgtm


view scoped events causes issues with multiple views on the same element

I just updated from 0.3.3 to HEAD, and the following commit introduced a bug into my app:
https://github.com/documentcloud/backbone/commit/17602c39278810c46a265631935c69b5ab059de7
We have some views that bind to the same element (#content in this particular instance). The problem is that delegateEvents is no longer unbinding the previously bound events (which it did before.) So the events are stacking.
I'm not sure what the best solution is. I can obviously write my own method to unbind and then re-delegate, but it'd be great of Backbone could be more (or less?) clever about it.

That's the use case, yes. We had a a parent view that child views inherited functionality from. Something like content with about that inherited from it. The parent had an el that was set to an existing DOM element. The inheriting views then were binding to the same DOM element.
I moved to the solution referenced above, which solved it. Rather than setting el, I set an id and adjust the render method to insert into the DOM, rather than setting new content.
I'm not sure that either approach is conceptually better or worse. It's just that the former doesn't really work as you might expect, and it's something a subset of people are trying (given the people attached to the issue.) Perhaps a documentation change can alleviate it?

They weren't duplicate events. The problem that I ran into was that as multiple children were added and then removed, the events of the children were sticking around and banging on each other. The original issue was filed because it was a change in functionality from the (now, much) earlier version of Backbone. In that earlier version, ALL events for the element were unbound, versus just those specific to the view. So, it worked before and then it didn't.
Perhaps others can chime in with different situations?

Same issue here after updaing our app to 0.5.3.
We have a number of pages(home, search results, booking), each of them having a predefined structure(a number of div's with id's) and what we used to do - is when you switch between pages, then we re-create views(passing id as an 'el' option). Now if you go from search results back to home - the index view is recreated on same DOM element and we have double events causing weird bugs.

You should now be able to clear delegated events on a shared DOM element by calling undelegateEvents on your view...

@jashkenas: An easier fix would be to disallow binding the same event, to the same element, multiple times. I expected the events to be created and destroyed together with the view, so that a new view started from scratch, without any events bound to it. Having a undelegateEvents helps a little, but still goes against my expectations.

It's not necessarily the same event. It's perfectly legal to have two different views applied to the same element with different sets of functions keyed to the same event targets...
If this is really giving you any trouble, I'd recommend doing the suggested Backbone thing, and never assigning to view.el directly, ensuring that each view has a unique, always-available element to delegate from ... really, it makes life much simpler.

@jashkenas: Thanks, I'll do that.


Please Add  load in 8 bits for InstructBLIP

Therefore, RTX 3090 users can run the 13B version

👍  - can't find a way to use the lavis libraries to be able to pass through any of the load_in_8_bits = True etc config. which means that it's quite difficult to load large models like the 7b or 13b vicuna into 8 bits. Which practically means you can't run the 13B InstructBLIP model on a 24G 3090.

The same issue.


added fade in animation for bar chart

Added fade in and out animation for bar chart between weekly cadet xp and daily active users

@Bunyod16 you didnt include the new feature in the page? Seems like the changes are only add two new files.


Incorrect use of extend_fsdp_sharding_meta() in cross_fused_attn()

extend_fsdp_sharding_meta() was updated in 6464ced to return a tuple of both the sharding meta and the FSDP axis name.
self_fused_attn() was updated to accommodate this API change, but cross_fused_attn() was not.
This PR applies the same fix from self_fused_attn() to cross_fused_attn().
Bug reported by @terrykong.

/te-ci jax


Move Form types to GmailBundle

Would it be a good idea to move the form types to GmailBundle, @galeaspablo?

Yes. But because the form type depends on syncsettingclass, you would need to move that over there as well. So you would need to abstract syncsettingclass in the gmailbundle in the same way the other classes (email, label, history) are abstracted.


oauthLogin.html can't be found

Hi !
I've installed 1.0.4 version of this plugin into my my local TeamCity instance v10.0.5 (build 42677). After adding the OAuth module and configuring it the new option appeared on the login screen.
However then I press on it I see an error:

No webpage was found for the web address: http://servername/oauthLogin.html?submitLogin=Log+in+via+oAuth
HTTP ERROR 404

How can I fix this?

Do you use docker version?
I haven't seen errors like that.
What is your operating system?

TeamCity is running on Windows Server 2012 inside of Virtual Box VM.
We plan to migrate it to a docker although.

I checked latest version on Windows Azure (Windows 2012) - all seems to work fine.
If issue persist, I would need more information to help.

Hello, @shcherbachev  I encountered the same problem today.  My problem was that I had listed multiple scopes in the OAuth scope option email profile and the url was not encoded on the way over to the OAuth provider resulting in the following exception (found in log/teamcity-winservice.txt) because a space is not a legal url character:
java.lang.IllegalArgumentException: Illegal character in query at index 160
In order to get it to work I had to manually escape my scope options to email%20profile and then it worked fine.
@pwielgolaski If you decide to escape it, let me know because the update would doubly escape my scope breaking my authentication.  Thanks!

Fixed by https://github.com/pwielgolaski/teamcity-oauth/commit/03b62a81a7d9b7d14e43c5ac5e0a5a186cf3393f
It is part of https://github.com/pwielgolaski/teamcity-oauth/releases/tag/teamcity-oauth-1.1.0


Proposal: add language runtime.

I would like to give feedback that you should integrate wasm runtime like wazero.io, because it is also developed in Go, while other wasm runtimes are in C or C++. This makes it necessary for developers to learn C++ or other programming languages if they want to contribute code. The integration of a runtime implemented in the same type of Go programming language as dapr would help developers get started with dapr more easily and do secondary development.

Hi @auula, dapr currently has supported wasm middleware which is based on wazero runtime. And wasm output binding is also on its way. Is this what you are looking for, or any other features that you want dapr to support?

Hi @auula, dapr currently has supported wasm middleware which is based on wazero runtime. And wasm output binding is also on its way. Is this what you are looking for, or any other features that you want dapr to support?

thank！I didn't read the official documentation carefully, wazero without dependency on cgo is a good choice!


Windows build

:question: What
This PR is for support build under win arch. We can close #113

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@guerinoni thanks for this PR. I think runs-on doesn't like the array of operating systems. You should probably add a job for each os

@tommyblue now should be fine but I can't see the workflows result :O

@guerinoni nope, still not working: https://github.com/guerinoni/baker/actions/runs/442485633

Codecov Report

Merging #113 (67d61f1) into main (af7b7c1) will increase coverage by 0.02%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #113      +/-   ##
==========================================
+ Coverage   45.92%   45.95%   +0.02%
==========================================
  Files          51       52       +1
  Lines        3671     3669       -2
==========================================
  Hits         1686     1686
+ Misses       1793     1791       -2
  Partials      192      192

Impacted Files
Coverage Δ

baker_syscall.go
0.00% <0.00%> (ø)

help_markdown.go
20.45% <0.00%> (+0.45%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update af7b7c1...67d61f1. Read the comment docs.

@guerinoni you only addressed a part of the comments I made. If my comment about the termSize function is not clear I can expand a bit or we could have a fast call about that.

@guerinoni can you expand a bit as to why a darwin github workflow file was needed? In what is it different from, say, the linux one?
And in the same manner, why a windows-specific one is required?

@arl the platforms are different and it can be useful building application under all 3 main platforms, in future something can be different

@arl the platforms are different and it can be useful building application under all 3 main platforms, in future something can be different

I agree with you but I'd prefer the PR to stay focused on making baker build on windows, so I don't see why darwin is involved. Also it could be nice, if we add a windows-specific workflow, to have the ci run baker tests and not just build.
Once this is done, we can discuss in another pr about adding darwin-specific ci build.
Also, once the lua filter PR gets merged we can also discuss about making girhub-ci creates artifacts, baker binaries one can download and use directly.

Thank you!

Thank you!


Get the handlebar instance

I'm trying to extend handlebars with helpers from https://github.com/shannonmoeller/handlebars-layouts. To register the helpers i need access to the underlying handlebars instance of express-handlebars.
How can i get the instance?

It is exposed via the handlebars attribute. So for this module, I think this would work:
const layouts = require('handlebars-layouts');
const hbs = exphbs.create();

layouts.register(hbs.handlebars);

The code above allows handlebars-layouts to register, but doesn't allow it to work with express-handlebars.
Calling {{#extend "layouts/foo"}} fails with the error
Error: Missing partial: 'layouts/foo'
    at Object.extend (/app/node_modules/handlebars-layouts/index.js:120:11)
    at Object.eval [as main] (eval at createFunctionContext (/app/node_modules/handlebars/dist/cjs/handlebars/compiler/javascript-compiler.js:257:23), <anonymous>:5:92)
    at main (/app/node_modules/handlebars/dist/cjs/handlebars/runtime.js:175:32)
    at ret (/app/node_modules/handlebars/dist/cjs/handlebars/runtime.js:178:12)
    at ret (/app/node_modules/handlebars/dist/cjs/handlebars/compiler/compiler.js:526:21)
    at ExpressHandlebars._renderTemplate (/app/node_modules/express-handlebars/lib/express-handlebars.js:247:12)
    at ExpressHandlebars.<anonymous> (/app/node_modules/express-handlebars/lib/express-handlebars.js:173:21)

This is how I'm setting the two up
const path = require('path')
const express = require('express')
const exphbs = require('express-handlebars')
const layouts = require('handlebars-layouts')
const handlerbarsHelpers = require('./handlebars-helpers')

const app = express()

const hbs = exphbs.create({
  helpers: handlerbarsHelpers,
  extname: 'hbs',
  partialsDir: path.join(__dirname, 'views', 'partials')
})

hbs.getPartials().then(partials => console.log(Object.keys(partials)))

app.engine('.hbs', hbs.engine)
app.set('view engine', '.hbs')
app.set('views', path.join(__dirname, 'views'))

layouts.register(hbs.handlebars)

app.get('/foo', (req, res) => {
  res.render('foo')
})

I can fix the issue by manually registering the partial with handlebars. Is there a way to avoid that?
hbs.getPartials().then(partials => hbs.handlebars.registerPartial('layouts/foo', partials['layouts/foo']))

You could also configure your handlebars instance separatly then pass that in via the handlebars config option when creating your express-handlebars instance.
Maybe something like this?
const handlebars = require('handlebars');
const layouts = require('handlebars-layouts');
const exphbs = require('express-handlebars');

handlebars.registerHelper(layouts(handlebars));

const hbs = exphbs.create({
  handlebars: handlebars,
  // ... Other config options here (helpers, partialsDir, etc.)
});

@romellem With your solution, the partials still need to be registered manually, as @dowtkrob suggested above.


high memory consumption from Docker images

What happened: The splunk/fluentd-hec:1.2.4 Docker image uses twice as much memory as the 1.1.1 version, and I am seeing more frequent occurences of Worker 0 finished unexpectedly with signal SIGKILL.
What you expected to happen: I expected the memory consumption to be about the same as before.
How to reproduce it (as minimally and precisely as possible): Use the 1.2.4 version w/ the same configuration as 1.1.1.
Anything else we need to know?:
Environment: AWS ECS

Hello,
even if you are using the same configuration, memory use might be different based on the load of log it is ingesting and the rate that buffer is being transferred to Indexers.

Hello,
even if you are using the same configuration, memory use might be different based on the load of log it is ingesting and the rate that buffer is being transferred to Indexers.

is pod getting killed by out of memory issue? if so, i would suggest to increase memory limit of it.
There is no significant changes from 1.1.1 to 1.2.4 except libraries we are updating with every major and minor release.

is pod getting killed by out of memory issue? if so, i would suggest to increase memory limit of it.
There is no significant changes from 1.1.1 to 1.2.4 except libraries we are updating with every major and minor release.

i've been testing this out on workloads that are pretty consistent in their log outputs.  it appears the Dockerfile has changed significantly during the releases: https://github.com/splunk/fluent-plugin-splunk-hec/compare/1.1.1..1.2.4#diff-f34da55ca08f1a30591d8b0b3e885bcc678537b2a9a4aadea4f190806b374ddcR1

i've been testing this out on workloads that are pretty consistent in their log outputs.  it appears the Dockerfile has changed significantly during the releases: https://github.com/splunk/fluent-plugin-splunk-hec/compare/1.1.1..1.2.4#diff-f34da55ca08f1a30591d8b0b3e885bcc678537b2a9a4aadea4f190806b374ddcR1

perhaps it's b/c the newer Docker image does not use jemalloc?  will try to test this out soon.

perhaps it's b/c the newer Docker image does not use jemalloc?  will try to test this out soon.

after more debugging and configuration tuning, i could not figure out the reason for the memory leak i was observing.  i ended up using fluent-bit since that has a splunk output, and the memory consumption is much lower (< 4MB compared to ~50MB baseline)


Beta React docs: CSS issue overlaping nav bar and main content

At https://beta.reactjs.org there is a CSS issue.
Pictures worth thousand words so...

Note:
quick work-around: if you add a max-width of 300px to the aside on the left, that would fix the thing.

Not able to repro. Is this Firefox? Which OS?

Sorry! forgot to mention, YES this is Firefox;
Firefox 104.0.2 (64-bit)

This happens when in the Mac's 13 inch screen
MacBook Pro (13 inch, 2019)


contains fails when value is wiki-link unless combined with a string

given this data:
location:: [[Lizzy]]
format:: [[DVD]], [[Blu]]
store:: Barnes, [[AmazonUS]], [[NetflixUS]]

The only way to get a result from location is with a where location=[[Lizzy]].
All where contains clauses fail.
Same applies to format, except that you can't use where format= because there are multiples, so dataview will never return results.
All contains clauses work fine with store because there is at least one string. So all of these work, including truncated searches:
where contains(store, "Barnes")
where contains(store, "Amaz")

Yep, I’ve seen that you’re using the :: format inside the note, but I still think DV interprets these YAML-like.
If so, using quotes like "[[DVD]], [[Blu]]" would make it a string, thus perfect for contains(). But if you’d need to get at single elements, using brackets like [ [[DVD]], [[Blu]] ] might be better (this creates a heavily nested map of seq, i.e. array).

@Moonbase59 right, using the brackets around it as per real yaml did the trick. Honestly I'm not sure why I didn't think of that. I use them all the time frontmatter. I think its best to avoid the wiki links in this case, even though I can get them to work now. It could probably break, and I don't want to have to redo them all at some point. Thanks for helping me out.

Happy to help!


org.apache.hadoop.fs.Path HdfsPathTypeArgument implementation

Since Hadoop developers are growing rapidly, I think this would be really beneficial for Hadoop developers to have HdfsPathTypeArgument.

I would like to contribute if this is something decided to be implemented.

Fixed by #88. If more Hadoop argument types are needed, please open a new issue.


License product name mismatch

There are a couple of places I have found clang format mentions:

LICENSE
SwiftLintLinter.php

Could you please fix at least license file, to allow me to use your code in my project.

@fen-x thanks for reporting

Thank you. That helps me a lot.

I just wonder will you use this linter as a part of your build pipeline or you plan to build something new on top of it?

As a part of build pipeline. Just want to be on the safe side from legal perspective.


Github codespaces short link redirects to the old repo

The short link used in the documentation:
https://aka.ms/ghcs-default-image
is still redirecting to the old repo
https://github.com/microsoft/vscode-dev-containers/tree/main/containers/codespaces-linux
It should probably redirect to this one
https://github.com/devcontainers/images/tree/main/src/universal/

Hi 👋
Good catch, thanks for reporting the bug.
@bamurtaugh wondering, if you have access to update the https://aka.ms/ghcs-default-image link to https://github.com/devcontainers/images/tree/main/src/universal/ ?

Just updated, thanks!


Can't open individual tabular from attribute tabular

fixed by https://github.com/ReMobidyc/ReMobidyc/pull/260


headless chrome in AWS Linux 2

I noticed the update of soft-token. However, my session is saved from before.
This is the error I'm getting.
[ec2-user@ip-172-31-18-118 ~]$ sudo python3 index.py
Traceback (most recent call last):
  File "index.py", line 21, in <module>
    session_path='/home/ec2-user/mintsession',  # Directory that the Chrome persistent session will be written/read from.
  File "/usr/local/lib/python3.7/site-packages/mintapi/api.py", line 529, in __init__
    chromedriver_download_path=chromedriver_download_path)
  File "/usr/local/lib/python3.7/site-packages/mintapi/api.py", line 618, in login_and_get_token
    chromedriver_download_path=chromedriver_download_path)
  File "/usr/local/lib/python3.7/site-packages/mintapi/api.py", line 435, in get_web_driver
    imap_password, imap_server, imap_folder)
  File "/usr/local/lib/python3.7/site-packages/mintapi/api.py", line 410, in _sign_in
    driver.find_element_by_id("ius-sign-in-mfa-password-collection-current-password").send_keys(password)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 479, in send_keys
    'value': keys_to_typing(value)})
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
  (Session info: headless chrome=92.0.4515.131)

I'm using AWS Linux 2.
Thank you

What is index.py? This is not a file in the mintapi repository :)

no, this is the script to run the program.
import mintapi
import datetime
import json
import os

os.system("sudo pkill -f /home/ec2-user/chromedriver")
os.system("sudo pkill -f /opt/google/chrome/chrome")

mint = mintapi.Mint(
        'xxxxx',  # Email used to log in to Mint
        'xxxxx',  # Your password used to log in to mint
        # Optional parameters
        mfa_method='sms',  # Can be 'sms' (default) or 'email'.
        # if mintapi detects an MFA request, it will trigger the requested method
        # and prompt on the command line.
        headless=True,  # Whether the chromedriver should work without opening a
        # visible window (useful for server-side deployments)
        mfa_input_callback=None,  # A callback accepting a single argument (the prompt)
        # which returns the user-inputted 2FA code. By default
        # the default Python `input` function is used.
        session_path='/home/ec2-user/mintsession',  # Directory that the Chrome persistent session will be written/read from.
        # To avoid the 2FA code being asked for multiple times, you can either set
        # this parameter or log in by hand in Chrome under the same user this runs
        # as.
        #imap_account='xxxxx',  # account name used to log in to your IMAP server
        #imap_password=None,  # account password used to log in to your IMAP server
        #imap_server='imap.gmail.com',  # IMAP server host name
        #imap_folder='INBOX',  # IMAP folder that receives MFA email
        #wait_for_sync=True,  # do not wait for accounts to sync
        #wait_for_sync_timeout=30,  # number of seconds to wait for sync
)

budgetw=mint.get_budgets()
#print(budgetw)

data = {}
data['bud'] = []
for i in budgetw['spend']:
        #print(i['cat'])
        #print(i['amt'])
        data['bud'].append({
                'cat': i['cat'],
                'amt': i['amt'],
                'datenow': str(datetime.datetime.now())
    })

with open('/home/ec2-user/data.txt', 'w') as outfile:
        json.dump(data, outfile)

data = {}
data['accw'] = []
accw=mint.get_accounts()
for i in accw:
        data['accw'].append({
                'account_mintid': i['id'],
                'account_nickname': i['accountName'],
                'account_name': i['yodleeName'],
                'account_fourdigits': i['yodleeAccountNumberLast4'],
                'account_type': i['accountType'],
                'account_value': i['value'],
                'account_lastupdate': str(i['lastUpdatedInDate']),
                'account_status': i['status'],
                'account_isclosed': i['isClosed']
        })

with open('/home/ec2-user/json_accounts.txt', 'w') as outfile:
        json.dump(data, outfile)

##06/01/20##
first_month = datetime.datetime.now()
first_month = first_month.strftime("%m/01/%y")
##2020-06##
file_month = datetime.datetime.now()
file_month = file_month.strftime("%Y-%m")

transactions = mint.get_transactions_json(include_investment=False, skip_duplicates=False, start_date = first_month)

with open('/home/ec2-user/transactions/'+ file_month +'.txt', 'w') as outfile:
        json.dump(transactions, outfile)

#print(transactions)

bills = mint.get_bills()
print(bills)

os.system("sudo pkill -f /home/ec2-user/chromedriver")
os.system("sudo pkill -f /opt/google/chrome/chrome")

mint.close()

index.py
sorry for the bad syntax.
versions:
Google Chrome 92.0.4515.131 ChromeDriver 84.0.4147.30 (48b3e868b4cc0aa7e8149519690b6f6949e110a8-refs/branch-heads/4147@{#310})

I opened Lightsail's instance with ubuntu 20, and I reinstall everything.
I removed the two-step-verification
I use mfa_method='sms'
I think the mfa_method='soft-token' method must be with more explanation in the read me file..
Is it possible for the next version of mintapi, to have an option to select which browser to use?
Thank you for your quick response..


Developer defined page not found

API for developers to define custom 404 page not found

Closed by #1485277


Trailing blanks are removed

Data: a
PDF: a

I think that this is not a bug.
Prawn seems to remove a trailing white space on the line below:
https://github.com/prawnpdf/prawn/blob/7f17308e0cabd876fd1ced4412749462c4e968d8/lib/prawn/text/formatted/arranger.rb#L300
https://github.com/prawnpdf/prawn/blob/37303f5adbe2089066a74776369035c2c228d7b4/lib/prawn/text/formatted/fragment.rb#L210-L211


How do we calculate AUC for SASRec?

Hello!
Great work and great paper!
I am working something related to CTR and Sequential Recommendation. However, I am having a hardtime understanding how the models like SASRec are evaluated on AUC score? SASRec is a Sequential Recommendation where the original paper has metrics like HR@k and NDCG@k. But for CTR prediction like TallRec, how do you get AUC score for SASRec and similar Sequential Recommendors.
I tried finding the code for the same. Could you please point me at the piece of code where you calculate AUC for SASRec? Really sorry if I missed.
Thanks and Regards
Millennium Bismay

I’m sorry, I tried to find this part of the code, but it seems to have been lost in the update after the server crash. Specifically, calling ChatGPT through the API returns the generation probabilities of the top 5 tokens. We extract the probabilities of generating “Yes” and “No” and recalculate their probability values for calculating the AUC. On the other hand, if “Yes” or “No” does not appear in the top 5 tokens, we use 1.0 minus the sum of the generation probabilities of the top 5 tokens as a substitute (which is often in the range of 1e-3 to 1e-5).

Thank you for your reply! Thank you for mentioning the technique used for caIculating AUC for ChatGPT. However, I am sorry, I think I might have not been very clear about the question I asked earlier.
I wanted to understand how to calculate the AUC for SASRec.

Thank you very much for your attention to our work.
We conduct testing in the CTR setting, and AUC is the most classic metric for CTR tasks. This is why we have chosen to use AUC as our evaluation metric.
In terms of implementation, we have transformed the activation function of the last layer of all baselines into the commonly used Sigmoid activation function in CTR tasks. This enables these baselines to be applied to CTR tasks. In this case, the output represents the CTR value of the target item for the user.
Specifically, each recommendation example consists of three parts: (seq, target, label). Here, seq represents the historical interactions, target represents the item to be evaluated, and label indicates whether the user likes the target item, with values of 0 and 1.
Taking SASRec as an example, we input the seq into SASRec and obtain the logits for the corresponding target. Finally, by applying the Sigmoid activation function, we obtain the corresponding CTR value. Since we have labels with values of 0 and 1, we can calculate the AUC to measure the performance of the model in the CTR task.

Thank you so much for taking time to provide detailed information. I will start implementing with the same idea. Just a request, if possible could you please share the code over mail? I couldn't find the implementation for SASRec and other baselines in the repo. My email is<EMAIL_ADDRESS>

您好，看了您的回答之后，我依然有些没理解的地方，想请教您。
您是如何定义历史序列的呢？因为对预测目标的分类是喜欢和不喜欢，但在历史序列中并不包含喜欢与不喜欢这样的信息。那您是如何处理的呢？

Likes and Dislikes are derived from ratings. If I am not wrong, TallRec implements like as rating>3 and rest dislikes. I personally thought ratings >= 3 for like. So its on you how you want to model.

谢谢！！但我想我可能没有表达清楚我的意思，您说的这些我在论文中有注意到。我其实想说的是，对于baseline，您是如何将评分信息加入到历史序列的呢，因为在我的理解中，历史序列是一个曾经和用户发生过交互的物品的序列，但这似乎并不能包含评分的信息，而只是表明用户是否看过这个电影（书籍）。

So, the data set has two parts - list of historical movie sequence and list
of like or dislike as 1 or 0. And hour final prediction basically comes
from a sogmoid layer on top of the final layer of SASRec.
On Sat, Apr 13, 2024 at 1:40 AM swordqiqi @.***> wrote:

谢谢！！但我想我可能没有表达清楚我的意思，您说的这些我在论文中有注意到。我其实想说的是，对于baseline，您是如何将评分信息加入到历史序列的呢，因为在我的理解中，历史序列是一个曾经和用户发生过交互的物品的序列，但这似乎并不能包含评分的信息，而只是表明用户是否看过这个电影（书籍）。
—
Reply to this email directly, view it on GitHub
https://github.com/SAI990323/TALLRec/issues/49#issuecomment-2053529814,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFQQJ5H6XHPD7UO2G5TRHS3Y5DHMVAVCNFSM6AAAAABD62T6AKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJTGUZDSOBRGQ
.
You are receiving this because you modified the open/close state.Message
ID: @.***>

我懂了，谢谢！！Thanks a lot！！！！！


update readme pipx install usage

I've updated readme to use pipx instead of pip
since python gives you an error with pip install now
Solves
#635
#594

@lucasliet This is not working on Windows.
You should mention that this issue is related to pip. However, users can try using pipx as an alternative.


Documentation: Migrate to Furo theme

About
I don't know if you are going to stick with the Alabaster theme forever, but looking at WhiteNoise for example, but also many other projects, this patch is giving it a shot for Responder as well to render its documentation using the Furo theme for Sphinx.
Status
This patch does not have any priority, needs additional work, and is just here to probe interests.
Thoughts
What do you think about it, @kennethreitz?

@kennethreitz said:

My only request is keeping alabaster and being cautious with switching to markdown.

In this spirit, let's close this PR again. 🌻


Bump framework version to 5.25.51

Purpose

Please refer the $subject

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/3580306826

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/3580306826
Status: failure

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/3580698388

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/3580698388
Status: failure

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/3581573097

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/3581573097
Status: failure

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/3581587830

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/3581587830
Status: failure

Closing due to this PR

https://github.com/wso2/product-is/pull/15239


Add iterators

[x] Add iterator class
[x] begin() returns an iterator to the first element
[x] end() returns an iterator to the last element

implemented in PR #18


Adds npm-shrinkwrap.json file

Ran npm shrinkwrap, which renamed my local package-lock.json to npm-shrinkwrap.json. Should work, will have to test publish to npm though.
Resolves #614 and resolves #532

Shall we add an entry to the changelog for this? It'll be getting its own release.

I can update the changelog and document usage!

Does it also close https://github.com/SuffolkLITLab/ALKiln/issues/594?

https://github.com/SuffolkLITLab/ALKiln/pull/624 should have closed that, but since we're working on the v5 branch, and v4 is still marked as the default branch, Github doesn't close things when merged. I'll close that one now.

532 is a discussion

that discussion's purpose was to decide on a mechanism to freeze dependencies, and this PR implements the one we decided on.

It'll be getting its own release

We haven't actually released v5 yet, have we? This one isna v5 only too, it won't work with v4.

We haven't actually released v5 yet, have we? This one isna v5 only too, it won't work with v4.

True, but I plan to make a new pre-release with this.

Added to the CHANGELOG and CONTRIBUTING!

Love the new docs! Instead of the ### Added section, should this be in a ### Security section?
I feel like we've covered these before, but I just can't locate the info - if someone just does npm install without changing the package.json, the shrinkwrap will stay the same? Secondly, do we have to edit the package.json? Can we no longer install using npm install --save some-package?

Instead of the ### Added section, should this be in a ### Security section

Happy to put it there, but my understanding was that Security was for things that are specific security fixes (i.e. we found this vulnerability and patched it, etc). This change is related to Security, and it's always good to update, but there's not the urgency of fixing a patch; we can't point to a specific library and say "this had a security flaw in it, this change avoids that", right?

if someone just does npm install without changing the package.json, the shrinkwrap will stay the same?

That's my experience with it. Just to test it, I checked on a different machine, checked out this branch and ran npm install, and nothing in npm-shrinkwrap.json changed. Not sure all the other corner cases to test, but we will stall have control over all of the changes of npm-shrinkwrap.json going into main.

Secondly, do we have to edit the package.json? Can we no longer install using npm install --save some-package?

I wanted to have one simple instruction for adding dependencies that didn't have any ambiguity, so I only documented that one. Extra benefit that the version in the package.json will more explicit and under our control, and not the default semver range operator like --save does apparently.
There are still a ton of ways to change package.json that were confusing to me, like npm rm, npm update, --save, --save-exact, --save-prod, and I couldn't tell what the difference was between each, so I just stuck with the simplest one. Those other steps probably still work fine; you can do whatever, and if you didn't expect to change npm-shrinkwrap.json, but it shows up in the PR diff (or vice versa), then you can always revert just that file. (I was looking at the npm install params that didn't have --save, only realized now that it was an older version of the docs, I thought it was a newer version).

This change is related to Security, and it's always good to update, but there's not the urgency of fixing a patch

Makes sense to me! All the rest are great answers too, thanks and looks great! If I could approve again, I would 🚀


[RFC][move][vm] Versioned deserializer

Threaded version information through the VM's deserializer
It was a bit tricky to do this because the deserializer was so light weight
An alternative option would be to stick the whole thing in some sort of deserializer struct/object and add the environment variable as a version, but that felt weird since there was only one thing in the env...

Motivation

We will need to case on the version information for future work

Test Plan

none yet really, the code is not used yet

Theoretically only the initial dispatch needs to be dynamic, based on the version number read from the beginning of the file format and an alternative pattern to support versioning is to make a modified version of every function on an effect path:

However this comes at the cost of code bloat and the performance gain and slight decrease in complexity in each path probably aren't enough to justify the cost. Based on this I'm starting to like the current proposal.
Maybe when const generics becomes stable in the future we could do something like this:
fn deserialize_module(bytes: &[u8]) -> Result<...> {
    let version = read version number;
    if version == V1 {
        deserialize_module::<V1>(bytes)
    }
    else version == V2 {
        deserialize_module::<V2>(bytes)
    }
}

Theoretically only the initial dispatch needs to be dynamic, based on the version number read from the beginning of the file format and an alternative pattern to support versioning is to make a modified version of every function on an effect path:

However this comes at the cost of code bloat and the performance gain and slight decrease in complexity in each path probably aren't enough to justify the cost. Based on this I'm starting to like the current proposal.
Maybe when const generics becomes stable in the future we could do something like this:
fn deserialize_module(bytes: &[u8]) -> Result<...> {
    let version = read version number;
    if version == V1 {
        deserialize_module::<V1>(bytes)
    }
    else version == V2 {
        deserialize_module::<V2>(bytes)
    }
}

@vgao1996 I did a large refactor based on your suggestions, let me know what you think before I land :)

@vgao1996 I did a large refactor based on your suggestions, let me know what you think before I land :)

/land

/land


[graalvm] Add necessary config for graalvm native-image compilation

Hello,
Following this recommended way from Oracle's Graalvm, this should make it easier for others that want to use sqlite-jdbc in a native-image compiled with GraalVM, this way users don't need to add those configs manually as GraalVM will find it in the classpath if using sqlite-jdbc lib.
This is a sample showing that config works
I was using this lib in other projects to fix those missing configs, but after compiling it in json correctly now it's possible to bring those configs to sqlite-jdbc to make it available to everyone, if this PR get merged, that lib could be deprecated/archived.

I tested this SNAPSHOT with clojure-lsp and it worked as well

Ping @xerial

Thanks! It's great to know sqlite-jdbc works with GraalVM in such a manner.

This adds the configuration unconditionally, at the very least there should be a condition added that org.sqlite.core.NativeDB is reachable (or a more appropriate entrypoint, this one is just what we use): https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/Reflection.md#conditional-configuration
Secondly, this also includes all resources on all operating systems, while only the relevant native library should be packaged. It may even be possible to statically link in the library and not include any of them as resources but that's a whole other story.
We've been running sqlite-jdbc in native-image for a while now with our own configuration which is a fair bit more detailed than this, allbeit as a Feature implementation. That way be can add detailed reachabillity handlers that only add configuration for what is required.
Config options such as allDeclaredMethods should be avoided in my opinion. This registers too much and only causes image bloat as it included both unused methods and reflection metadata for methods/fields that are never accessed anyway.
I'm also curious why the reflection config is needed, as we don't have any.
I can find out if I am allowed to share our configuration setup if you would be interested.


Events module does not load the configuration properly

D.ŁΔŴ at 5/23/2020 7:17:18 PM +00:00

Have a problem with the meta event module. It resets to default every time i start blish, i usually turn of some meta i don't care about.  Is something wrong on my side or is it the module?

REF: https://discordapp.com/channels/531175899588984842/534492012263636992/713832955029618708

Fixed in a previous release.


Ported to C#

Thanks! Interesting.
Will close this as not really relevant for this repo, but feel free to start your own!
Thanks for sharing.

I think the takeaway here is to split the data into a common project that could be re-used in other libraries, and then Crawler-Detect becomes a project that uses that data.

I agree. The value of the work that's been done is in the regexps and this
can easily be using in other languages.
2016-03-16 13:57 GMT+01:00 Alister Bulman<EMAIL_ADDRESS>
I think the takeaway here is to split the data into a common project that
could be re-used in other libraries, and then Crawler-Detect becomes a
project that uses that data.
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/JayBizzle/Crawler-Detect/pull/62#issuecomment-197310135

--
Niels Bosma
+46 76 774 27 25 https://si.gnatu.re/#
<EMAIL_ADDRESS>http://seotoolsforexcel.com
[image: Twitter] https://twitter.com/nielsbosma
[image: LinkedIn] https://linkedin.com/in/bosmaniels
Bosma Interactive AB
Lars Forssellsgata 30
Stockholm
111 52
Sweden
[image: banner] http://seotoolsforexcel.com/

@alister @nielsbosma Yes, that is a great idea. Something I have considered, but not really sure how to go about it. What format should the regex's be presented in? Have you guys got any suggestions?

I'd suggest a single 'master' format - a plain .txt file for the crawlers (one per line), and then a second file for the ignore list. For a release, run a little script to make some .JSON and a .php var_export() of the data that can be included directly as php code. Adding more exporters to the data-repo would be trivial.

It has also been ported to GO here
https://github.com/hangxin1940/crawler_detect

There's a similar project that targets multiple languages and keeps the detection rules in yaml files. You might want to take a look on how they do it.
https://github.com/ua-parser/uap-core
https://raw.githubusercontent.com/ua-parser/uap-core/master/regexes.yaml

@windware thanks for that. Certainly an interesting approach and will definitely look into this further soon 👍

RAW regex output now available here https://github.com/JayBizzle/Crawler-Detect/tree/master/raw


publish to npm

NPM is out of date: https://www.npmjs.com/package/leaflet-spin

Thanks for reporting it !
I've updated the repository.


Move Olivia to CommComm emeriti

I don't have the time nor the motivation to make contributions to the project in my capacity as a member. I'll still be around, especially at the collab summit, and make contributions occasionally, but I don't feel like my member status is justified.
Also, I believe it'd be best if someone else took over the role of champion for the Website Redesign initiative from now on. Volunteers welcome!
Thanks to all of the people here who work on making this community more enjoyable for everyone!

❤you will be missed – excited to see you at the collab summit!

I will miss your perspective Olivia, but also support you doing what you feel is best. Take care!

FWIW, @maddhruv asked to be the new Website Redesign CommComm champion. I'm okay with that, does anyone have anything against it?

We'll miss your regular contributions and informed perspective 💛 Def looking forward to connecting at the summit.

Ah, we discussed this after the live streamed meeting and nominated @chowdhurian and myself to assume co-champion roles... Its mentioned in the PRd meeting minutes: https://github.com/nodejs/website-redesign/pull/35/files#diff-c4baa80c0cd8984563bed3a033b56835R26
@maddhruv, I know you weren't able to attend today, and I don't want anyone to be left out of succession discussions! Lets open an issue in the repo and discuss this there 🙂

@bnb can you remove me from the GitHub team?

Done. Also submitted a PR to remove you from the mailing list.

@bnb already submitted one, but i'll close mine


Tarotcards: hermit's configuration file does not work

No matter how much armor you configure to give, the actual armor value you get is always 10.

this is because for some cards you need to restart the game after changing the config
ill look into a way to refreshing the config without restarting the game soon

Try to exit the game and reopen the game. The config remains unchanged. I have tried many times

please post your config file

please post your config file
here you are  tarotcards-common.txt

Attached with a screenshot that does not work when using this config file to start t

he game

looks like this only happens in 1.18.2, ill fix soon

https://github.com/shiroroku/TarotCards/commit/2c630ff6e1a80fada907a907fe59241fc4c838c9
finally got around to a fix, new version 1.2.5 for 1.18.2 will be out soon


Error: Can't resolve 'source-map-support/register' in 'libs/response-lib.js'

Following the instructions to upgrade to serverless-bundle I get the following error when trying to deploy with serverless deploy
ERROR in /project/libs/response-lib.js
Module not found: Error: Can't resolve 'source-map-support/register' in '/project/libs'
 @ /project/libs/response-lib.js 1:0-37
 @ /project/serverless-project/createCard.js
Unhandled rejection Error: Webpack compilation error, see above
    at _.forEach.compileStats (/project/serverless-project/node_modules/serverless-webpack/lib/compile.js:38:19)
    at arrayEach (/project/serverless-project/node_modules/lodash/lodash.js:516:11)
    at Function.forEach (/project/serverless-project/node_modules/lodash/lodash.js:9342:14)
    at BbPromise.fromCallback.then.stats (/project/serverless-project/node_modules/serverless-webpack/lib/compile.js:31:11)
    at tryCatcher (/project/serverless-project/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/project/serverless-project/node_modules/bluebird/js/release/promise.js:517:31)
    at Promise._settlePromise (/project/serverless-project/node_modules/bluebird/js/release/promise.js:574:18)
    at Promise._settlePromise0 (/project/serverless-project/node_modules/bluebird/js/release/promise.js:619:10)
    at Promise._settlePromises (/project/serverless-project/node_modules/bluebird/js/release/promise.js:699:18)
    at Promise._fulfill (/project/serverless-project/node_modules/bluebird/js/release/promise.js:643:18)
    at /project/serverless-project/node_modules/bluebird/js/release/nodeback.js:42:21
    at finalCallback (/project/serverless-project/node_modules/webpack/lib/MultiCompiler.js:247:12)
    at runWithDependencies.err (/project/serverless-project/node_modules/webpack/lib/MultiCompiler.js:270:6)
    at done (/project/serverless-project/node_modules/neo-async/async.js:2931:13)
    at runCompilers (/project/serverless-project/node_modules/webpack/lib/MultiCompiler.js:174:48)
    at err (/project/serverless-project/node_modules/webpack/lib/MultiCompiler.js:181:7)
    at compiler.run (/project/serverless-project/node_modules/webpack/lib/MultiCompiler.js:263:7)
    at finalCallback (/project/serverless-project/node_modules/webpack/lib/Compiler.js:253:39)
    at hooks.done.callAsync.err (/project/serverless-project/node_modules/webpack/lib/Compiler.js:269:13)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/project/serverless-project/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
    at AsyncSeriesHook.lazyCompileHook (/project/serverless-project/node_modules/tapable/lib/Hook.js:154:20)
    at onCompiled (/project/serverless-project/node_modules/webpack/lib/Compiler.js:267:21)
    at hooks.afterCompile.callAsync.err (/project/serverless-project/node_modules/webpack/lib/Compiler.js:664:15)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/project/serverless-project/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at AsyncSeriesHook.lazyCompileHook (/project/serverless-project/node_modules/tapable/lib/Hook.js:154:20)
    at compilation.seal.err (/project/serverless-project/node_modules/webpack/lib/Compiler.js:661:31)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/project/serverless-project/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at AsyncSeriesHook.lazyCompileHook (/project/serverless-project/node_modules/tapable/lib/Hook.js:154:20)
    at hooks.optimizeAssets.callAsync.err (/project/serverless-project/node_modules/webpack/lib/Compilation.js:1403:35)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/project/serverless-project/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at AsyncSeriesHook.lazyCompileHook (/project/serverless-project/node_modules/tapable/lib/Hook.js:154:20)
    at hooks.optimizeChunkAssets.callAsync.err (/project/serverless-project/node_modules/webpack/lib/Compilation.js:1394:32)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/project/serverless-project/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at AsyncSeriesHook.lazyCompileHook (/project/serverless-project/node_modules/tapable/lib/Hook.js:154:20)
    at hooks.additionalAssets.callAsync.err (/project/serverless-project/node_modules/webpack/lib/Compilation.js:1389:36)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/project/serverless-project/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at AsyncSeriesHook.lazyCompileHook (/project/serverless-project/node_modules/tapable/lib/Hook.js:154:20)
    at hooks.optimizeTree.callAsync.err (/project/serverless-project/node_modules/webpack/lib/Compilation.js:1385:32)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/project/serverless-project/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at AsyncSeriesHook.lazyCompileHook (/project/serverless-project/node_modules/tapable/lib/Hook.js:154:20)
    at Compilation.seal (/project/serverless-project/node_modules/webpack/lib/Compilation.js:1322:27)
    at compilation.finish.err (/project/serverless-project/node_modules/webpack/lib/Compiler.js:658:18)
    at hooks.finishModules.callAsync.err (/project/serverless-project/node_modules/webpack/lib/Compilation.js:1245:4)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/project/serverless-project/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:24:1)
    at AsyncSeriesHook.lazyCompileHook (/project/serverless-project/node_modules/tapable/lib/Hook.js:154:20)
    at Compilation.finish (/project/serverless-project/node_modules/webpack/lib/Compilation.js:1237:28)
    at hooks.make.callAsync.err (/project/serverless-project/node_modules/webpack/lib/Compiler.js:655:17)
    at _err0 (eval at create (/project/serverless-project/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:11:1)
    at _addModuleChain (/project/serverless-project/node_modules/webpack/lib/Compilation.js:1169:12)
    at processModuleDependencies.err (/project/serverless-project/node_modules/webpack/lib/Compilation.js:1081:9)
    at processTicksAndRejections (internal/process/next_tick.js:74:9)

Do you have a repo I can reproduce this with? Or if I can get the package.json and Lambda function code. I'd like to see what it is complaining about.

package.json
{
  "name": "week-trello",
  "version": "1.1.0",
  "description": "A Node.js starter for the Serverless Framework with async/await and unit test support",
  "main": "handler.js",
  "scripts": {
    "test": "serverless-bundle test",
    "deploy": "serverless deploy"
  },
  "author": "",
  "license": "MIT",
  "repository": {
    "type": "git",
    "url": "https://github.com/AnomalyInnovations/serverless-nodejs-starter.git"
  },
  "devDependencies": {
    "aws-sdk": "^2.401.0",
    "serverless-offline": "^3.25.6",
    "serverless-bundle": "^1.2.2"
  },
  "dependencies": {
    "http": "0.0.0",
    "oauth": "^0.9.15",
    "request": "^2.88.0",
    "request-promise": "^4.2.2",
    "url": "^0.11.0",
    "uuid": "^3.3.2"
  }
}

lambda code createCard.js
import requestlib from "request-promise";
import { success, failure } from "../libs/response-lib";

export async function main(event, context) {
  const data = JSON.parse(event.body);

  const options = {
    method: "POST",
    uri: `https://api.trello.com/1/cards`,
    form: {
      name: data.name,
      idList: data.idList,
      idLabels: data.idLabel,
      pos: data.position,
      key: process.env.API_KEY,
      token: process.env.SECRET
    }
  };

  try {
    let result = await requestlib(options);
    return success(result);
  } catch (e) {
    console.log(e);
    return failure({ status: false });
  }
}

Thanks for that. I'll give this a try and let you know.

Hmm I gave it a try and couldn't repro it. But I used the response-lib.js from here https://github.com/AnomalyInnovations/serverless-stack-demo-api/blob/master/libs/response-lib.js.
Maybe yours is different? Can you post that here? And your serverless.yml as well.
Also just to make sure, did you delete your package-lock.json before doing an npm install when migrating to the plugin?

I did delete my package-lock.json before doing npm install
my response-lib.js is the same as yours
export function success(body) {
  return buildResponse(200, body);
}

export function failure(body) {
  return buildResponse(500, body);
}

function buildResponse(statusCode, body) {
  return {
    statusCode: statusCode,
    headers: {
      "Access-Control-Allow-Origin": "*",
      "Access-Control-Allow-Credentials": true
    },
    body: JSON.stringify(body)
  };
}

Here is serverless.yml
# NOTE: update this with your service name
service: week-trello

# Use the serverless-webpack plugin to transpile ES6
plugins:
  - serverless-bundle
  - serverless-offline

package:
  individually: true

provider:
  name: aws
  runtime: nodejs8.10
  stage: prod
  region: us-east-1
  # To load environment variables externally
  # rename env.example to env.yml and uncomment
  # the following line. Also, make sure to not
  # commit your env.yml.
  #
  environment: ${file(env.yml):${self:provider.stage}}

  # 'iamRoleStatements' defines the permission policy for the Lambda function.
  # In this case Lambda functions are granted with permissions to access DynamoDB.
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:DescribeTable
        - dynamodb:Query
        - dynamodb:Scan
        - dynamodb:GetItem
        - dynamodb:PutItem
        - dynamodb:UpdateItem
        - dynamodb:DeleteItem
      Resource: "arn:aws:dynamodb:us-east-1:*:*"

functions:
  createCard:
    handler: createCard.main
    events:
      - http:
          path: trello/card
          method: post
          cors: true
          authorizer: aws_iam
  finishCard:
    handler: finishCard.main
    events:
      - http:
          path: trello/finishCard
          method: post
          cors: true
          authorizer: aws_iam
  getBoards:
    handler: getBoards.main
    events:
      - http:
          path: trello/boards
          method: post
          cors: true
          authorizer: aws_iam
  setupWeek:
    handler: setupWeek.main
    events:
      - http:
          path: trello/setupWeek
          method: post
          cors: true
          authorizer: aws_iam

# Create our resources with separate CloudFormation templates
resources:
  # API Gateway Errors
  - ${file(../resources/api-gateway-errors.yml)}

@jayair is there anything else I can provide to help with this?

@chrisanderss0n Are the paths correct here? In your createCard.js:
import { success, failure } from "../libs/response-lib";

But your createCard.js is in your project root right? Shouldn't this be the path?
import { success, failure } from "./libs/response-lib";

This is how my project is setup:

I have multiple services each with their own serverless.yml, but they all share the libs folder.

@jayair if you could provide me with a place to begin looking I can see if I can figure this out and submit a PR.

@chrisanderss0n I had tried your setup and got it to work with import { success, failure } from "./libs/response-lib"; instead of the path you were using, import { success, failure } from "../libs/response-lib";.
The best way to check this would be to create a simple repo that replicates the error. Would that be possible?

@jayair I'm facing the same problem :(
@chrisanderss0n have you figured this out?
Below is my folder structure. My serverless project is the user folder. One level above (outside the user project) I've got the lib folder with some code I'd like to share with my serverless projects.
api
lib/someCode.js
user/api/handler.js
This is how my handler is importing the code.
`javascript
import { failure, success } from "../../lib/responses";
export const main = async (event, context) => {
//code....
`

@vinyoliver I know in @chrisanderss0n's case I found the path was set incorrectly. I'm not sure what the issue is for your case. Can you make a repo or post the steps to recreate it?

@jayair sorry for the late response. I got back to this issue today. I've created a new repo using the same structure as my project. I still get the same error; Maybe I'm missing something
https://github.com/vinyoliver/error-serverless
Tks in advance!

I've finally figured out! After comparing my project to this example project example I noticed the only difference between them was that my root package.json was missing the serverless-bundle dependency.
After adding serverless-bundle dependency into my root package.json everything worked as expected!
I guess would be helpful to mention this on the README. The project example I got from this post:
https://seed.run/blog/how-to-structure-a-real-world-monorepo-serverless-app. Unfortunately, they do not mention that this dependency is required in the root package.json to make it work.

Ah I see. Thanks for pointing it out.

As @vinyoliver mentioned, the solution is to install serverless as a dev dependency into your project
"yarn add --dev serverless"


Typo: now -> know

This pull request didn't trigger Jenkins as its author isn't in the whitelist.
An organization member must perform one of the following:

To have this branch tested by Jenkins, use the "ok to test" command.
To have a one time test done, use the "test this please" command.

Those commands are simple Github comments of the format: "jenkins: COMMAND"

Good catch. Can you sign-off your commit (Signed-off-by line) as mentioned in CONTRIBUTING.md?
Thanks

Sorry, I doubt I did that right, I've never made a pull request before. Where do I add my sign-off?

In the commit message itself.
If using the command line, the "-s" flag to "git commit" will do it for you.

I think you better do it yourself. I have no idea what I'm doing.


Can't install

Hi. I'm try to run "go get github.com/white-pony/go-fann"
and get an error:
#github.com/white-pony/go-fann
white-pony/go-fann/fann-cascade.go:4:10: fatal error: 'fann.h' file not found
#include <fann.h>
^
1 error generated.

I'm already encounter this problem. But after installed FANN, that error fixed.


modules/github-runner: Improve description of url

We spent a whole afternoon debugging this, because upstream has very
bad software quality and the error messages were incredibly
misleading.
So let’s document it for the sanity of other people.
Btw, I think the implementation of our module is pretty brittle,
especially the part about diffing tokens to check whether they
changed. We should rather just request a new builder registration
every time, it’s not that much overhead, and always set replace so
it is idempotent.
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

You cannot, however, do a registration on each start as the token is rather short-lived (~ 1 hour). To use this approach, we'd need support for PATs. It still wouldn't work for the registration tokens though.

Hm, that’s a good point.
PAT support has been added to the runner end of last year iirc, but you are right that wouldn’t help much for the organization tokens.
Most of this debugging was that the github-runner codebase is truly not very good (I had to read a nontrivial amount of it and work with their horrible logging scheme to figure out why the NotFound error was thrown), and that apparently the (completely undocumented) registration endpoint changed behavior in the meantime.

PAT support has been added to the runner end of last year iirc, but you are right that wouldn’t help much for the organization tokens.

That shouldn't be a problem: using an appropriately scoped PAT, you can query a registration token on a per repository level and also on an organization level. Still, there are good reasons you might not want to deploy a powerful PAT on a machine executing the GitHub Actions runner. Therefore, we should keep supporting the expiring registration tokens. Nevertheless, refactoring/rewriting the current module as well as introducing support for PATs is a good and appreciated idea.

Most of this debugging was that the github-runner codebase is truly not very good (I had to read a nontrivial amount of it and work with their horrible logging scheme to figure out why the NotFound error was thrown), and that apparently the (completely undocumented) registration endpoint changed behavior in the meantime.

What should I say—you're just right here.


PR contribution 291 - refactor modules/logging-center/main.tf hardcoded services list out into env/common tfvars

See example implementation fix in the two for audit and guardrails
https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/pull/291/files#diff-cb9302fb17255576bca335da2817cda2a821b776209c180f929f6527396f89fd
services                                = ["storage.googleapis.com", "logging.googleapis.com", "monitoring.googleapis.com"]

Hardcoded service list needs to move out of modules into a tfvar in environments/common
For an example fix - see #284  - 2 PRs as reference
https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/pull/310/files#diff-acb71cc23936666b11db107a6fe60a986ed76ee48c2150b4e035b7519b4e973dR83
https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/pull/313

20240406: Closing issue during retrofit/rebase of this TEF V1 based/modified repo to TEF V4 standards
This issue may participate in the LZ refactor after rebase
Query on all issues related to the older V1 version via the tag
https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/labels/2024-pre-tef-v4