Cannot compile on Mac OS X due to BoringSSL

System information
@:~/projects/tensorflow <master>$ cat tf_env.txt

== cat /etc/issue ===============================================
Darwin mn-mortutay 16.5.0 Darwin Kernel Version 16.5.0: Fri Mar  3 16:52:33 PST 2017; root:xnu-3789.51.2~3/RELEASE_X86_64 x86_64
Mac OS X 10.12.4

== are we in docker =============================================
No

== compiler =====================================================
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

== uname -a =====================================================
Darwin mn-mortutay 16.5.0 Darwin Kernel Version 16.5.0: Fri Mar  3 16:52:33 PST 2017; root:xnu-3789.51.2~3/RELEASE_X86_64 x86_64

== check pips ===================================================
numpy (1.13.1)
protobuf (3.3.0)
tensorflow (1.2.1)

== check for virtualenv =========================================
False

== tensorflow import ============================================
tf.VERSION = 1.2.1
tf.GIT_VERSION = v1.2.0-5-g435cdfc
tf.COMPILER_VERSION = v1.2.0-5-g435cdfc
Sanity check: array([1], dtype=int32)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import *
  File "tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "tensorflow/python/pywrap_tensorflow.py", line 52, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):
  File "tensorflow/python/pywrap_tensorflow.py", line 41, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
ImportError: No module named pywrap_tensorflow_internal

Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/install_sources#common_installation_problems

for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.

== env ==========================================================
LD_LIBRARY_PATH is unset
DYLD_LIBRARY_PATH is unset

== nvidia-smi ===================================================
tf_env_collect.sh: line 105: nvidia-smi: command not found

== cuda libs  ===================================================

Building from commit:
* d74f65bac (Yun Peng, 3 hours ago) Make Windows Bazel GPU build work again (#11901)

Describe the problem
I'm following the instructions for Mac source installation (https://www.tensorflow.org/install/install_sources, "Prepare environment for Mac OS" and then "Build the pip package") but I'm running into issues with BoringSSL.
It looks like the -Wunused-but-set-parameter and -Wno-free-nonheap-object flags are causing the compilation to fail, since clang does not support them.
Expected behavior: The build system should detect this and handle it correctly, eg. by not using the un-available flags
Actual behavior: Error / fails to build
Source code / logs
@:~/projects/tensorflow <master>$ bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
WARNING: /Users/mortutay/projects/tensorflow/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:exporter': No longer supported. Switch to SavedModel immediately.
WARNING: /Users/mortutay/projects/tensorflow/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:gc': No longer supported. Switch to SavedModel immediately.
INFO: Found 1 target...
INFO: From Compiling external/protobuf_archive/python/google/protobuf/internal/api_implementation.cc:
warning: unknown warning option '-Wunused-but-set-parameter'; did you mean '-Wunused-parameter'? [-Wunknown-warning-option]
warning: unknown warning option '-Wno-free-nonheap-object'; did you mean '-Wno-sequence-point'? [-Wunknown-warning-option]
2 warnings generated.
INFO: From Compiling external/swig/Source/Swig/typemap.c [for host]:
warning: unknown warning option '-Wunused-but-set-parameter'; did you mean '-Wunused-parameter'? [-Wunknown-warning-option]
warning: unknown warning option '-Wno-free-nonheap-object'; did you mean '-Wno-sequence-point'? [-Wunknown-warning-option]
2 warnings generated.
INFO: From Compiling external/grpc/src/core/lib/debug/trace.c:
warning: unknown warning option '-Wunused-but-set-parameter'; did you mean '-Wunused-parameter'? [-Wunknown-warning-option]
warning: unknown warning option '-Wno-free-nonheap-object'; did you mean '-Wno-sequence-point'? [-Wunknown-warning-option]
2 warnings generated.
INFO: From Compiling external/grpc/src/cpp/codegen/codegen_init.cc:
warning: unknown warning option '-Wunused-but-set-parameter'; did you mean '-Wunused-parameter'? [-Wunknown-warning-option]
warning: unknown warning option '-Wno-free-nonheap-object'; did you mean '-Wno-sequence-point'? [-Wunknown-warning-option]
2 warnings generated.
INFO: From Compiling external/grpc/src/core/ext/transport/chttp2/alpn/alpn.c:
warning: unknown warning option '-Wunused-but-set-parameter'; did you mean '-Wunused-parameter'? [-Wunknown-warning-option]
warning: unknown warning option '-Wno-free-nonheap-object'; did you mean '-Wno-sequence-point'? [-Wunknown-warning-option]
2 warnings generated.
ERROR: /private/var/tmp/_bazel_mortutay/dacb21c644505cd819865fa365d2b69e/external/boringssl/BUILD:116:1: C++ compilation of rule '@boringssl//:crypto' failed (Exit 1).
error: unknown warning option '-Wunused-but-set-parameter'; did you mean '-Wunused-parameter'? [-Werror,-Wunknown-warning-option]
error: unknown warning option '-Wno-free-nonheap-object'; did you mean '-Wno-sequence-point'? [-Werror,-Wunknown-warning-option]
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 2.829s, Critical Path: 0.91s

Hi,
I recently had issues compiling BoringSSL because the -Werror complier flag was set in the BoringSSL build. If you error log is complete, it looks like the same warnings that were tolerated in building other parts of the code, were treated as fatal errors in the BoringSSL build.
If the -Werror flag is your problem if would be best to find where it is set. However, I used a brute force solution to get the job done.
find . -type f -not -regex ".*\.\(o\|h\|c\|a\|t\|cc\|so\)" -exec sed -i -e "s/-Werror/-Wno-excessive-errors/g" {} \;

Hope this helps.

@mountaintom Thanks for the analysis!  Your explanation looks spot-on to me.
@av8ramit Do you know who might be a good candidate to follow-up on Mac builds?

Experienced the same issue with boringSSL, dug into it from and found Werror is being set in add_boringssl_s390x.patch on line 71, which is causing the build failure.
+    ":windows": [
+        "-DWIN32_LEAN_AND_MEAN",
+    ],
+    "//conditions:default": [
+        # Assembler option --noexecstack adds .note.GNU-stack to each object to
+        # ensure that binaries can be built with non-executable stack.
+        "-Wa,--noexecstack",
+
+        # This is needed on Linux systems (at least) to get rwlock in pthread.
+        "-D_XOPEN_SOURCE=700",
+
+        # This list of warnings should match those in the top-level CMakeLists.txt.
+        "-Wall",
+        "-Werror",
+        "-Wformat=2",
+        "-Wsign-compare",
+        "-Wmissing-field-initializers",
+        "-Wwrite-strings",
+        "-Wshadow",
+        "-fno-common",
+    ],
+})

-Werror can just be replaced with Wno-excessive-errors, as suggested by @mountaintom , and TF will successfully build. Not sure if this is a sufficient fix as I'm not familiar with the library, but it does bypass the issue for now.

+1, the issue still there when compiling from source in master branch, after used the way of @lissahyacinth , as below:
$ bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package --verbose_failures
WARNING: /Users/sunkai/work/src/tensorflow/tensorflow/core/BUILD:1632:1: in includes attribute of cc_library rule //tensorflow/core:framework_headers_lib: '../../external/nsync/public' resolves to 'external/nsync/public' not below the relative path of its package 'tensorflow/core'. This will be an error in the future. Since this rule was created by the macro 'cc_header_only_library', the error might have been caused by the macro implementation in /Users/sunkai/work/src/tensorflow/tensorflow/core/BUILD:1632:1.
ERROR: /Users/sunkai/work/src/tensorflow/tensorflow/tools/pip_package/BUILD:100:1: no such package '@boringssl//': Traceback (most recent call last):
File "/Users/sunkai/work/src/tensorflow/tensorflow/workspace.bzl", line 119
_apply_patch(repo_ctx, repo_ctx.attr.patch_file)
File "/Users/sunkai/work/src/tensorflow/tensorflow/workspace.bzl", line 110, in _apply_patch
_execute_and_check_ret_code(repo_ctx, cmd)
File "/Users/sunkai/work/src/tensorflow/tensorflow/workspace.bzl", line 94, in _execute_and_check_ret_code
fail("Non-zero return code({1}) when ...))
Non-zero return code(2) when executing 'patch -p1 -d /private/var/tmp/_bazel_sunkai/f32adcfaaeaeed39ed3812ab214ba306/external/boringssl -i /Users/sunkai/work/src/tensorflow/third_party/boringssl/add_boringssl_s390x.patch':
Stdout: patching file src/include/openssl/base.h
patching file BUILD
Stderr: patch: **** malformed patch at line 88:      #"//conditions:default": ["-DOPENSSL_NO_ASM"],
and referenced by '//tensorflow/tools/pip_package:licenses'.
ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package' failed; build aborted.
INFO: Elapsed time: 3.261s
Is there any idea? Thanks.

Is there some update? Looking forward to the response, thanks.

I don't have problem with the way @lissahyacinth provided.
Please take a look at this error:
Stderr: patch: **** malformed patch at line 88: #"//conditions:default": ["-DOPENSSL_NO_ASM"],
Looks like your patch file is not correct.

This seems to be a bug with newer Bazel releases. I started having this with all my C++ projects that use Bazel. They all show these two warnings when compiling C++ files.
It looks like it adds these two flags when compiling with clang (I'm on Linux), even though it shouldn't.
@gunan marking you as you pointed these warnings on the duplicate bug.

@quaeler Could you paste your the version of bazel which works well on mac #12266? As reported by @RenatoUtsch on bazelbuild/bazel#3622 , the warning seems to be brought by new version of bazel.

Yep:
autogenic:tensorflow loki$ bazel version
Build label: 0.5.3-homebrew
Build target: bazel-out/darwin_x86_64-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Sun Aug 6 12:54:10 2017<PHONE_NUMBER>)
Build timestamp:<PHONE_NUMBER>
Build timestamp as int:<PHONE_NUMBER>
autogenic:tensorflow loki$

Interesting, Bazel does not add these flags on my Mac OS, while it adds them to the exact same rule in Linux. Maybe this is related to something else, and not only Bazel's version.

Okay, upon digging a but more this does not seem to be related to bazel's version, but still seems to be a bazel bug. Follow bazelbuild/bazel#3622 if interested

Thanks @RenatoUtsch . I have put my env info on bazelbuild/bazel#3622.

@keithhans thanks for the point.
But, I didn't change the line 88(should be 87 in original add_boringssl_s390x.patch) of #"//conditions:default": ["-DOPENSSL_NO_ASM"].

I find the reason why it didn't take effect by "replacing" -Werror with -Wno-excessive-errors since I wrongly added # to try to comment -Werror instead of replacing. Now it works and thanks.

Where does "-Wunused-but-set-parameter" come from? What's your bazel version? If it is not 0.5.4, could you do an upgrade?
Bazel shouldn't add this arg to a compiler that doesn't support it.

Yeah, removing -Werror is just a hack and doesn't solve the real problem.
I could definitely reproduce bazel adding those two flags wrongly to clang
on 0.5.3. In a few hours I can update and try it again with 0.5.4.
On Mon, Aug 28, 2017, 06:17 Changming Sun<EMAIL_ADDRESS>wrote:

Where does "-Wunused-but-set-parameter" come from? What's your bazel
version? If it is not 0.5.4, could you do an upgrade?
Bazel shouldn't add this arg to a compiler that doesn't support it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/12123#issuecomment-325303387,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAzC7dzgjiAyFdwVS-QJqG3gCEay3zu3ks5scoWugaJpZM4Oxbpe
.

This is fixed with Bazel 0.5.4.
Just upgrade and these warnings won't happen anymore. No need for the -Werror hack too.

Closing based on the latest comments.


get leader vote only after leader persisted

Currently get leader vote before persisted, which may result in data inconsistent if leader crashes.
E.g., 3 nodes: 1,2,3, leader is 1.
Get vote of 2, and 1 not persisted
leader crash-> 2 get voted -> 1 up -> 2 crash -> 1 voted again
The system has two live nodes (1, 3), but the committed log is lost.
Or consider the single quorum scenario.
In current implementation, it will commit even before log persisted, which is against the WAL semantic.

@leisurelyrcxf
I don't quite understand the 3 nodes scenario you described above. Could you please add more details like when logs are appended and what committed logs can be lost.
Please also note that vote related messages won't be sent before logs are persistently stored with fsync().  Please check the engine.go and node.go for details.
For the mentioned single quorum case, yes, the log is marked as committed in memory before being persistently stored, however, a bold however,  (1) client is not notified for such commit event before the log is persistently stored, please see engine.go and node.go for details, (2) not any node will be notified for such commit event. I don't think it violates anything.

btw, if you can email me your WeChat account name, we can talk on WeChat in more details.
My email address is<EMAIL_ADDRESS>

ha. I finally got the logic.
	if ud.FastApply { 		if len(ud.CommittedEntries) > 0 && len(ud.EntriesToSave) > 0 { 			lastApplyIndex := ud.CommittedEntries[len(ud.CommittedEntries)-1].Index 			lastSaveIndex := ud.EntriesToSave[len(ud.EntriesToSave)-1].Index 			firstSaveIndex := ud.EntriesToSave[0].Index 			if lastApplyIndex >= firstSaveIndex && lastApplyIndex <= lastSaveIndex { 				ud.FastApply = false 			} 		} 	}
When see a CommittedEntries before persisted, the check will pass. and FastApply will become false, this prevent apply before persist.

Can close this MR. Thank you!


Custom divIcon for marker endpoint

I have a custom icon which is a divIcon. I can add this to a standard leaflet map with
            L.marker(new L.LatLng(arr[0], arr[1]) {icon: pulseIcon});

When using markerClusterGroup I want to keep the coloured circles with their value but I'd like to override icon at the base (when you can zoom no further). It seems that the only thing you can do is override the default icon L.Icon.Default with an image and corresponding shadow. Is there any way to use a divIcon instead. I tried using
            for (var i = 0; i < addressPoints.length; i++) {
                    var a = addressPoints[i];
                    var title = a[2];
                    var marker = L.marker(new L.LatLng(a[0], a[1]), { title: title }, {icon: pulseIcon});
                    marker.bindPopup(title);
                    markers.addLayer(marker);
            }

            map.addLayer(markers);

where pulseIcon is my divIcon but it's ignored. Is this possible, if not can I add a feature request. Every example I've found only uses the default blue marker. It's also worth noting that even modifying the default icon causes problems because, for example iconSize appears to get ignored.

You'll need to use this:
https://github.com/Leaflet/Leaflet.markercluster#customising-the-clustered-markers
And maybe use the cluster._zoom variable

Thanks for the reply. In the end I actually switched to PruneCluster as it offered this ootb and deadlines are deadlines.


Project restructuring sandbox

This issue covers "Required next work:" from #315

[ ] Some tools/libraries are now quite old (typescript for instance)
[ ] The build has unexpected problems (https://github.com/TypeFox/monaco-languageclient/issues/321)
[x] Do we really need lerna? The only thing being released is the client. Would it make sense to switch to npm workspaces (this options wasn't available in 2018, of course)?
[x] Verify build on every commit: with GitHub Actions. (https://github.com/TypeFox/monaco-languageclient/actions)
[x] Switch dependency of monaco-editor-core tomonaco-editor and make dependency obvious (see #337)

I will start a new branch and test things (sandbox-331).

@CGNonofr We have GitHub Actions! 🎉

To not poluate the root folder with too many subfolders, I would have put everything inside an example folder
packages/client // npm module
packages/example/node // server part
packages/example/client // browser client
packages/example/browser // complete in browser

(Even If I'm not sure about the relevance of the browser example)
I see no problem from mergeing to whatever bundler you want, but is there is reason? is there an issue with webpack?

(Even If I'm not sure about the relevance of the browser example)

Let's not waste what is already there, right?
Currently webpack build is broken on my branch after upgrading versions. vite is just fast and easier to handle, but that's personal preference, of course. We don't need production builds for the examples and here plain node (server) and vite for client/browser related stuff is faster and easier to configure.

@CGNonofr Making good progress now. client itself and examples/node and examples/client have been stripped of unneeded dependencies. Using monaco-editor now in the client example!

Will you do a single PR including all your changes?
What do you expect the work to be done?

Will you do a single PR including all your changes?

I fear yes, because I don't know how to separate the things properly.

What do you expect the work to be done?

Maybe today you can have a first look (draft PR).

Ok, all examples work: https://github.com/TypeFox/monaco-languageclient/tree/sandbox-331
I need to update the README and will then open a draft PR.
If you want to test it yourself (gitpod configuration is already adjusted). Best with separate checkout:
# this needs to be done
npm i
# cleans and compiles everything
npm run build
# start the node LS server embedded or external
npm run start-example-node
npm run start-example-node:ext
# run vite in debug mode for both client based examples
npm run dev

Actually for vite to work the client and browser example don't need to be compiled.
I have created a new project monaco-editor-workers that is used to configure workers (classic or module workers) in the MonacoEnvironment , so no-one needs to think about this stuff. It is applied in the client and browser example.

Resolved with #340


⚠ Samsung has degraded performance

In c4ab8bf, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 3031 ms

Resolved: Samsung performance has improved in 5ffbef3.


Catch failure to change visibility timeout and maintain a counter

Description
Catch failure to change visibility timeout and maintain a counter
Increments a counter sqsVisibilityTimeoutChangeFailedCount when fails to change visibility timeout
Issues Resolved
Resolves #[Issue number to be closed when this PR is merged]
Check List

[ ] New functionality includes testing.
[ ] New functionality has a documentation issue. Please link to it in this PR.

[ ] New functionality has javadoc added

[X ] Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Resolves #3653


Add HighlightMarker plugin

Yes? No? Maybe? :)

Hello! I'm the build bot for the Mono project. I need approval from a Mono team member to build this pull request. A team member should reply with "approve" to approve a build of this pull request, "whitelist" to whitelist this and all future pull requests from this contributor, or "build" to explicitly request a build, even if one has already been done. Contributors can ignore this message.

Closing as not adding in UI controls at this time to plugins list.


-t option not working

Hi, thank you for this script. It has been extremely useful throughout my offsec learning journey!
Today I happened to try the -t (thorough) option and it seemed to not be working. The offending piece of code seem to be line 1327:
while getopts "h:k:r:e:st" option; do
Changing it to
while getopts "h:k:r:e:s:t" option; do
seem to fix it. Please ignore if I'm wrong. Once again, thanks! :)

Hi, thanks for getting in touch!
When you run with the -t switch, do you see the following in the output?
[+] Thorough tests = Enabled
If you save the results (-r) you should also see occurrences of thorough only tests, some of the more common examples include:
grep 'Files owned by our user' <report> grep 'Hidden files' <report>
If you're seeing differing results, can you post an excerpt of the scan and details of the OS on which you're testing?

Hi, I was followed a video tutorial that suggested I use the (-t) option and used the following command
curl http://<IP_ADDRESS>/LinEnum.sh | bash -s -- -t
Previously, I didn't even notice there was this option! However this time I noticed the output says
[+] Thorough tests = Disabled
which did not look like and prompted me to take a look at the code. I just made a guess based on observation that adding the (:) char might fix it and repeated the steps after making the change. The output did change to
[+] Thorough tests = Enabled
I did not save the file with the (-r) option but I did manually copy the output to a file and the results do differ.
Below is the details of the OS (from LinEnum's output):
[-] Kernel information:
Linux hostname 3.10.0-862.3.3.el7.x86_64 #1 SMP Fri Jun 15 04:15:27 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

[-] Kernel information (continued):
Linux version 3.10.0-862.3.3.el7.x86_64<EMAIL_ADDRESS>(gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC) ) #1 SMP Fri Jun 15 04:15:27 UTC 2018

[-] Specific release information:
CentOS Linux release 7.5.1804 (Core)
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

CentOS Linux release 7.5.1804 (Core)

Apologies for the delayed reply.
I just did a quick test using your command and it seems to work without any issues (in this instance the target is an Ubuntu based system).
userxyz@hostxyz:~$ curl http://x.x.x.x/LinEnum.sh | bash -s -- -t
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 45655  100 45655    0     0  3273k      0 --:--:-- --:--:-- --:--:-- 3429k

#########################################################
# Local Linux Enumeration & Privilege Escalation Script #
#########################################################
# www.rebootuser.com
# version 0.97

[-] Debug Info
[+] Thorough tests = Enabled


Unsure how to get this working

Is there .htaccess stuff needed?
Since it does not work - also not throwing an error or anything. Difficult to gauge if it is working.
this is my index.php file /public_hml/index.php
<?php

use Devsrealm\TonicsRouterSystem\Container\Container;
use Devsrealm\TonicsRouterSystem\Events\OnRequestProcess;
use Devsrealm\TonicsRouterSystem\Handler\Router;
use Devsrealm\TonicsRouterSystem\RequestInput;
use Devsrealm\TonicsRouterSystem\Resolver\RouteResolver;
use Devsrealm\TonicsRouterSystem\Response;
use Devsrealm\TonicsRouterSystem\Route;
use Devsrealm\TonicsRouterSystem\RouteNode;
use Devsrealm\TonicsRouterSystem\RouteTreeGenerator;
use Devsrealm\TonicsRouterSystem\State\RouteTreeGeneratorState;

session_start();

// load composer dependencies
require '../vendor/autoload.php';

$onRequestProcess = new OnRequestProcess(
    new RouteResolver(
        new Container()
    ),
    new Route(
        new RouteTreeGenerator(
            new RouteTreeGeneratorState(), new RouteNode()
        )
    )
);

$router = new Router(
    $onRequestProcess,
    $onRequestProcess->getRouteObject(),
    new Response(
        $onRequestProcess, new RequestInput()
    )
);

$route = $router->getRoute();

$route->get('/', function () {
    echo 'Welcome To My Home Page';
});

@ErikThiart Sorry it took so long to reply, yes, you got the wiring up right, there is an issue with your web server settings, have you been able to get it working?


[Enhancement]add pytorch backend support for llama

Added pytorch backend support for llama, based on https://github.com/NVIDIA/FasterTransformer/pull/575.
For a simple test, run under examples/pytorch/llama/:
mpirun -n 4 --allow-run-as-root python llama_example.py --tensor_para_size=4 --pipeline_para_size=1  --ckpt_path path_to_weight --tokenizer_path path_to_tokenizer --lib_path path_to_libth_transformer.so --max_batch_size 1 --start_id_file start_ids.csv

It should print:
[Context]
0, 18637, 29892, 526, 366, 1136, 455, 2470, 29973, 1815, 366, 5193, 304, 592, 29973

[Generated]
[18637 29892   526   366  1136   455  2470 29973  1815   366  5193   304
   592 29973 18637 29892   526   366  1136   455  2470 29973  1815   366
  5193<PHONE_NUMBER>3 18637 29892   526   366  1136   455  2470 29973
  1815   366  5193<PHONE_NUMBER>3 18637 29892   526   366  1136]

[Output]
Hey, are you consciours? Can you talk to me? Hey, are you consciours? Can you talk to me? Hey, are you consciours? Can you talk to me? Hey, are you cons

which meets with the result of its cpp version.

Is there any guide how to create correct gemm_config.in for llama parameters?

Hello, may I ask if this change can support the CPP version of llama?Or does it only support the Pytorch version of llama?

Fantasitic~! I change a bit and test  the performance between ft-llama and huggingface implement. It simply show the ft-llama get 3 times speed up.

FT-LLama: generates 10 batches, taking 4.867 secs to generate 470 tokens, 96.571 tokens/sec.
Huggingface: generates 10 batches, taking 9.923 secs to generate 340 tokens, 34.265 tokens/sec.

However, the output seems different~~~.
The new llama_example.py

# Copyright (c) 2021-2023, NVIDIA CORPORATION.  All rights reserved.
# Copyright (c) 2021, NAVER Corp.  Authored by CLOVA.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# from __future__ import print_function

from torch.nn.utils.rnn import pad_sequence
import os
import sys
import argparse
import configparser
import timeit
import torch
import torch.distributed as dist
from transformers import AutoTokenizer

dir_path = os.path.dirname(os.path.realpath(__file__))
sys.path.append(dir_path + "/../../..")
from examples.pytorch.llama.utils.llama import Llama
from transformers import AutoTokenizer,AutoModelForCausalLM

class Config(object):
    def __init__(self, config_dict):
        for key, val in config_dict.items():
            self.__setattr__(key, val)

def get_model_config(args):
    if "-gpu" not in args.ckpt_path:
        return Config({
            "head_num":32,
            "size_per_head":128,
            "inter_size":11008,
            "vocab_size":32000,
            "layer_num":32,
            "rotary_embedding":128,
            "layernorm_eps":1e-6,
            "start_id":1,
            "end_id":2,
            "use_gptj_residual":False,
            "weight_data_type":"fp16",
        })
    config = configparser.ConfigParser()
    config.read(os.path.join(args.ckpt_path, "config.ini"))
    return Config({
        "head_num":int(config.get('llama', 'head_num')),
        "size_per_head":int(config.get('llama', 'size_per_head')),
        "inter_size":int(config.get('llama', 'inter_size')),
        "vocab_size":int(config.get('llama', 'vocab_size')),
        "layer_num":int(config.get('llama', 'num_layer')),
        "rotary_embedding":int(config.get('llama', 'rotary_embedding')),
        "layernorm_eps":float(config.get('llama', 'layernorm_eps')),
        "start_id":int(config.get('llama', 'start_id')),
        "end_id":int(config.get('llama', 'end_id')),
        "use_gptj_residual":False,
        "weight_data_type":config.get('llama', 'weight_data_type'),
    })

def get_infer_config(args):
    return Config({
        "output_len":args.output_len,
        "beam_width":args.beam_width,
        "top_k":args.top_k,
        "top_p":args.top_p,
        "temperature":args.temperature,
        "len_penalty":args.len_penalty,
        "beam_search_diversity_rate":args.beam_search_diversity_rate,
        "tensor_para_size":args.tensor_para_size,
        "pipeline_para_size":args.pipeline_para_size,
        "max_batch_size":args.max_batch_size,
        "max_seq_len":args.max_seq_len,
        "repetition_penalty":args.repetition_penalty,
        "inference_data_type":args.inference_data_type,
    })

def get_system_config(args):
    return Config({
        "ckpt_path":args.ckpt_path,
        "tokenizer_path":args.tokenizer_path,
        "lib_path":args.lib_path,
    })

def get_model(model_config, infer_config,system_config):
    if "-gpu" in system_config.ckpt_path:
        print('load [fastertransformer] model !')
        model  = Llama(model_config.head_num, model_config.size_per_head, model_config.inter_size, model_config.vocab_size,
                    model_config.rotary_embedding, model_config.layernorm_eps,
                    model_config.start_id, model_config.end_id, model_config.layer_num,
                    infer_config.max_seq_len,
                    infer_config.tensor_para_size,
                    infer_config.pipeline_para_size,
                    model_config.use_gptj_residual,
                    system_config.lib_path,
                    inference_data_type=infer_config.inference_data_type,
                    weights_data_type=model_config.weight_data_type)

        if not model.load(ckpt_path=system_config.ckpt_path):
            print("[WARNING] Checkpoint file not found. Model loading is skipped.")
    else:
        print('load [hugging face] model !')
        model = AutoModelForCausalLM.from_pretrained(system_config.ckpt_path).cuda()
    return model

def get_inputs_ids(args, tokenizer,device):
    # Inputs
    contexts = []
    if args.start_id_file:
        with open(args.start_id_file, 'r') as f:
            contexts = f.read().splitlines()
            batch_size = min(len(contexts), args.max_batch_size)
        contexts = contexts[:batch_size]
        start_ids = [torch.IntTensor([int(i) for i in c.strip().split(',')]) for c in contexts]
    elif args.sample_input_file:  # conditional case
        with open(args.sample_input_file, "r") as f:
            contexts = f.read().splitlines()
            batch_size = min(len(contexts), args.max_batch_size)
        contexts = contexts[:batch_size]
        start_ids = [torch.tensor(tokenizer.encode(c), dtype=torch.int32, device=device) for c in contexts]
    else:  # unconditional case
        raise
        batch_size = infer_config.max_batch_size
        contexts = ['<|endoftext|>'] * batch_size
        start_ids = [torch.IntTensor([model_config.end_id])] * batch_size
    return start_ids, contexts

def get_model_result(model, start_ids,random_seed_tensor,infer_config):
        if isinstance(model, Llama):
            start_lengths = torch.IntTensor([len(ids) for ids in start_ids])
            batch_size    = len(start_ids)
            return model(start_ids    =start_ids,
                        start_lengths=start_lengths,
                        output_len=start_lengths + infer_config.output_len,
                        beam_width=infer_config.beam_width,
                        top_k=infer_config.top_k * torch.ones(size=[batch_size], dtype=torch.int32),
                        top_p=infer_config.top_p * torch.ones(size=[batch_size], dtype=torch.float32),
                        beam_search_diversity_rate=infer_config.beam_search_diversity_rate * torch.ones(size=[batch_size], dtype=torch.float32),
                        temperature=infer_config.temperature * torch.ones(size=[batch_size], dtype=torch.float32),
                        len_penalty=infer_config.len_penalty * torch.ones(size=[batch_size], dtype=torch.float32),
                        repetition_penalty=infer_config.repetition_penalty * torch.ones(size=[batch_size], dtype=torch.float32),
                        random_seed=random_seed_tensor,
                        return_output_length=False,
                        return_cum_log_probs=0)
        else:
            output_ids = model.generate(start_ids,max_length=512)
            return output_ids

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('--output_len', type=int, default=32,
                        help='output sequence length to generate.')
    parser.add_argument('--beam_width', type=int, default=1,
                        help='beam width for beam search. Using sampling when beam width is 1.')
    parser.add_argument('--top_k', type=int, default=1,
                        help='top k candidate num')
    parser.add_argument('--top_p', type=float, default=0.,
                        help='top p probability threshold')
    parser.add_argument('--temperature', type=float, default=1.,
                        help='temperature')
    parser.add_argument('--len_penalty', type=float, default=0.,
                        help='len_penalty')
    parser.add_argument('--beam_search_diversity_rate', type=float, default=0.,
                        help='beam_search_diversity_rate')
    parser.add_argument('--tensor_para_size', type=int, default=1,
                        help='tensor parallel size')
    parser.add_argument('--pipeline_para_size', type=int, default=1,
                        help='pipeline parallel size')
    parser.add_argument('--ckpt_path', type=str,
                        help='path to the checkpoint file.')
    parser.add_argument('--tokenizer_path', type=str,
                        help='directory where the tokenizer file is located.')
    parser.add_argument('--lib_path', type=str, default='./lib/libth_transformer.so',
                        help='path to the pyt_fastertransformer dynamic lib file.')
    parser.add_argument('--sample_input_file', type=str,
                        help='path to the sample input file.')
    parser.add_argument('--start_id_file', type=str,
                        help='path to the start id file.')
    parser.add_argument('--max_batch_size', type=int, default=8,
                        help='max batch size.')
    parser.add_argument('--repetition_penalty', type=float, default=1.,
                        help='repetition penalty')
    parser.add_argument('--max_seq_len', type=int, default=1024,
                        help='max sequence length for position embedding table.')
    parser.add_argument('--inference_data_type', '--data_type', type=str, choices=['fp32', 'fp16'], default='fp16')
    parser.add_argument('--time', action='store_true',
                        help='whether or not to measure time elapsed.')
    parser.add_argument('--enable_random_seed', action='store_true',
                        help='is enable the random seed.')

    args = parser.parse_args()

    print("\n=============== Arguments ===============")
    for arg in vars(args):
        print("{}: {}".format(arg, getattr(args, arg)))
    print("=========================================\n")

    model_config = get_model_config(args)

    #### resource_configuration
    system_config= get_system_config(args)

    #### inference configuration
    infer_config = get_infer_config(args)

    #### set the multiprocess group
    if infer_config.tensor_para_size * infer_config.pipeline_para_size > 1:
        dist.init_process_group(backend=dist.Backend.MPI)
    rank         = dist.get_rank() if dist.is_initialized() else 0
    device_count = dist.get_world_size() if dist.is_initialized() else 1
    device       = rank % device_count
    torch.cuda.set_device(device)
    device       = torch.cuda.current_device()

    # sentencepiece needed
    tokenizer = AutoTokenizer.from_pretrained(system_config.tokenizer_path, use_fast=False)

    # get ids
    start_ids, contexts = get_inputs_ids(args, tokenizer,device)
    batch_size= len(start_ids)
    print("[INFO] batch size: {}".format(batch_size))
    start_ids     = pad_sequence(start_ids, batch_first=True, padding_value=model_config.end_id).cuda()

    if args.enable_random_seed == True:
        random_seed_tensor = torch.randint(0, 10000, size=[batch_size], dtype=torch.int64)
    else:
        random_seed_tensor = torch.zeros([batch_size], dtype=torch.int64)

    # Prepare model.
    print("building model...............")
    llama = get_model(model_config, infer_config, system_config)
    print("done!")
    with torch.no_grad():
        print(f"[INFO] input size {start_ids.shape}")
        tokens_batch = get_model_result(llama, start_ids,random_seed_tensor,infer_config)
        print(f"[INFO] output size {tokens_batch.shape}")
        if tokens_batch is not None and rank == 0:
            tokens_batch = tokens_batch.cpu().numpy()
            if not isinstance(llama, Llama): tokens_batch = [tokens_batch]
            start_lengths = torch.IntTensor([len(ids) for ids in start_ids])
            for i, (context, tokens) in enumerate(zip(contexts, tokens_batch)):
                for beam_id in range(infer_config.beam_width):
                    token = tokens[beam_id][start_lengths[i]:]  # exclude context input from the output
                    output = tokenizer.decode(token)
                    print(f'[INFO] batch {i}, beam {beam_id}:\n[Context]\n{context}\n\n[Generated]\n{token}\n\n[Output]\n{output}\n')

        # Measure inference time.
        if args.time:
            iterations = 10
            # warmup
            for i in range(iterations):
                tokens_batch = get_model_result(llama, start_ids,random_seed_tensor,infer_config)
            batch_num = 0
            token_num = 0
            time = timeit.default_timer()
            for i in range(iterations):
                tokens_batch = get_model_result(llama, start_ids,random_seed_tensor,infer_config)
                batch_num += 1
                for j, tokens in enumerate(tokens_batch):
                    token_num += tokens.shape[-1] - start_lengths[j]
            time_elapsed = timeit.default_timer() - time
            throughput = token_num / time_elapsed
            print(f"[INFO] FT-LLAMA:{args.ckpt_path}:\n      generates {batch_num} batches, taking {time_elapsed:0.3f} secs "
                  f"to generate {token_num} tokens, {throughput:0.3f} tokens/sec.")

if __name__ == '__main__':
    main()

And use the commend
python llama_example.py --tensor_para_size=1 --pipeline_para_size=1  --ckpt_path ~/pretrain_weights/vicuna/vicuna-7b-v1.1/ --tokenizer_path ~/pretrain_weights/vicuna/vicuna-7b-v1.1/ --lib_path ~/projects/FasterTransformer2/FasterTransformer/build/lib/libth_transformer.so --max_batch_size 1 --start_id_file start_ids.csv --time

python llama_example.py --tensor_para_size=1 --pipeline_para_size=1  --ckpt_path ~/pretrain_weights/vicuna/vicuna-7b-fastertransformer_fp16/1-gpu/ --tokenizer_path ~/pretrain_weights/vicuna/vicuna-7b-v1.1/ --lib_path ~/projects/FasterTransformer2/FasterTransformer/build/lib/libth_transformer.so --max_batch_size 1 --start_id_file start_ids.csv --time

The result should be

=============== Arguments ===============
output_len: 32
beam_width: 1
top_k: 1
top_p: 0.0
temperature: 1.0
len_penalty: 0.0
beam_search_diversity_rate: 0.0
tensor_para_size: 1
pipeline_para_size: 1
ckpt_path: /mnt/lustre/zhangtianning/pretrain_weights/vicuna/vicuna-7b-v1.1/
tokenizer_path: /mnt/lustre/zhangtianning/pretrain_weights/vicuna/vicuna-7b-v1.1/
lib_path: /mnt/lustre/zhangtianning/projects/FasterTransformer2/FasterTransformer/build/lib/libth_transformer.so
sample_input_file: None
start_id_file: start_ids.csv
max_batch_size: 1
repetition_penalty: 1.0
max_seq_len: 1024
inference_data_type: fp16
time: True
enable_random_seed: False
=========================================

[INFO] batch size: 1
building model...............
load [hugging face] model !

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:13<00:13, 13.31s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:18<00:00,  8.55s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:18<00:00,  9.27s/it]
done!
[INFO] input size torch.Size([1, 15])
[INFO] output size torch.Size([1, 49])
[INFO] batch 0, beam 0:
[Context]
0, 18637, 29892, 526, 366, 1136, 455, 2470, 29973, 1815, 366, 5193, 304, 592, 29973

[Generated]
[   13 29902 29915 29885  7423 29892   306 29915 29885   451  9985   411
   278  1840   376  3200   455  2470  1213  6527   366  3113  3867   901
  3030   470  5649   825<PHONE_NUMBER>1   304 29973     2]

[Output]

I'm sorry, I'm not familiar with the term "consciours." Could you please provide more context or explain what you are referring to?</s>

[INFO] FT-LLAMA:/mnt/lustre/zhangtianning/pretrain_weights/vicuna/vicuna-7b-v1.1/:
      generates 10 batches, taking 9.923 secs to generate 340 tokens, 34.265 tokens/sec.
[WARNING] gemm_config.in is not found; using default GEMM algo
[FT][WARNING] Skip NCCL initialization since requested tensor/pipeline parallel sizes are equals to 1.

=============== Arguments ===============
output_len: 32
beam_width: 1
top_k: 1
top_p: 0.0
temperature: 1.0
len_penalty: 0.0
beam_search_diversity_rate: 0.0
tensor_para_size: 1
pipeline_para_size: 1
ckpt_path: /mnt/lustre/zhangtianning/pretrain_weights/vicuna/vicuna-7b-fastertransformer_fp16/1-gpu/
tokenizer_path: /mnt/lustre/zhangtianning/pretrain_weights/vicuna/vicuna-7b-v1.1/
lib_path: /mnt/lustre/zhangtianning/projects/FasterTransformer2/FasterTransformer/build/lib/libth_transformer.so
sample_input_file: None
start_id_file: start_ids.csv
max_batch_size: 1
repetition_penalty: 1.0
max_seq_len: 1024
inference_data_type: fp16
time: True
enable_random_seed: False
=========================================

[INFO] batch size: 1
building model...............
load [fastertransformer] model !
[INFO] WARNING: Have initialized the process group
done!
[INFO] input size torch.Size([1, 15])
[INFO] output size torch.Size([1, 1, 62])
[INFO] batch 0, beam 0:
[Context]
0, 18637, 29892, 526, 366, 1136, 455, 2470, 29973, 1815, 366, 5193, 304, 592, 29973

[Generated]
[   13    13 29930 18637 29892   526   366  1136   455  2470 29973  1815
   366  5193<PHONE_NUMBER>3    13 29930 18637 29892   526   366  1136
   455  2470 29973  1815   366  5193<PHONE_NUMBER>3    13 29930 18637
 29892   526   366  1136   455  2470 29973  1815   366  5193   304]

[Output]

* Hey, are you consciours? Can you talk to me?
* Hey, are you consciours? Can you talk to me?
* Hey, are you consciours? Can you talk to

[INFO] FT-LLAMA:/mnt/lustre/zhangtianning/pretrain_weights/vicuna/vicuna-7b-fastertransformer_fp16/1-gpu/:
      generates 10 batches, taking 4.867 secs to generate 470 tokens, 96.571 tokens/sec.

It seem only the max_batch_size=1 can work. For any batch input with batch_size>1, it fails
Traceback (most recent call last):
  File "/mnt/petrelfs/zhangtianning/projects/FasterTransformer2/FasterTransformer/examples/pytorch/llama/llama_example.py", line 233, in <module>
    main()
  File "/mnt/petrelfs/zhangtianning/projects/FasterTransformer2/FasterTransformer/examples/pytorch/llama/llama_example.py", line 163, in main
    tokens_batch = llama(
  File "/mnt/cache/zhangtianning/anaconda3/envs/llm2/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/mnt/petrelfs/zhangtianning/projects/FasterTransformer2/FasterTransformer/examples/pytorch/llama/../../../examples/pytorch/llama/utils/llama.py", line 283, in forward
    outputs = self.model.forward(input_ids,
RuntimeError: forward() Expected a value of type 'int' for argument '_3' but instead found type 'Tensor'.
Position: 3
Value: tensor([47, 47], dtype=torch.int32)
Declaration: forward(__torch__.torch.classes.FasterTransformer.LlamaOp _0, Tensor _1, Tensor _2, int _3, int? _4, Tensor? _5, Tensor? _6, Tensor? _7, Tensor? _8, Tensor? _9, Tensor? _10, Tensor? _11, int? _12) -> (Tensor[] _0)
Cast error details: Unable to cast Python instance to C++ type (compile in debug mode for details)

@veya2ztn
thx for your job
the above llama_example.py script seem to have the flowing bugs:
Traceback (most recent call last):
File "llama_example.py", line 277, in
main()
File "llama_example.py", line 225, in main
tokenizer = AutoTokenizer.from_pretrained(system_config.tokenizer_path, use_fast=False)
File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 690, in from_pretrained
raise ValueError(
ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported.

@veya2ztn thx for your job the above llama_example.py script seem to have the flowing bugs:
Traceback (most recent call last): File "llama_example.py", line 277, in main() File "llama_example.py", line 225, in main tokenizer = AutoTokenizer.from_pretrained(system_config.tokenizer_path, use_fast=False) File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 690, in from_pretrained raise ValueError( ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported.

upgrade your transformer
see from transformers import AutoTokenizer

@veya2ztn thx for your job the above llama_example.py script seem to have the flowing bugs:
Traceback (most recent call last): File "llama_example.py", line 277, in main() File "llama_example.py", line 225, in main tokenizer = AutoTokenizer.from_pretrained(system_config.tokenizer_path, use_fast=False) File "/opt/conda/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 690, in from_pretrained raise ValueError( ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported.

upgrade your transformer see from transformers import AutoTokenizer

@veya2ztn
Thx for your reply. Your provided script did work,however when bs sets to be greater than 1, error occurs, set the output_len to a fixed value could avoid the problem, but each output is fixed length, it need a lot of paddings, this is counterintuitive, isn't it ? The output should end when meets eos.

Same question.

May I ask if it supports single process multiple GPUs. I also have the same confusion that the length of each output is fixed.

May I ask if it supports si

also fail for the multiple GPUs example.

@veya2ztn I have some probably stupid questions, how to compile it?
First, I build Pytorch 2.0.1 with MPI backend from source.

First, I build Pytorch 2.0.1 with MPI backend from source.
Then, I follow the guide docs/decoder_guide.md to compile FT without docker, run the example and got error:
CUDA Error: (null) .../FasterTransformer/3rdparty/trt_fused_multihead_attention/fused_multihead_attention.h 345

I come across this issue #177 , did you also compiled with docker?

I dont compile in docker.  If you want to compile a MPI backbone from source,
Try:
torch11.2 + cuda 11.3 +gcc 7.5
I compile successfully under this configuration. But it seems that the MPI for CUDA is not avaliable.
Let me know if you get success

@veya2ztn I successfully compiled in docker and achieved similar performance as you mentioned, thanks a lot!

Thank you for your great work, but when I test 13B, 30B, 60B models, the following error occurs：
[FT][ERROR] CUDA runtime error: an illegal memory access was encountered /workspace/Llama-FT/FasterTransformer/src/fastertransformer/utils/allocator.h:462
You can reproduce the problem with the following command:
convert model:
python ../examples/cpp/llama/huggingface_llama_convert.py      -saved_dir=./llama-13b-hf/c-model      -in_file=./llama-13b-hf      -infer_gpu_num=2      -weight_data_type=fp16      -model_name=llama_13b
run model:
export CUDA_LAUNCH_BLOCKING=1 mpirun -n 2 --allow-run-as-root python ../examples/pytorch/llama/llama_example.py      --tensor_para_size=2     --pipeline_para_size=1       --ckpt_path ./llama-13b-hf/c-model/2-gpu      --tokenizer_path ./llama-13b-hf      --lib_path ./lib/libth_transformer.so      --max_batch_size 4     --inference_data_type fp16      --output_len 170      --time      --start_id_file ../examples/pytorch/llama/start_ids.csv

usually, its mean OOM
获取Outlook for Androidhttps://aka.ms/AAb9ysg

From: BasicCoder @.>
Sent: Tuesday, June 20, 2023 3:20:07 PM
To: NVIDIA/FasterTransformer @.>
Cc: veya2ztn @.>; Mention @.>
Subject: Re: [NVIDIA/FasterTransformer] [Enhancement]add pytorch backend support for llama (PR #611)
Thank you for your great work, but when I test 13B, 30B, 60B models, the following error occurs：
[FT][ERROR] CUDA runtime error: an illegal memory access was encountered /workspace/Llama-FT/FasterTransformer/src/fastertransformer/utils/allocator.h:462
You can reproduce the problem with the following command:
convert model:
python ../examples/cpp/llama/huggingface_llama_convert.py -saved_dir=./llama-13b-hf/c-model -in_file=./llama-13b-hf -infer_gpu_num=2 -weight_data_type=fp16 -model_name=llama_13b
run model:
export CUDA_LAUNCH_BLOCKING=1 mpirun -n 2 --allow-run-as-root python ../examples/pytorch/llama/llama_example.py --tensor_para_size=2 --pipeline_para_size=1 --ckpt_path ./llama-13b-hf/c-model/2-gpu --tokenizer_path ./llama-13b-hf --lib_path ./lib/libth_transformer.so --max_batch_size 4 --inference_data_type fp16 --output_len 170 --time --start_id_file ../examples/pytorch/llama/start_ids.csv
―
Reply to this email directly, view it on GitHubhttps://github.com/NVIDIA/FasterTransformer/pull/611#issuecomment-1598248181, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEKHE2KJHTPKCRCLACH5E7LXMFFKPANCNFSM6AAAAAAYDJFH4A.
You are receiving this because you were mentioned.Message ID: @.***>

I don't think it's OOM，In this test, I am using 2* A100 80G in tensors parallel. In the same environment and tensors parallel configuration, the result of the 7B model is normal, but a larger model size will cause errors.
I used export CUDA_LAUNCH_BLOCKING=1, the error line number information is:
check_cuda_error(cudaMemset(ptr, val, size));
so i think it’s a cuda error.

I don't think it's OOM，In this test, I am using 2* A100 80G in tensors parallel. In the same environment and tensors parallel configuration, the result of the 7B model is normal, but a larger model size will cause errors. I used export CUDA_LAUNCH_BLOCKING=1, the error line number information is: check_cuda_error(cudaMemset(ptr, val, size)); so i think it’s a cuda error.

Have you solved the problem？

I don't think it's OOM，In this test, I am using 2* A100 80G in tensors parallel. In the same environment and tensors parallel configuration, the result of the 7B model is normal, but a larger model size will cause errors. I used export CUDA_LAUNCH_BLOCKING=1, the error line number information is: check_cuda_error(cudaMemset(ptr, val, size)); so i think it’s a cuda error.

Have you solved the problem？

NO. I haven't resolved it yet.

This is probably caused by pytorch. I solved this problem by rebuild the pytorch with mpi from source

I don't think it's OOM，In this test, I am using 2* A100 80G in tensors parallel. In the same environment and tensors parallel configuration, the result of the 7B model is normal, but a larger model size will cause errors. I used export CUDA_LAUNCH_BLOCKING=1, the error line number information is: check_cuda_error(cudaMemset(ptr, val, size)); so i think it’s a cuda error.

Have you solved the problem？

NO. I haven't resolved it yet.

This is probably caused by pytorch. I solved this problem by rebuild the pytorch with mpi from source

I don't think it's OOM，In this test, I am using 2* A100 80G in tensors parallel. In the same environment and tensors parallel configuration, the result of the 7B model is normal, but a larger model size will cause errors. I used export CUDA_LAUNCH_BLOCKING=1, the error line number information is: check_cuda_error(cudaMemset(ptr, val, size)); so i think it’s a cuda error.

Have you solved the problem？

NO. I haven't resolved it yet.

This is probably caused by pytorch. I solved this problem by rebuild the pytorch with mpi from source

Good news, can you share more information about your execution environment? pytorch/mpi/cuda version?

I don't think it's OOM，In this test, I am using 2* A100 80G in tensors parallel. In the same environment and tensors parallel configuration, the result of the 7B model is normal, but a larger model size will cause errors. I used export CUDA_LAUNCH_BLOCKING=1, the error line number information is: check_cuda_error(cudaMemset(ptr, val, size)); so i think it’s a cuda error.

Have you solved the problem？

NO. I haven't resolved it yet.

This is probably caused by pytorch. I solved this problem by rebuild the pytorch with mpi from source

Good news, can you share more information about your execution environment? pytorch/mpi/cuda version?
host:
driver 525.116.03
cuda 12.0

docker container:
driver 525.116.03
cuda 11.7
openmpi 4.1.2
cdunn 11.x
magma 117
pytorch build from source

I don't think it's OOM，In this test, I am using 2* A100 80G in tensors parallel. In the same environment and tensors parallel configuration, the result of the 7B model is normal, but a larger model size will cause errors. I used export CUDA_LAUNCH_BLOCKING=1, the error line number information is: check_cuda_error(cudaMemset(ptr, val, size)); so i think it’s a cuda error.

Have you solved the problem？

NO. I haven't resolved it yet.

This is probably caused by pytorch. I solved this problem by rebuild the pytorch with mpi from source

Good news, can you share more information about your execution environment? pytorch/mpi/cuda version?

host:
driver 525.116.03
cuda 12.0
docker container:
driver 525.116.03
cuda 11.7
openmpi 4.1.2
cdunn 11.x
magma 117
pytorch build from source

hello what role does the mpi play in the cli? why does pytorch example need mpi?

Hi, @vitrun , the upstream branch by @void-main now support int8 inference.  Would you consider support it also?

Update: Got it working with batches by lowering output_len to 512.

@vitrun Thanks for your work. I m trying to use your code to run llama 2 13b model on my v100-32G, 1-gpu works well for me, but when i try 2-gpu, i got this error:
[INFO] batch size: 1
[INFO] batch size: 1
[INFO] WARNING: Have initialized the process group
[INFO] WARNING: Have initialized the process group
[WARNING] gemm_config.in is not found; using default GEMM algo
[WARNING] gemm_config.in is not found; using default GEMM algo
[FT][INFO] NCCL initialized rank=1 world_size=2 tensor_para=NcclParam[rank=1, world_size=2, nccl_comm=0x5631dea2d0a0] pipeline_para=NcclParam[rank=0, world_size=1, nccl_comm=0x5631dea80c70]
[FT][INFO] NCCL initialized rank=0 world_size=2 tensor_para=NcclParam[rank=0, world_size=2, nccl_comm=0x5592093795d0] pipeline_para=NcclParam[rank=0, world_size=1, nccl_comm=0x5592093cd030]
[FT][ERROR] CUDA runtime error: an illegal memory access was encountered /mnt/work/llama_ft_vitrun/FasterTransformer/src/fastertransformer/utils/allocator.h:462
[FT][ERROR] CUDA runtime error: an illegal memory access was encountered /mnt/work/llama_ft_vitrun/FasterTransformer/src/fastertransformer/utils/allocator.h:462
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[58456,1],0]
  Exit code:    255
--------------------------------------------------------------------------

here is my command line
mpirun -n 2 --allow-run-as-root python llama_example.py \
	--ckpt_path ${ckpt_path} \
	--lib_path ${lib_path} \
	--tokenizer_path ${tokenizer_name_or_path} \
	--tensor_para_size=2 --pipeline_para_size=1 --max_batch_size 1 --start_id_file start_ids.csv

any help will be grateful.

I don't think it's OOM，In this test, I am using 2* A100 80G in tensors parallel. In the same environment and tensors parallel configuration, the result of the 7B model is normal, but a larger model size will cause errors. I used export CUDA_LAUNCH_BLOCKING=1, the error line number information is: check_cuda_error(cudaMemset(ptr, val, size)); so i think it’s a cuda error.

hello  have you solved this problem? illegal memory access error.


feat: [FTL-8015] - Moved projects data to state

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add table aws_vpc_security_group_rules

Integration test logs

Logs
$node --version
v16.13.2

$ node tint.js
No env file present for the current environment:  staging
 Falling back to .env config
No env file present for the current environment:  staging
customEnv TURBOT_TEST_EXPECTED_TIMEOUT undefined

SETUP: tests/aws_vpc_security_group_rules []

PRETEST: tests/aws_vpc_security_group_rules

TEST: tests/aws_vpc_security_group_rules
Running terraform

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_security_group.named_test_resource will be created
  + resource "aws_security_group" "named_test_resource" {
      + arn                    = (known after apply)
      + description            = "Test Security Group."
      + egress                 = (known after apply)
      + id                     = (known after apply)
      + ingress                = (known after apply)
      + name                   = "turbottest18134"
      + name_prefix            = (known after apply)
      + owner_id               = (known after apply)
      + revoke_rules_on_delete = false
      + tags                   = {
          + "name" = "turbottest18134"
        }
      + tags_all               = {
          + "name" = "turbottest18134"
        }
      + vpc_id                 = (known after apply)
    }

  # aws_security_group_rule.ingress will be created
  + resource "aws_security_group_rule" "ingress" {
      + cidr_blocks              = [
          + "<IP_ADDRESS>/16",
        ]
      + description              = "TLS from VPC"
      + from_port                = 443
      + id                       = (known after apply)
      + protocol                 = "tcp"
      + security_group_id        = (known after apply)
      + self                     = false
      + source_security_group_id = (known after apply)
      + to_port                  = 443
      + type                     = "ingress"
    }

  # aws_vpc.main will be created
  + resource "aws_vpc" "main" {
      + arn                                  = (known after apply)
      + cidr_block                           = "<IP_ADDRESS>/16"
      + default_network_acl_id               = (known after apply)
      + default_route_table_id               = (known after apply)
      + default_security_group_id            = (known after apply)
      + dhcp_options_id                      = (known after apply)
      + enable_classiclink                   = (known after apply)
      + enable_classiclink_dns_support       = (known after apply)
      + enable_dns_hostnames                 = (known after apply)
      + enable_dns_support                   = true
      + id                                   = (known after apply)
      + instance_tenancy                     = "default"
      + ipv6_association_id                  = (known after apply)
      + ipv6_cidr_block                      = (known after apply)
      + ipv6_cidr_block_network_border_group = (known after apply)
      + main_route_table_id                  = (known after apply)
      + owner_id                             = (known after apply)
      + tags                                 = {
          + "name" = "turbottest18134"
        }
      + tags_all                             = {
          + "name" = "turbottest18134"
        }
    }

Plan: 3 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + account_id     = "111111111111"
  + ingress_sgr_id = (known after apply)
  + resource_aka   = (known after apply)
  + resource_id    = (known after apply)
  + resource_name  = "turbottest18134"
  + vpc_id         = (known after apply)
aws_vpc.main: Creating...
aws_vpc.main: Still creating... [10s elapsed]
aws_vpc.main: Creation complete after 10s [id=vpc-0fdbd8db678c3b7e7]
aws_security_group.named_test_resource: Creating...
aws_security_group.named_test_resource: Creation complete after 7s [id=sg-00056b3f3b3e3920a]
aws_security_group_rule.ingress: Creating...
aws_security_group_rule.ingress: Creation complete after 3s [id=sgrule-1626592594]

Warning: Deprecated Resource

  with data.null_data_source.resource,
  on variables.tf line 45, in data "null_data_source" "resource":
  45: data "null_data_source" "resource" {

The null_data_source was historically used to construct intermediate values
to re-use elsewhere in configuration, the same can now be achieved using
locals

(and one more similar warning elsewhere)

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Outputs:

account_id = "111111111111"
ingress_sgr_id = "sgrule-1626592594"
resource_aka = "arn:aws:ec2:us-east-1:111111111111:security-group/sg-00056b3f3b3e3920a"
resource_id = "sg-00056b3f3b3e3920a"
resource_name = "turbottest18134"
vpc_id = "vpc-0fdbd8db678c3b7e7"
Template Error: undefined
The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received an instance of Object

ERROR DETECTED: Stopping test run and entering teardown phase.
The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received an instance of Object

TEARDOWN: tests/aws_vpc_security_group_rules

SUMMARY:

1/1 passed.

Example query results

Results
> select group_id, security_group_rule_id, from_port, to_port, ip_protocol, description, cidr_ipv4, arn, title  from aws_vpc_security_group_rules
+----------------------+------------------------+-----------+---------+-------------+-------------+-------------------+-----------------------------------------------------------------------------------+------------------------------------------------------------+
| group_id             | security_group_rule_id | from_port | to_port | ip_protocol | description | cidr_ipv4         | arn                                                                               | title                                                      |
+----------------------+------------------------+-----------+---------+-------------+-------------+-------------------+-----------------------------------------------------------------------------------+------------------------------------------------------------+
| sg-00000000000000000 | sgr-02e4ca1ee06752c83  | -1        | -1      | icmp        | <null>      | <IP_ADDRESS>/0         | arn:aws:ec2:ap-northeast-2:111111111111:security-group-rule/sgr-02e4ca1ee06752c83 | sgr-02e4ca1ee06752c83_egress_icmp_-1_-1_<IP_ADDRESS>/0          |
| sg-00000000000000000 | sgr-0714c4511e7ae422d  | -1        | -1      | -1          | <null>      | <IP_ADDRESS>/0         | arn:aws:ec2:ap-northeast-2:111111111111:security-group-rule/sgr-0714c4511e7ae422d | sgr-0714c4511e7ae422d_ingress_-1_-1_-1_<IP_ADDRESS>/0           |
| sg-01111111111111111 | sgr-02485a92cd6ff6c15  | -1        | -1      | -1          | <null>      | <IP_ADDRESS>/0         | arn:aws:ec2:ap-northeast-2:111111111111:security-group-rule/sgr-02485a92cd6ff6c15 | sgr-02485a92cd6ff6c15_ingress_-1_-1_-1_<IP_ADDRESS>/0           |
| sg-00000000000000000 | sgr-0ad7ff1d9d627d82f  | -1        | -1      | -1          | <null>      | <null>            | arn:aws:ec2:ap-northeast-2:111111111111:security-group-rule/sgr-0ad7ff1d9d627d82f | sgr-0ad7ff1d9d627d82f_egress_-1_-1_-1_::/0                 |
| sg-02222222222222222 | sgr-0c65a372537609bdb  | -1        | -1      | -1          | <null>      | <IP_ADDRESS>/32        | arn:aws:ec2:ap-northeast-2:111111111111:security-group-rule/sgr-0c65a372537609bdb | sgr-0c65a372537609bdb_egress_-1_-1_-1_<IP_ADDRESS>/32    |
| sg-00000000000000000 | sgr-050992310bfc7508c  | -1        | -1      | icmpv6      | <null>      | <null>            | arn:aws:ec2:ap-northeast-2:111111111111:security-group-rule/sgr-050992310bfc7508c | sgr-050992310bfc7508c_egress_icmpv6_-1_-1_::/0             |
| sg-02222222222222222 | sgr-0214fd8884a465a07  | -1        | -1      | -1          | <null>      | <IP_ADDRESS>/0         | arn:aws:ec2:ap-northeast-2:111111111111:security-group-rule/sgr-0214fd8884a465a07 | sgr-0214fd8884a465a07_egress_-1_-1_-1_<IP_ADDRESS>/0            |
| sg-01111111111111111 | sgr-0eee13951944c6b2d  | 3389      | 3389    | tcp         | asdf        | <IP_ADDRESS>/32        | arn:aws:ec2:ap-northeast-2:111111111111:security-group-rule/sgr-0eee13951944c6b2d | sgr-0eee13951944c6b2d_egress_tcp_3389_3389_<IP_ADDRESS>/32      |
| sg-02222222222222222 | sgr-02fd2f97af5b64749  | -1        | -1      | -1          | <null>      | <null>            | arn:aws:ec2:ap-northeast-2:111111111111:security-group-rule/sgr-02fd2f97af5b64749 | sgr-02fd2f97af5b64749_egress_-1_-1_-1_sg-02222222222222222 |
| sg-02222222222222222 | sgr-0297fd188572ff485  | -1        | -1      | -1          | <null>      | <IP_ADDRESS>/0         | arn:aws:ec2:ap-northeast-2:111111111111:security-group-rule/sgr-0297fd188572ff485 | sgr-0297fd188572ff485_ingress_-1_-1_-1_<IP_ADDRESS>/0           |
| sg-00000000000000000 | sgr-045c2ece99018db86  | 0         | 65535   | tcp         | rextest     | <null>            | arn:aws:ec2:ap-northeast-2:111111111111:security-group-rule/sgr-045c2ece99018db86 | sgr-045c2ece99018db86_egress_tcp_0_65535_::/0              |
| sg-00000000000000000 | sgr-0477dfb205c83dcb3  | 0         | 65535   | udp         | <null>      | <IP_ADDRESS>/0         | arn:aws:ec2:ap-northeast-2:111111111111:security-group-rule/sgr-0477dfb205c83dcb3 | sgr-0477dfb205c83dcb3_egress_udp_0_65535_<IP_ADDRESS>/0         |
+----------------------+------------------------+-----------+---------+-------------+-------------+-------------------+-----------------------------------------------------------------------------------+------------------------------------------------------------+

@cjsrkd3321 Thank you for using Steampipe and good to see this update PR which is using a separate API than the existing table approach. We will be evaluating this.
Just curious to know, did you find the current table aws_vpc_security_group_rule not catering any data as expected or any other observation?

Hello. first, thanks for your reply. I need a 'sgr-*' column but the existing table was not. Of course, I tried to update the existed table(aws_vpc_security_group_rule) but I thought it wasn't the right way. For example, I checked our AWS account's SG vulnerability. And I asked the people in charge for action.
But, they were confused that "is this rule right?". So, I wrote and PR this code.
I don't know if you understood because my English is not good enough. If you need any additional information, please let me know.

@cjsrkd3321 Thanks for the quick response, understand that your team has confusion, sorry for the inconvenience.
If I understand correctly, after analyzing the new table columns with the current table, we find only an additional column is security_group_rule_id, is this the only column you are looking for to have? or is something I am missing to interpret?
As per the image attached

Yes. That's right. And I created a new table to follow the new API(DescribeSecurityGroupRules) form.

Hey @cjsrkd3321 , thanks for opening this PR! We really like the changes you made in this PR, especially using the API to get the VPC security group rules directly. We believe this is better than how we're doing it in our current aws_vpc_security_group_rule table, so we're updating our aws_vpc_security_group_rules table in this PR to make similar API calls to what you've done in this PR (so the  security_group_rule_id column would also be added as well).
We decided to update our existing table instead of adding a new one alongside it because we'd like to keep just 1 table in this plugin to represent VPC security group rules, so we won't merge in this PR, but would like to thank you for the direction you provided here.
If you have time, please try out using the updated aws_vpc_security_group_rule table in the rules branch and let us know if this satisfies your requirements. If you have any feedback on the changes, please let us know in that PR as well!


Jasper not working

I have installed julius and espeak
when I running jasper I get
Traceback (most recent call last):
  File "jasper.py", line 12, in <module>
    from client import tts, stt, jasperpath, diagnose
  File "/home/tomo/jasper/client/tts.py", line 41, in <module>
    import diagnose
  File "/home/tomo/jasper/client/diagnose.py", line 9, in <module>
    import pip.req
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 74, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/usr/lib/python2.7/dist-packages/pip/vcs/mercurial.py", line 9, in <module>
    from pip.download import path_to_url
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 25, in <module>
    from requests.compat import IncompleteRead
ImportError: cannot import name IncompleteRead

http://stackoverflow.com/a/27425458


Put the whole STM32F0 family behind a single feature

Signed-off-by: Daniel Egger<EMAIL_ADDRESS>

Implemented in https://github.com/Disasm/stm32-usbd/commit/76d79a10590ff561e9b053d750098cb66e7e256d


add filters

in theory we could implement all filter/column selection as in https://developmentseed.org/tifeatures/endpoints/#features

maybe in tipg


arm: Add missing include

Add missing include to prevent 'EINVAL' undeclared when using CONFIG_NULL_POINTER_EXCEPTION_DETECTION_DWT=y
Fixes https://github.com/zephyrproject-rtos/zephyr/issues/55144

Please fix the compliance check. No need to close and re-open a new PR. Just force-push on top of this one.

Sorry, its my first PR for this project.
I rebased and updated the commit message, but it still complains :(

No worries, there's always a learning curve with git and working with upstream repos, and I constantly have to look things up I don't regularly use myself.
You need to remove the extra merge commit, and only push out a single commit with your changes. Try this:
$ git reset --soft HEAD~1
$ git commit --no-edit --amend
$ git push -f origin <your_branch_name>

Thank you very much!
It looks better now.

Thank you very much!
It looks better now.

Ugg, wrong instructions, that added everything from the merge conflict into this commit. 🤦 1 second.

Sorry, my bad. To clean this up:
$ git reset --hard origin/main

Then use the editor to add the missing header ref back, and commit it with a sign-off via:
git commit -s

Then force push to your fork/branch again:
$ git push -f origin <branch>

This one is easy, at least:
author email (CaCO3<EMAIL_ADDRESS>needs to match one of the signed-off-by entries.

You can set name and email, then amend the email of the last commit via:
$ git config user.email<EMAIL_ADDRESS>$ git config user.name "First Last"
$ git commit --amend --reset-author
$ git push -f origin my-branch

Thanks, the last part was just a typo :)


Pinned-Dependencies incorrectly flags a pinned dependency downloaded from GitHub

Is your feature request related to a problem? Please describe.
I have a GitHub Actions workflow that uses rhysd/actionlint to lint GitHub Actions workflow files. As per its own documentation it is used by running a bash script to download and install the tool.
I however changed the value of main to the SHA so that it is pinned:
- name: Lint workflows
  shell: bash
  env:
    ACTIONLINT_VERSION: '7b75d16d41920ec126e6f3269db0c6f3ab613c38' # v1.6.25
  run: |
    echo "::add-matcher::.github/actionlint-matcher.json"
    bash <(curl --silent --show-error "https://raw.githubusercontent.com/rhysd/actionlint/${ACTIONLINT_VERSION}/scripts/download-actionlint.bash")
    ./actionlint -color

This was then flagged as an unpinned dependency as the code doesn't walk the whole step to find the referenced environment variable is a SHA to pin the download. It would be nice if it did, but it's understandable that it doesn't support that.
I then refactored to remove the environment variable like so:
- name: Lint workflows
  shell: bash
  run: |
    echo "::add-matcher::.github/actionlint-matcher.json"
    bash <(curl --silent --show-error "https://raw.githubusercontent.com/rhysd/actionlint/7b75d16d41920ec126e6f3269db0c6f3ab613c38/scripts/download-actionlint.bash")
    ./actionlint -color

This however still flags the warning:
Reason

dependency not pinned by hash detected -- score normalized to 6

Details

Warn: downloadThenRun not pinned by hash: .github/workflows/lint.yml:37
Info: GitHub-owned GitHubActions are pinned
Info: Third-party GitHubActions are pinned
Info: Dockerfile dependencies are pinned
Info: Pip installs are pinned

Describe the solution you'd like
The Pinned-Dependencies rule is able to correctly determine that URLs of this format to raw GitHub content in repositories are pinned.
Describe alternatives you've considered
None.
Additional context
None.

I'm going to have a quick look to see if I can fix this myself.


telegram_username

I removed username but i want to use allowed_telegram_usernames configuration, how can i do it

Unfortunately, you can't do it with current state of the code. But you can easily tweak it by setting user_id, instead of username in filters.User


Function.extractParams is deprecated.

Assigning custom properties to a marker when creating/copying it is
deprecated. Please, consider storing the custom properties you need in
some other object in your package, keyed by the marker's id property.
Function.extractParams (/usr/share/atom/resources/app.asar/node_modules/text-buffer/lib/marker.js:43:14)
MarkerLayer.markRange (/usr/share/atom/resources/app.asar/node_modules/text-buffer/lib/marker-layer.js:196:71)
DisplayMarkerLayer.markBufferRange (/usr/share/atom/resources/app.asar/node_modules/text-buffer/lib/display-marker-layer.js:90:62)
TextEditor.markBufferRange (/usr/share/atom/resources/app.asar/src/text-editor.js:1753:38)
<unknown> (/home/user/elmjutsu/lib/symbol-finder.js:125:40)
process._tickCallback (internal/process/next_tick.js:103:7)

I have noticed this new Deprecation message:
Assigning custom properties to a marker when creating/copying it is deprecated. Please, consider storing the custom properties you need in some other object in your package, keyed by the marker's id property.
Function.extractParams - :77541:20
MarkerLayer.markRange - :78187:77
DisplayMarkerLayer.markBufferRange - :134819:58
TextEditor.markBufferRange - :4451:42
Object.flashRange - ...atom\packages\elmjutsu\lib\helper.js:429:41
 - ...atom\packages\elmjutsu\lib\go-to.js:86:31

Any progress on this? :wink:


chore: release 9.1.3

:robot: I have created a release *beep* *boop*
9.1.3 (2022-08-02)
Bug Fixes

Use the new nodeRef API for Collapse (9527654)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/reactstrap/reactstrap/releases/tag/v9.1.3 :sunflower:


Issues computing connectivity with CTF system

I got a report that the connectivity computation was problematic. If you have a CTF-270 system, the function find_ch_connectivity gives you the connectivity of the CTF-275 system. Code to reproduce:
import mne
from mne.channels import find_ch_connectivity

contrast = mne.evoked.read_evokeds('./S1_discrimination_conds-ave.fif',
                                   condition='contrast_RP', verbose=None)

connectivity, ch_names = find_ch_connectivity(contrast.info, ch_type='mag')
assert connectivity.shape[-1] == len(contrast.info['ch_names'])

here is the file: https://www.dropbox.com/s/bae8rjdu068ws14/S1_discrimination_conds-ave.fif?dl=0.
My suggestion would be to make _compute_ch_connectivity public and deprecate the find_ch_connectivity function. The guesswork in the function introduces more problems than solves.

for MEG we use fieldtrip neighbors files:
https://github.com/fieldtrip/fieldtrip/tree/master/template/neighbours
I don't see any file for CTF 270.
any idea how to know what CTF system is used, maybe given the channel names?

okay so apparently it's a CTF-275 system with 5 bad channels

Arfff
What’s the expected behavior then? I would personally interpolate

@jasmainak - Fieldtrip neighbours structure gives adjacency information by channel name so you should still be able to use those files. Can you choose channels when reading connectivity in mne?

Yes, you are correct. This is what I realized after posting the issue. I think it's not technically a bug but more a surprising behavior. As people expect the same number of channels in the connectivity matrix as the info.
Regarding this particular data, the info did not have these 5 channels from the beginning (as far as I know).

we could subselect the neighbors structure but it means having holes or
maybe disconnected components...

@agramfort In the extreme that is possible, but I doubt it would be a common scenario. We could warn when some channels from adjacency are not present in info (and maybe the other way too). Disconnected components are already allowed at the source level (left and right hemi).

so warning and subsampling?

Yes - and there is another benefit of accepting the ch_names present in info: if the order of channels is different in fieldtrip neighbours file and the info - they would be sorted in the correct order (I'm not sure it is done now).

that would be fine with me.
+1 for PR on this


Save inbox preference

I added a feature for saving the last selected inbox filter.
Fixes #2669

Should we merge this @BasThomas? 😊


add a default scale to html2canvas

this helps to solve the default unit issue, which uses mm instead of pt, and issues #2911, #2907, and possibly #2885 and #2909.

Thanks for the PR. I'm wondering, though, if we should do this in a bugfix/minor release because it might break existing code that did not set the scale option. I will think about it and maybe I will come up with a more backwards compatible solution.

Yeah, I was debating whether we should do this as well. It's not even a bug fix but rather a nice feature to add for inexperienced users as I see this kind of question a lot online. Maybe it can be an option instead of a default, something like opt.autoscale: false. If the user set "autoscale" to true, then this code is activated and do the math to get the scale automatically.

I think a width option like in #2925 is probably the better option. We still need to figure out how to achieve this, though. Hadn't had the time to look into it myself.