Issue #244 iframe not resizing correctly when images are included in …

Added back support for checking each mutation for images.  Replaced event listener optimization check on image height/width with check on image.complete flag as height/width seem to be set in IE and Firefox with erroneous values even if not set on the actual image tag. Replaced bind approach for
registering image load event listener with event listener function to avoid registering multiple listeners for same image and also so the listener can be removed.  Also added listener for error event so that images which fail to load are handled cleanly.

Thank you for this, I'll take a look at it next week when I have a bit more spare time.

Released in v3.1.1.


Destructured assignment/default value sometimes inverted

Bug report

What is the current behavior?
Some destructured assignments are getting rewritten with invalid syntax... the = and : to rename are in the wrong order.
If the current behavior is a bug, please provide the steps to reproduce.
I'm not sure how to get this to happen outside of my app, but I have a component that does something like:
import { DataURIs } from '../constants';

const { BLANK_IMAGE } = DataURIs;

function test (data = {}) {
	const {
		src = BLANK_IMAGE,
	} = data;
}

and src becomes something like {src = constants_BLANK_IMAGE : src_constants_BLANK_IMAGE}... which is backwards.

What is the expected behavior?
no syntax error.

Other relevant information:
webpack version:  5.24.0
Node.js version: 15.8.0
Operating System:
Additional tools:

Duplicate https://github.com/webpack/webpack/issues/12769


Add station ID configuration example.

This addresses PhirePhly/aprx#40.

I don't feel that this section is a complete enough discussion on the topic of how APRS stations should beacon, and I feel that the needed documentation goes beyond the scope of the operating manual for Aprx.
Furthermore, you did not duplicate Bob's desired behavior for proportional pathing. The suggested path is:

""
""
"WIDE2-1"
""
""
"WIDE2-2"

I do think there should be something additional about station ID in there. That said, you're right on the configs, I had an extra WIDE2-1 in there.


Edit URLs in external editor

Fixes: #3288
Since URLs can be long and unwieldy they should be edited in an external editor instead of in the action bar.

Thanks! 🎉 👍


please help how can i write logic for caching (refetching updated data)

hello,
the scenario is on main page i show list of posts, and then it automatically cache but the problem is if i try to refetch data it does not show latest updated but only cached data :) i tried calling client.cache.reset() but then it remove all cached data. show app start loading each and every pages again..
in apollo there is a function called refetchQuery so only particular query can be updated and other query has no impact of cache..
how can do something like this. ???

I'm passing an extra fake variable into a query to force this. I call it _counter and I increment it each time I want to refetch. Seems to work fine, and as hacks go it doesn't feel bad at all. The more idiomatic way to do it is probably to pass a QueryController into the query and call controller.refresh or something to make it query again.

Thanks buddy, would help a lot

I wasn't able to find any reference to QueryController but passing a dummy variable worked for me.


bundle name fix

Since the correct bundle name is not being send to the native rules it is not using the module_name defined in the BUILD file, this causes this error:
omarzl@mbp-omar issue_bundle_name_demo % bazel build //:Mixed-LibFramework
INFO: Analyzed target //:Mixed-LibFramework (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /Users/omarzl/Documents/Rappi/dev/bazel/issue_bundle_name_demo/BUILD.bazel:5:14: Compiling Swift module Mixed_Lib failed (Exit 1): worker failed: error executing command bazel-out/host/bin/external/build_bazel_rules_swift/tools/worker/worker swiftc @bazel-out/applebin_ios-ios_x86_64-fastbuild-ST-1665503f7bef/bin/Mixed_Lib.swiftmodule-0.params
/private/var/tmp/_bazel_omarzl/d4b15372538f584ee97a74abbcce56ce/execroot/__main__/bazel-out/applebin_ios-ios_x86_64-fastbuild-ST-1665503f7bef/bin/Swift-Lib.intermediate-intermediates/module.modulemap:1:24: error: expected module declaration
framework module Swift-Lib {
Target //:Mixed-LibFramework failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.662s, Critical Path: 0.58s
INFO: 3 processes: 3 internal.
FAILED: Build did NOT complete successfully

Here is more information about this issue https://github.com/line/rules_apple_line/issues/20

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Omar Zuñiga Lagunas seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Typecast error

when I tried to run locally by downloading it and creating a Mule project in AnyPoint studio using it.
I am getting error as while running it-
Failed to execute goal org.mule.tools.maven:mule-maven-plugin:4.2.0:process-classes (default-process-classes) on project metrics-toolkit: A type incompatibility occurred while executing org.mule.tools.maven:mule-maven-plugin:4.2.0:process-classes: class java.lang.String cannot be cast to class java.lang.Number (java.lang.String and java.lang.Number are in module java.base of loader 'bootstrap')

I got the same error


KeyFilePath varies from default installation path of keys

Describe the bug
This is a bug that reveals that for the default UnencryptedFileSystemKeystore the KeyDir assumes a specific structure for the reading of the keys. This structure is out of alignment with any default construction as of the latest near-cli 1.6.0
The key paths by default are ~/.near-credentials/default/nameofaccount.json
The issue is the assumptions made here:
https://github.com/near/near-api-js/blob/master/src/key_stores/unencrypted_file_system_keystore.ts#L114
To Reproduce
npm install near-api-js
const nearAPI = require('near-api-js')
const { keyStores } = nearAPI;
const KEY_PATH = "~/.near-credentials/default/"
const keyStore = new keyStores.UnencryptedFileSystemKeyStore(KEY_PATH);
const result = await keyStore.getKey("yournetwork","youracccountid")

copy pasta into an index.js
Expected behavior
You expect to receive your key instead you'll throw a can't find key error
Desktop (please complete the following information):

OS: Mac
Version 10.15.7

Additional context
Should be a simple fix, the network order assumption should perhaps be removed, or the cli should put the keys in the proper paths, alternatively the error message should throw a put structure keys in directory as follows.
I'm most inclined for removing the path assumption from the Keystore.
What do you all think?

@zcstarr I think its working as intended. The key path should be something like ~/.near-credentials. default is the name of a network (unfortunate naming, but that is currently being fixed by @thisisjoshford https://github.com/near/near-cli/issues/677).
So it should be used like this:
const nearAPI = require('near-api-js')
const { keyStores } = nearAPI;
const KEY_PATH = "~/.near-credentials/"
const keyStore = new keyStores.UnencryptedFileSystemKeyStore(KEY_PATH);
const result = await keyStore.getKey("default","youracccountid")

So yes this would be the case, the issue is that when you pass the keystore to any consuming object, like within the accounts logic, for instance see signTransaction. You don't always have access to specifying the keystore.getKey network. It maybe seems a bit fragile the assumptions 🤔
It essentially means you always have to structure your keys in a specific pattern.  I think maybe keyStore could use a refactor so the path to the key is the path to your keys.  This might be a breaking change...

@thisisjoshford , @mehtaphysical  Any further thoughts here ^^ . I have the thought that this will be one of those situations that bites more, as people build different applications on top of the keystore.
The path assumption I think regardless of default, non-default being so low level shapes use cases downstream, perhaps too aggressively. I think this bites even more when people have restrictions on directories that they can read/write from.

I think it would be nice to add more options here. I don't think we want to change this key store and create a breaking change.
We can make a new key store. Maybe a UnencriptedTomlFileKeyStore with the following format:
[NETWORK_ID.ACCOUNT_ID]
account_id = "ACCOUNT_ID"
public_key = "PUBLIC_KEY"
private_key = "PRIVATE_KEY"

For example:
[testnet."me.testnet"]
account_id = "me.testnet"
public_key = "PUBLIC_KEY"
private_key = "PRIVATE_KEY"

[mainnet."me.near"]
account_id = "me.near"
public_key = "PUBLIC_KEY"
private_key = "PRIVATE_KEY"

I like this idea, I think it'll make the keystore a little more flexible. How do you feel about Json with Json Schema?
My thought here is that if we just spec out a schema see below. Then we can do cool things like add type ahead support in all the environments for configuring the keystore. Like the monaco editor allows you to add support for json schema in configuration files.
This would mean that in online editing environments you can add typeahead and light error correcting.
Additionally if we use json we can support in emory representations of the keystore that have a similar mapping if not 1-1 corresponding mapping to the JSON config representation of the keystore.
The interface would be:
UnencryptedJsonKeystore(keyConfig: jsonKeyStoreConfig)
UnencryptedJsonFileKeystore(path: string)

UnencryptedJsonFileKeystore just uses JsonKeystore under the hood, it's a convenience wrapper that reads files from disk.
here's a spec via gist and inline here so it's maybe easier to read it's verbose 😁
{
    "$id": "near-keystore-config-schema.json",
    "$schema": "http://json-schema.org/draft-07/schema#",
    "description": "Near JSON Keystore Configuration",
    "type": "object",
    "$ref": "#/definitions/jsonKeystoreConfig",
    "definitions": {
        "jsonKeystoreConfig": {
            "title": "jsonKeystoreConfig",
            "type": "object",
            "properties": {
                "mainnet": {
                    "$ref": "#/definitions/keyConfigs"
                },
                "testnet": {
                    "$ref": "#/definitions/keyConfigs"
                }
            },
            "additionalProperties": {
                "$ref": "#/definitions/keyConfigs"
            },
            "$comment": "keys are assumed to be the networkId"
        },
        "keyConfigs": {
            "type": "array",
            "items": {
                "$ref": "#/definitions/keyConfig"
            }
        },
        "keyConfig" : {
            "title": "keyConfig",
            "type": "object",
            "properties": {
                "account_id": {
                    "$ref": "#/definitions/account_id"
                },
                "public_key": {
                    "$ref": "#/definitions/public_key"
                },
                "private_key": {
                    "$ref": "#/definitions/private_key"
                }
            }
        },
        "account_id": {
            "title": "account_id",
            "type": "string",
            "pattern": "^(([a-z0-9]+[\\-_])*[a-z0-9]+\\.)*([a-z0-9]+[\\-_])*[a-z0-9]+$",
            "minLength": 2,
            "maxLength": 64
        },
        "public_key": {
            "title": "public_key",
            "type": "string",
            "pattern": "^ed25519:[a-zA-Z0-9]+$"
        },
        "private_key": {
            "title": "private_key",
            "type": "string",
            "pattern": "^ed25519:[a-zA-Z0-9]+$"
        }
    }
}

example
{
    "mainnet": [
        {
            "account_id": "foobar.near",
            "private_key": "ed25519:asdfasdfXAwfeafafwaef",
            "public_key": "ed25519:asdfasdfXAwfeafafwaefbobbyB33FXawfaw33faf3f3a3"
        },
        {
            "account_id": "baz.near",
            "private_key": "ed25519:KxTsdfasdfXAwfeafafwaef",
            "public_key": "ed25519:ZsdfasdfXAwfeafafwaefbobbyB33FXawfaw33faf3f3a3"
        }
    ],
    "testnet": [
        {
            "account_id": "foobar.testnet",
            "private_key": "ed25519:asdfasdfXAwfeafafwaef",
            "public_key": "ed25519:asdfasdfXAwfeafafwaefbobbyB33FXawfaw33faf3f3a3"
        }
    ],
    "customlocal": [
        {
            "account_id": "foobar.customlocal",
            "private_key": "ed25519:asdfasdfXAwfeafafwaef",
            "public_key": "ed25519:asdfasdfXAwfeafafwaefbobbyB33FXawfaw33faf3f3a3"
        }
    ]
}

This looks good to me!


"import_meta_graph" fails

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): no
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): CentOS7
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: no
TensorFlow installed from (source or binary): pip
TensorFlow version (use command below): 1.13.1
Python version: 3.5.6
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: cuda 10.0 with cuDNN7
GPU model and memory: GTX1080 with 8GB

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with: 1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)" 2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"
Describe the current behavior
error occurs when calling import_meta_graph
Describe the expected behavior
Code to reproduce the issue
import tensorflow as tf
import tensorflow.contrib.tensorrt as trt

meta_graph_path = "bert-eng/model.ckpt-7300.meta"
graph = tf.Graph()
with graph.as_default():
    with tf.Session() as sess:

        saver = tf.train.import_meta_graph(meta_graph_path) // error occurs here

When I run the above code, I get error at line:
saver = tf.train.import_meta_graph(meta_graph_path) // error occurs here

Other info / logs
stack trace:
2019-04-25 10:40:52.107550: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-04-25 10:40:52.119933: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency:<PHONE_NUMBER> Hz
2019-04-25 10:40:52.122488: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x5431820 executing computations on platform Host. Devices:
2019-04-25 10:40:52.122547: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
Traceback (most recent call last):
  File "tf_to_trt.py", line 15, in <module>
    saver = tf.train.import_meta_graph(meta_graph_path)
  File "/home/msl/.virtualenvs/venv_waveglow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1435, in import_meta_graph
    meta_graph_or_file, clear_devices, import_scope, **kwargs)[0]
  File "/home/msl/.virtualenvs/venv_waveglow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1457, in _import_meta_graph_with_return_elements
    **kwargs))
  File "/home/msl/.virtualenvs/venv_waveglow/lib/python3.5/site-packages/tensorflow/python/framework/meta_graph.py", line 806, in import_scoped_meta_graph_with_return_elements
    return_elements=return_elements)
  File "/home/msl/.virtualenvs/venv_waveglow/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/home/msl/.virtualenvs/venv_waveglow/lib/python3.5/site-packages/tensorflow/python/framework/importer.py", line 399, in import_graph_def
    _RemoveDefaultAttrs(op_dict, producer_op_list, graph_def)
  File "/home/msl/.virtualenvs/venv_waveglow/lib/python3.5/site-packages/tensorflow/python/framework/importer.py", line 159, in _RemoveDefaultAttrs
    op_def = op_dict[node.op]
KeyError: 'ExperimentalFunctionBufferingResource'

@ThisIsIsaac Could create a reproducible code (a gist or colab) to reproduce the issue. The above code is not complete. Thanks!

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

Closing due to lack of recent activity. Please update the issue when new information becomes available, and we will open a new issue. Thanks!

Similar Issues #27752
#29751


An error while building nuproj projects.

I'm getting the following exception on my machine:
Error   MSB4018 The "ReadPdbSourceFiles" task failed unexpectedly.
System.IO.FileNotFoundException: Retrieving the COM class factory for component with CLSID {3BFCEA48-620F-4B6B-81F7-B9AF75454C7D} failed due to the following error: 8007007e The specified module could not be found. (Exception from HRESULT: 0x8007007E).
   at System.RuntimeTypeHandle.CreateInstance(RuntimeType type, Boolean publicOnly, Boolean noCheck, Boolean& canBeCached, RuntimeMethodHandleInternal& ctor, Boolean& bNeedSecurityCheck)
   at System.RuntimeType.CreateInstanceSlow(Boolean publicOnly, Boolean skipCheckThis, Boolean fillCache, StackCrawlMark& stackMark)
   at System.RuntimeType.CreateInstanceDefaultCtor(Boolean publicOnly, Boolean skipCheckThis, Boolean fillCache, StackCrawlMark& stackMark)
   at System.Activator.CreateInstance(Type type, Boolean nonPublic)
   at System.Activator.CreateInstance(Type type)
   at NuProj.Tasks.PdbReader.ReadSourceFiles(String pdbPath) in C:\git\nuproj\src\NuProj.Tasks\ReadPdbSourceFiles.cs:line 36
   at NuProj.Tasks.ReadPdbSourceFiles.Execute() in C:\git\nuproj\src\NuProj.Tasks\ReadPdbSourceFiles.cs:line 25
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
   at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext() MSCorEE.NuGet   C:\Users\hmemcpy\.nuget\packages\NuProj\0.10.27-beta-g2363d7cd98\tools\NuProj.targets   963

This happens for every nuproj project. Perhaps this is more of a question/issue for the nuproj repo...
Windows 10, VS2015. Any ideas?

Hmm, I don't seem to have msdia*.dll on my machine, which this guid seems to belong to.

I get lots of results for msdia*.dll when I recursively search from c:\program files (x86)\. Have you searced there?
c:\Program Files (x86)>dir /s msdia*.dll /b
c:\Program Files (x86)\Common Files\Microsoft Shared\Phone Tools\12.0\Debugger\target\armv4i\MSDIA120.dll
c:\Program Files (x86)\Common Files\Microsoft Shared\Phone Tools\12.0\Debugger\target\x86\MSDIA120.dll
c:\Program Files (x86)\Common Files\Microsoft Shared\Phone Tools\14.0\Debugger\target\armv4i\MSDIA140.dll
c:\Program Files (x86)\Common Files\Microsoft Shared\Phone Tools\14.0\Debugger\target\x64\MSDIA140.dll
c:\Program Files (x86)\Common Files\Microsoft Shared\Phone Tools\14.0\Debugger\target\x86\MSDIA140.dll
c:\Program Files (x86)\Common Files\Microsoft Shared\VC\amd64\msdia80.dll
c:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\Packages\Debugger\msdia120.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\msdia120.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\msdia140.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\ClientDiagnostics\MemoryProfiler\DataModel\msdia140.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\ClientDiagnostics\MemoryProfiler\x86\msdia120.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\msdia140typelib_clr0200.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\x64\msdia140typelib_clr0200.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Platform\DiagnosticsHub\amd64\msdia140.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Platform\DiagnosticsHub\x86\msdia140.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Remote Debugger\x64\msdia140.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Remote Debugger\x86\msdia140.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Packages\Debugger\msdia140.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Packages\Debugger\X64\msdia140.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\DIA SDK\bin\msdia140.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\DIA SDK\bin\amd64\msdia140.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\DIA SDK\bin\arm\msdia140.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Team Tools\Dynamic Code Coverage Tools\msdia140.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Team Tools\Dynamic Code Coverage Tools\amd64\msdia140.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Team Tools\Performance Tools\x64\msdia120.dll
c:\Program Files (x86)\Microsoft Visual Studio 14.0\Team Tools\Performance Tools\x64\msdia140.dll
c:\Program Files (x86)\MSBuild\Microsoft\.NetNative\arm\ilc\tools\msdia120.dll
c:\Program Files (x86)\MSBuild\Microsoft\.NetNative\x64\ilc\tools\msdia120.dll
c:\Program Files (x86)\MSBuild\Microsoft\.NetNative\x86\ilc\tools\msdia120.dll
c:\Program Files (x86)\Windows Kits\10\App Certification Kit\msdia120.dll
c:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\msdia140.dll

Given that VS is what seems to offer this DLL, you might try 'repairing' your VS installation.

Oh, my bad! Turns out Everything (which is generally great), doesn't automatically do wildcard searches, so my search for "MSDIA" didn't get me any valid results.
Yeah, I have tons of MSDIA110,120 and 140 dlls...

Well, 'repairing' your VS installation is still the only idea I have.

Had the same problem as @hmemcpy, and repairing the installation didn't change anything.
Any other ideas?


Storj is no more available on "ROI" tab

Hi,
I've been using this application for 2 months but it seems that, recently, STORJ is no more available on it.

Is that normal?
If so, why did you choose to de-reference it?
Thank you!

It is normal because STORJ pool became illiquid: https://v1.uniswap.info/token/0xb64ef51c888972c908cfacf59b47c1afbc0ab8ac
I guess most of the LPs migrated to Uniswap V2 already.

Thanks!
I thought ZumZoom was designed also for Uniswap V2. Is it planned to include V2 pools?

It is not planned yet but it will probably happen sooner or later :)


Wip issue 55

Description

Related Issue(s)

#{issue number}

Verification

[ ] Your code builds clean without any errors or warnings
[ ] Manual testing done (required)
[ ] Relevant automated test added (if you find this hard, leave it and we'll help out)
[ ] All tests run green

Documentation

[ ] User documentation is updated with a separate linked PR in altinn-studio-docs. (if applicable)

Closing this as another solution was implemented.


Config Parameters for Open Wire Test

Added parameters to config file so changes in battery module configuration do not require changes in Voltage.c file.

Why are there 28 commits in this pull request? I think it would make more sense to just rebase your one commit to be in a new branch based on rtos instead of VoltTempMonitorTaskTest


Can`t input less than 1$ without  using keyboard arrows in alias: "currency" with decimals

I think there must be possibility to input decimals on second mouse click,
disable focus on decimal separator on second click, it works so if I input some digit before decmals
https://user-images.githubusercontent.com/56801823/183105276-9d744e40-e8e1-4606-a000-c31d42c26191.mp4
"react": "^17.0.2",
"inputmask": "^5.0.7",
Windows 10, Chrome,

@Anton1921980 ,
Try typing:

.02
0.02
1.02

or use the numericInput: true options and type

2
102

@Anton1921980 ,
Ofcourse you can also use https://github.com/RobinHerbots/Inputmask/blob/5.x/README.md#positioncaretonclick

  alias: "currency",
    positionCaretOnClick: "none"

@Anton1921980 ,
Now that I have reread your issue, .... hmm yes, ...  this is already the case once you have typed something.  Maybe this should also work without the need to type something.  I will change that.

@Anton1921980 ,
Done!


Calculation in DagAtlas tends to “hang”

Very interesting idea 💡 of extracting DAGs from Text (DEFT)! So I want to do some experiments with this code.
Running the notebook (short story case) locally of 2 of my PC’s seems to hang in the DagAtlas calculation (even though the first comparisons seem to go alright, the last one does not seem to end after waiting more than 24 hours …).
Running this on Google Colab ir running it on Kaggle did finish the DagAtlas calculation successfully. But than in both cases no graphical output was produced. Up till now no idea about the reason for this.
I hope you can shed some light on these problems!
What is also a problem are the extremely long calculations, even with only 3 texts, as you already mentioned. I am planning to look into that if I am convinced that this proof of concept really works!

Thank you, from the bottom of my heart, for trying this out.
The DagAtlas calculation is definitely the bottle neck. Stop the
calculation for the short stories notebook if it takes longer than 20
minutes, because that is how long it takes on my old slow PC. Where  does
it hang? Can you send me the notebook? It prints out after each bridge is
found. It also prints out how many similarity comparisons it is going to
make. On my slow PC, it can make about 300,000 comparisons per 20 minutes
or about 1M comparisons per hour.
On Wed, Apr 26, 2023 at 12:38 PM Jan Bours @.***> wrote:

Very interesting idea 💡 of extracting DAGs from Text (DEFT)! So I want
to do some experiments with this code.
Running the notebook (short story case) locally of 2 of my PC’s seems to
hang in the DagAtlas calculation (even though the first comparisons seem to
go alright, the last one does not seem to end after waiting more than 24
hours …).
Running this on Google Colab ir running it on Kaggle did finish the
DagAtlas calculation successfully. But than in both cases no graphical
output was produced. Up till now no idea about the reason for this.
I hope you can shed some light on these problems!
What is also a problem are the extremely long calculations, even with only
3 texts, as you already mentioned. I am planning to look into that if I am
convinced that this proof of concept really works!
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYLV4STGWZW3P5I3RGDXDFFPBANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

I do not know exactly where it hangs, but I will try to pinpoint it by putting some traces in the code. As soon as I figured it out I will come back to you here, OK?

Making your efforts even more admirable, I think this article would be interesting to read:
Causal Parrots: Large Language Models May Talk Causality But Are Not Causal
https://www.matej-zecevic.de/research/assets/Causal_Parrots__Large_Language_Models_May_Talk_Causality_But_Are_Not_Causal_.pdf
Causal Parrots: Large Language Models May Talk Causality But Are Not Causal 2
3 MORITZ WILLIG∗ and MATEJ ZEČEVIĆ∗, Technical University of Darmstadt, Germany
DEVENDRA SINGH DHAMI, Technical University of Darmstadt and hessian.AI, Germany KRISTIAN KERSTING, Technical University of Darmstadt, hessian.AI and DFKI, Germany
Some argue scale is all what is needed to achieve AI, covering even causal models. We make it clear that large language models (LLMs) cannot be causal and give reason onto why sometimes we might feel otherwise. To this end, we define and exemplify a new subgroup of Structural Causal Model (SCM) that we call meta SCM which encode causal facts about other SCM within their variables. We conjecture that in the cases were LLM succeed in doing causal inference, underlying was a respective meta SCM that exposed correlations between causal facts in natural language on whose data the LLM was ultimately trained. If our hypothesis holds true, then this would imply that LLMs are like parrots in that they simply recite the causal knowledge embedded in the data. Our analysis provides favoring evidence that current LLMs are even weak ‘causal parrots.’
Additional Key Words and Phrases: large language models, correlation of causal facts conjecture, empirical analysis
ACM Reference Format:
Moritz Willig, Matej Zečević, Devendra Singh Dhami, and Kristian Kersting. 2023. Causal Parrots: Large Language Models

Thanks. I agree with this abstract. The way I like to put it is this way
LLM + explicit CI engine + DAG atlas
will be much closer to human level AI than LLM alone.
On Wed, Apr 26, 2023 at 6:04 PM Jan Bours @.***> wrote:

Making your efforts even more admirable, I think this article would be
interesting to read:
Causal Parrots: Large Language Models May Talk Causality But Are Not Causal
https://www.matej-zecevic.de/research/assets/Causal_Parrots__Large_Language_Models_May_Talk_Causality_But_Are_Not_Causal_.pdf
Causal Parrots: Large Language Models May Talk Causality But Are Not
Causal 2
3 MORITZ WILLIG∗ and MATEJ ZEČEVIĆ∗, Technical University of Darmstadt,
Germany
DEVENDRA SINGH DHAMI, Technical University of Darmstadt and hessian.AI,
Germany KRISTIAN KERSTING, Technical University of Darmstadt, hessian.AI
and DFKI, Germany
Some argue scale is all what is needed to achieve AI, covering even causal
models. We make it clear that large language models (LLMs) cannot be causal
and give reason onto why sometimes we might feel otherwise. To this end, we
define and exemplify a new subgroup of Structural Causal Model (SCM) that
we call meta SCM which encode causal facts about other SCM within their
variables. We conjecture that in the cases were LLM succeed in doing causal
inference, underlying was a respective meta SCM that exposed correlations
between causal facts in natural language on whose data the LLM was
ultimately trained. If our hypothesis holds true, then this would imply
that LLMs are like parrots in that they simply recite the causal knowledge
embedded in the data. Our analysis provides favoring evidence that current
LLMs are even weak ‘causal parrots.’
Additional Key Words and Phrases: large language models, correlation of
causal facts conjecture, empirical analysis
ACM Reference Format:
Moritz Willig, Matej Zečević, Devendra Singh Dhami, and Kristian Kersting.
2023. Causal Parrots: Large Language Models
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1524099507,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYIJNVDT6FME43A3XY3XDGLULANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

I think that could be true!
Meanwhile I am debugging the code in VS Code on my PC locally. As far as I can see up till now the code does not hang, but takes a lot of time, and produces mostly similarities equal to 0.
Perhaps it is possible to optimize the NLP part of the code?
This article gives some tips and tricks:

The short stories jupyter notebook should not hang. It takes 18.56 minutes
to run on my computer. It tells you how many similarity calculations it's
going to make. I've highlighted that number in yellow. Do you get the same
yellow numbers?
This is the printout I get from running the simplification step:
Initiating DagAtlas object: 0.00 minutes
Starting comparison of 2 titles: 0.00 minutes
title1 and its num of nodes: bill-the-bloodhound 317
title2 and its num of nodes: extricating-young-gussie 396product of
numbers of nodes= 125532
Starting bridges: 0.00 minutes
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
9 bridges
10 bridges
11 bridges
12 bridges
13 bridges
14 bridges
15 bridges
Before saving 2 dags: 7.39 minutes
Exiting 2 titles comparison: 7.39 minutes
Starting comparison of 2 titles: 7.39 minutes
title1 and its num of nodes: bill-the-bloodhound 317
title2 and its num of nodes: wiltons-holiday 249product of numbers of
nodes= 78933
Starting bridges: 7.39 minutes
1 bridges
2 bridges
3 bridges
4 bridges
Before saving 2 dags: 11.82 minutes
Exiting 2 titles comparison: 11.82 minutes
Starting comparison of 2 titles: 11.82 minutes
title1 and its num of nodes: extricating-young-gussie 396
title2 and its num of nodes: wiltons-holiday 249product of numbers of
nodes= 98604
Starting bridges: 11.82 minutes
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
9 bridges
10 bridges
11 bridges
12 bridges
13 bridges
14 bridges
15 bridges
16 bridges
17 bridges
18 bridges
19 bridges
20 bridges
Before saving 2 dags: 18.58 minutes
Exiting 2 titles comparison: 18.58 minutes
On Thu, Apr 27, 2023 at 2:07 PM Jan Bours @.***> wrote:

I think that could be true!
Meanwhile I am debugging the code in VS Code on my PC locally. As far as I
can see up till now the code does not hang, but takes a lot of time, and
produces mostly similarities equal to 0.
Perhaps it is possible to optimize the NLP part of the code?
This article gives some tips and tricks:
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1526115064,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYNGCY3YZMCGGOGSNITXDKYT7ANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

I have restarted the calculations. The start of my output is the same as yours:
Output exceeds the size limit. Open the full output data in a text editorInitiating DagAtlas object: 0.00 minutes
Starting comparison of 2 titles: 0.00 minutes
title1 and its num of nodes: bill-the-bloodhound 317
title2 and its num of nodes: extricating-young-gussie 396
product of numbers of nodes= 125532
Starting bridges: 0.00 minutes
Then I have put in some traces so I cab floow what happens "under teh hood":
Initiating DagAtlas object: 0.00 minutes
Starting comparison of 2 titles: 0.00 minutes
title1 and its num of nodes: bill-the-bloodhound 317
title2 and its num of nodes: extricating-young-gussie 396
product of numbers of nodes= 125532
Starting bridges: 0.00 minutes
SIMI_THRESHOLD: 3
JB: nd1 nd2 <Node.Node object at 0x00000139283E2730> <Node.Node object at 0x0000013928405C40>
simi.ztz_similarity(ztz1, ztz2): 0.0
JB: nd1 nd2 <Node.Node object at 0x00000139283E2730> <Node.Node object at 0x0000013928405CA0>
simi.ztz_similarity(ztz1, ztz2): 0.0
JB: nd1 nd2 <Node.Node object at 0x00000139283E2730> <Node.Node object at 0x0000013928405D00>
simi.ztz_similarity(ztz1, ztz2): 0.0
...
Up till now all similarities are 0.0 (!).
And I do not see:
1 bridges
2 bridges
3 bridges
Etc.
as in your output.
But that will have to do with all similarities being 0.0 I expect.
I will have to wait till all calculations are done, but I begin to suspect the similarity calculations are wrong for some reason!
What do you think?

I think I better test the similarity calculations seperately.

You can test the similarity calculations separately by going to the
similarity.py file and running it. It will run two examples, main1() and
main2()
On Thu, Apr 27, 2023 at 3:58 PM Jan Bours @.***> wrote:

I think I better test the similarity calculations seperately.
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1526315790,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYO5B3JKMXGMM5ZT3DTXDLFWXANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

I suspect you might have nlkt installed, but not wordnet
On Thu, Apr 27, 2023 at 3:58 PM Jan Bours @.***> wrote:

I think I better test the similarity calculations seperately.
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1526315790,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYO5B3JKMXGMM5ZT3DTXDLFWXANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

This is what I get when I run similarity.py. Make sure you get the same.
************ simi definition from: similarity_nltk

Cats are beautiful animals.
Dogs are awesome.
simi(1, 2)= 0.885
simi(2, 1)= 0.837
Cats are beautiful animals.

Dolphins are swimming mammals.
simi(1, 2)= 0.616
simi(2, 1)= 0.733
Cats are beautiful animals.
Cats are beautiful animals.
simi(1, 2)= 1000
simi(2, 1)= 1000
Cats are beautiful animals.
Cats are beauti animals.
simi(1, 2)= 7.0
simi(2, 1)= 11.0
************ simi definition from: similarity_nltk
apple
horse
simi(1, 2)= 0.056
simi(2, 1)= 0.056
Paul
John
simi(1, 2)= 0.083
simi(2, 1)= 0.083
The cat sat on the mat.
The dog lay on the rug.
simi(1, 2)= 0.353
simi(2, 1)= 0.353

Process finished with exit code 0
On Thu, Apr 27, 2023 at 3:58 PM Jan Bours @.***> wrote:

I think I better test the similarity calculations seperately.
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1526315790,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYO5B3JKMXGMM5ZT3DTXDLFWXANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

I will do that tomorrow.
Your suspicion that Wordnet could not be installed probably is correct, for I don’t remember installing it separately.
So I will install Wordnet and run then run similarity.py and check if I get the same results that you posted.
Of course it is strange that I don’t get an error or it least a warning that Wordnet is not installed …

okay. Thanks
On Thu, Apr 27, 2023 at 7:36 PM Jan Bours @.***> wrote:

I will do that tomorrow.
Your suspicion that Wordnet could not be installed probably is correct,
for I don’t remember installing it separately.
So I will install Wordnet and run then run similarity.py and check if I
get the same results that you posted.
Of course it is strange that I don’t get an error or it least a warning
that Wordnet is not installed …
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1526765255,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYILHOSP6RIRLI6OCWLXDL7GZANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

I managed to get ity working!
These were the actions that I took:
1)
import nltk
nltk.download('wordnet')

I ran the code from
https://nlpforhackers.io/wordnet-sentence-similarity/
This gave the following message:
"# **********************************************************************
Resource omw-1.4 not found.
Please use the NLTK Downloader to obtain the resource:

>>> import nltk
>>> nltk.download('omw-1.4')

For more information see: https://www.nltk.org/data.html

Attempted to load corpora/omw-1.4.zip/omw-1.4/

Searched in:
- 'C:\Users\HP/nltk_data'
...
- 'C:\nltk_data'
- 'D:\nltk_data'
- 'E:\nltk_data'
**********************************************************************"
After which I did:
import nltk
nltk.download('omw-1.4')
I am not really sure if this has anyting to do with the problem iself though.

I also added this (not sure if this will resolve my probloem with the graphical output):
For GraphViz in VS Code !
os.environ["PATH"] += os.pathsep + 'C:\Program Files\Graphviz\bin'
(VS Code restart necessary afterwards?)

Now my output form similarity.py is (not exactly the same, don''t know why):
************ simi definition from: similarity_nltk

Cats are beautiful animals.
Dogs are awesome.
simi(1, 2)= 1.045
simi(2, 1)= 1.045
Cats are beautiful animals.

Dolphins are swimming mammals.
simi(1, 2)= 0.733
simi(2, 1)= 0.733
Cats are beautiful animals.
Cats are beautiful animals.
simi(1, 2)= 1000
simi(2, 1)= 1000
Cats are beautiful animals.
Cats are beauti animals.
simi(1, 2)= 11.0
simi(2, 1)= 11.0
************ simi definition from: similarity_nltk
apple
horse
simi(1, 2)= 0.056
simi(2, 1)= 0.056
horse
apple
simi(1, 2)= 0.056
simi(2, 1)= 0.056
Paul
John
simi(1, 2)= 0.083
simi(2, 1)= 0.083
The cat sat on the mat.
The dog lay on the rug.
simi(1, 2)= 0.353
simi(2, 1)= 0.353

Hi Jan,
Interesting. Perhaps we have different versions of wordnet?
So your computer takes 16 minutes total, whereas mine takes 19 minutes. Not
a huge
improvement :(
I haven't run the movie scripts notebook. That one should take 15 times
longer.
I omitted one cell of the movie scripts notebook. I will put it back in. If
you can
run the movie scripts notebook for me, I would be eternally grateful. I
haven't done it with
my computer because I am afraid it will overheat and die. It's a very old
computer, and was already second
hand when I bought it.
On Fri, Apr 28, 2023 at 5:22 AM Jan Bours @.***> wrote:

My output from DagAtlas.py is now:
Initiating DagAtlas object: 0.00 minutes
Starting comparison of 2 titles: 0.00 minutes
title1 and its num of nodes: bill-the-bloodhound 317
title2 and its num of nodes: extricating-young-gussie 396
product of numbers of nodes= 125532
Starting bridges: 0.00 minutes
SIMI_THRESHOLD: 3
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
9 bridges
10 bridges
11 bridges
12 bridges
13 bridges
14 bridges
15 bridges
16 bridges
17 bridges
18 bridges
19 bridges
20 bridges
21 bridges
22 bridges
23 bridges
24 bridges
25 bridges
26 bridges
27 bridges
28 bridges
Before saving 2 dags: 7.19 minutes
Exiting 2 titles comparison: 7.23 minutes
Starting comparison of 2 titles: 7.23 minutes
JB: path1 = short_stories_dag_atlas/bill-the-bloodhound.pkl
title1 and its num of nodes: bill-the-bloodhound 317
title2 and its num of nodes: wiltons-holiday 249
product of numbers of nodes= 78933
Starting bridges: 7.23 minutes
SIMI_THRESHOLD: 3
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
Before saving 2 dags: 11.78 minutes
Exiting 2 titles comparison: 11.79 minutes
Starting comparison of 2 titles: 11.79 minutes
JB: path1 = short_stories_dag_atlas/extricating-young-gussie.pkl
JB: path2 = short_stories_dag_atlas/wiltons-holiday.pkl
title1 and its num of nodes: extricating-young-gussie 396
title2 and its num of nodes: wiltons-holiday 249
product of numbers of nodes= 98604
Starting bridges: 11.79 minutes
SIMI_THRESHOLD: 3
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
9 bridges
10 bridges
11 bridges
12 bridges
13 bridges
14 bridges
15 bridges
16 bridges
17 bridges
18 bridges
19 bridges
20 bridges
21 bridges
22 bridges
23 bridges
24 bridges
25 bridges
26 bridges
27 bridges
28 bridges
29 bridges
Before saving 2 dags: 16.83 minutes
Exiting 2 titles comparison: 16.88 minutes
Running it a second time gives:
Initiating DagAtlas object: 0.00 minutes
Starting comparison of 2 titles: 0.00 minutes
title1 and its num of nodes: bill-the-bloodhound 317
title2 and its num of nodes: extricating-young-gussie 396
product of numbers of nodes= 125532
Starting bridges: 0.00 minutes
SIMI_THRESHOLD: 3
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
9 bridges
10 bridges
11 bridges
12 bridges
13 bridges
14 bridges
15 bridges
16 bridges
17 bridges
18 bridges
19 bridges
20 bridges
21 bridges
22 bridges
23 bridges
24 bridges
25 bridges
26 bridges
27 bridges
28 bridges
Before saving 2 dags: 6.65 minutes
Exiting 2 titles comparison: 6.70 minutes
Starting comparison of 2 titles: 6.70 minutes
JB: path1 = short_stories_dag_atlas/bill-the-bloodhound.pkl
title1 and its num of nodes: bill-the-bloodhound 317
title2 and its num of nodes: wiltons-holiday 249
product of numbers of nodes= 78933
Starting bridges: 6.70 minutes
SIMI_THRESHOLD: 3
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
Before saving 2 dags: 10.86 minutes
Exiting 2 titles comparison: 10.91 minutes
Starting comparison of 2 titles: 10.91 minutes
JB: path1 = short_stories_dag_atlas/extricating-young-gussie.pkl
JB: path2 = short_stories_dag_atlas/wiltons-holiday.pkl
title1 and its num of nodes: extricating-young-gussie 396
title2 and its num of nodes: wiltons-holiday 249
product of numbers of nodes= 98604
Starting bridges: 10.91 minutes
SIMI_THRESHOLD: 3
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
9 bridges
10 bridges
11 bridges
12 bridges
13 bridges
14 bridges
15 bridges
16 bridges
17 bridges
18 bridges
19 bridges
20 bridges
21 bridges
22 bridges
23 bridges
24 bridges
25 bridges
26 bridges
27 bridges
28 bridges
29 bridges
Before saving 2 dags: 15.93 minutes
Exiting 2 titles comparison: 15.98 minutes
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1527255752,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYOIC7IY5PIN263CZYTXDOD37ANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

Okay. I have updated the movie scripts jupyter notebook. I have added the
code that creates the DAG atlas.
On Fri, Apr 28, 2023 at 5:22 AM Jan Bours @.***> wrote:

My output from DagAtlas.py is now:
Initiating DagAtlas object: 0.00 minutes
Starting comparison of 2 titles: 0.00 minutes
title1 and its num of nodes: bill-the-bloodhound 317
title2 and its num of nodes: extricating-young-gussie 396
product of numbers of nodes= 125532
Starting bridges: 0.00 minutes
SIMI_THRESHOLD: 3
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
9 bridges
10 bridges
11 bridges
12 bridges
13 bridges
14 bridges
15 bridges
16 bridges
17 bridges
18 bridges
19 bridges
20 bridges
21 bridges
22 bridges
23 bridges
24 bridges
25 bridges
26 bridges
27 bridges
28 bridges
Before saving 2 dags: 7.19 minutes
Exiting 2 titles comparison: 7.23 minutes
Starting comparison of 2 titles: 7.23 minutes
JB: path1 = short_stories_dag_atlas/bill-the-bloodhound.pkl
title1 and its num of nodes: bill-the-bloodhound 317
title2 and its num of nodes: wiltons-holiday 249
product of numbers of nodes= 78933
Starting bridges: 7.23 minutes
SIMI_THRESHOLD: 3
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
Before saving 2 dags: 11.78 minutes
Exiting 2 titles comparison: 11.79 minutes
Starting comparison of 2 titles: 11.79 minutes
JB: path1 = short_stories_dag_atlas/extricating-young-gussie.pkl
JB: path2 = short_stories_dag_atlas/wiltons-holiday.pkl
title1 and its num of nodes: extricating-young-gussie 396
title2 and its num of nodes: wiltons-holiday 249
product of numbers of nodes= 98604
Starting bridges: 11.79 minutes
SIMI_THRESHOLD: 3
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
9 bridges
10 bridges
11 bridges
12 bridges
13 bridges
14 bridges
15 bridges
16 bridges
17 bridges
18 bridges
19 bridges
20 bridges
21 bridges
22 bridges
23 bridges
24 bridges
25 bridges
26 bridges
27 bridges
28 bridges
29 bridges
Before saving 2 dags: 16.83 minutes
Exiting 2 titles comparison: 16.88 minutes
Running it a second time gives:
Initiating DagAtlas object: 0.00 minutes
Starting comparison of 2 titles: 0.00 minutes
title1 and its num of nodes: bill-the-bloodhound 317
title2 and its num of nodes: extricating-young-gussie 396
product of numbers of nodes= 125532
Starting bridges: 0.00 minutes
SIMI_THRESHOLD: 3
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
9 bridges
10 bridges
11 bridges
12 bridges
13 bridges
14 bridges
15 bridges
16 bridges
17 bridges
18 bridges
19 bridges
20 bridges
21 bridges
22 bridges
23 bridges
24 bridges
25 bridges
26 bridges
27 bridges
28 bridges
Before saving 2 dags: 6.65 minutes
Exiting 2 titles comparison: 6.70 minutes
Starting comparison of 2 titles: 6.70 minutes
JB: path1 = short_stories_dag_atlas/bill-the-bloodhound.pkl
title1 and its num of nodes: bill-the-bloodhound 317
title2 and its num of nodes: wiltons-holiday 249
product of numbers of nodes= 78933
Starting bridges: 6.70 minutes
SIMI_THRESHOLD: 3
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
Before saving 2 dags: 10.86 minutes
Exiting 2 titles comparison: 10.91 minutes
Starting comparison of 2 titles: 10.91 minutes
JB: path1 = short_stories_dag_atlas/extricating-young-gussie.pkl
JB: path2 = short_stories_dag_atlas/wiltons-holiday.pkl
title1 and its num of nodes: extricating-young-gussie 396
title2 and its num of nodes: wiltons-holiday 249
product of numbers of nodes= 98604
Starting bridges: 10.91 minutes
SIMI_THRESHOLD: 3
1 bridges
2 bridges
3 bridges
4 bridges
5 bridges
6 bridges
7 bridges
8 bridges
9 bridges
10 bridges
11 bridges
12 bridges
13 bridges
14 bridges
15 bridges
16 bridges
17 bridges
18 bridges
19 bridges
20 bridges
21 bridges
22 bridges
23 bridges
24 bridges
25 bridges
26 bridges
27 bridges
28 bridges
29 bridges
Before saving 2 dags: 15.93 minutes
Exiting 2 titles comparison: 15.98 minutes
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1527255752,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYOIC7IY5PIN263CZYTXDOD37ANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

I already made a version of the movie script notebook myself and let it run successfully.
I discovered that the visualisation with:

Excellent. Can you please upload the notebook, and the pickled files, so
that I can include them in the repo. State at the beginning of the notebook
that you did it.
On Fri, Apr 28, 2023 at 5:33 PM Jan Bours @.***> wrote:

I already made a version of the movie script notebook myself and let it
run successfully.
I discovered that the visualisation with:
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1528117092,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYLHUPNPQY7FEBEROL3XDQZSBANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

Yes, I will upload the notebook and the pickled files tomorrow. At this moment the notebook is still running 🏃♂ 🏃♀ …

Thanks Jan.
On Fri, Apr 28, 2023 at 6:02 PM Jan Bours @.***> wrote:

Yes, I will upload the notebook and the pickled files tomorrow. At this
moment the notebook is still running 🏃♂ 🏃♀ …
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1528137438,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYOSB7CRD77LBVY2CCTXDQ45JANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

I share my results from my Mappa Mundi notebook on Google Colab (the notebook needs some editing; I will share the note itself later).

RESULTS FOR THE SHORT STORIES:
main1(reps_threshold=4, draw=True)
==================================
bill-the-bloodhound
{'(3,0)->(166,1)': 4, '(71,0)->(166,1)': 4, '(77,0)->(166,1)': 4, '(95,0)->(154,0)': 4, '(95,0)->(166,1)': 8, '(95,0)->(200,0)': 6, '(95,0)->(227,0)': 6, '(95,0)->(307,0)': 6, '(154,0)->(166,1)': 7, '(166,1)->(200,0)': 4, '(166,1)->(227,0)': 4, '(166,1)->(307,0)': 4}

MAP LEGEND
title: bill-the-bloodhound
arrow repetitions threshold: 4
number of arrows shown: 12
number of arrows dropped: 67
(3,0):
(FULL) If I simply said he was a detective, and let it go at that, I should be obtaining the reader's interest under false pretences.
(PART) simply said detective
(71,0):
(FULL) 'Sing,' said the agent.
(PART) said agent
(77,0):
(FULL) The boss wants you.
(PART) boss wants
(95,0):
(FULL) In the second place, to have to do his creeping about and spying practically in her presence--
(PART) place creeping
(154,0):
(FULL) That is what we have all been wondering all this time.'
(PART) wondering time
(166,1):
(FULL) Later, maybe, it was a courteous old retired colonel who stopped you and inquired the way to Trafalgar Square.
(PART) inquired way
(200,0):
(FULL) 'Never mind,' said Henry.
(PART) mind said
(227,0):
(FULL) Your boss said "Trail along."

(PART) said mascot
==================================
wiltons-holiday
{'(74,1)->(226,0)': 4, '(104,0)->(226,0)': 4, '(104,0)->(334,0)': 4, '(14,0)->(226,0)': 4, '(14,0)->(334,0)': 4, '(226,0)->(334,0)': 13, '(226,0)->(307,0)': 4, '(234,0)->(334,0)': 8, '(8,0)->(226,0)': 8, '(8,0)->(50,0)': 6, '(8,0)->(165,0)': 6, '(8,0)->(334,0)': 12, '(8,0)->(113,0)': 6, '(74,1)->(334,0)': 4, '(158,0)->(334,0)': 4, '(50,0)->(334,0)': 5, '(50,0)->(113,0)': 4, '(165,0)->(334,0)': 5, '(113,0)->(334,0)': 4}

MAP LEGEND
title: wiltons-holiday
arrow repetitions threshold: 4
number of arrows shown: 19
number of arrows dropped: 82
(8,0):
(FULL) Within two hours, then, of Clay's chat with Wilton, everyone in the place knew that, jolly and hearty as the new-comer might seem, there was that gnawing at his heart which made his outward cheeriness simply heroic.
(PART) chat place knew
(14,0):
(FULL) It is a thing he does not like to have known.
(PART) thing like known
(50,0):
(FULL) Some men said that she had fine eyes.
(PART) men said
(74,1):
(FULL) The only reason why anyone comes away from a summer at Marois Bay unbetrothed is because there are so many girls that he falls in love with that his holiday is up before he can, so to speak, concentrate.
(PART) girls falls love
(104,0):
(FULL) But then, I am nothing to write home about, whereas the smallest gleam of intelligence should have told Wilton that he was a kind of Ouida guardsman.
(PART) write home
(113,0):
(FULL) 'They do not marry girls like Mary.'
(PART) marry girls
(158,0):
(FULL) I cannot possibly tell her the whole thing was a fraud.
(PART) possibly tell thing fraud
(165,0):
(FULL) 'Make it a whole mind,' I said, 'and you win out.'
(PART) mind said
(226,0):
(FULL) 'I must go round the other way,' she said.
(PART) way said
(234,0):
(FULL) It was not fair, he felt, that she should jockey at the start in this way and keep him hanging about here catching cold.
(PART) jockey start way
(307,0):
(FULL) The tide does not come up any farther.
(PART) tide come farther
(334,0):
(FULL) 'There is a jelly-fish just where you are going to sit,' said Wilton.
(PART) jelly fish going sit said
==================================
extricating-young-gussie
{'(272,0)->(275,0)': 16, '(272,0)->(416,0)': 5, '(275,0)->(416,0)': 5, '(13,0)->(272,0)': 4, '(13,0)->(275,0)': 4, '(155,0)->(272,0)': 7, '(155,0)->(275,0)': 7, '(184,0)->(272,0)': 7, '(184,0)->(275,0)': 7, '(205,3)->(272,0)': 7, '(205,3)->(275,0)': 7, '(122,0)->(272,0)': 5, '(122,0)->(275,0)': 5, '(201,0)->(272,0)': 5, '(201,0)->(275,0)': 5, '(86,1)->(272,0)': 4, '(86,1)->(275,0)': 4}

MAP LEGEND
title: extricating-young-gussie
arrow repetitions threshold: 4
number of arrows shown: 17
number of arrows dropped: 202
(13,0):
(FULL) She bosses my cousin, Gussie Mannering-Phipps.
(PART) bosses cousin
(86,1):
(FULL) What is bred in the bone will come out in the wash, and all that kind of thing, you know.'
(PART) kind thing know
(122,0):
(FULL) You cannot lose your way.
(PART) lose way
(155,0):
(FULL) I cannot express it better than by saying that the thought uppermost in my mind, as I walked about the place they call Times Square, was that there were three thousand miles of deep water between me and my Aunt Agatha.
(PART) walked place
(184,0):
(FULL) 'What do you want in this place?
(PART) want place
(201,0):
(FULL) I am told that way back in the nineties he only had two.
(PART) told way
(205,3):
(FULL) There was the deuce of a rush for him, but Gussie had got away to a good start, and the rest of the singers, dancers, jugglers, acrobats, and refined sketch teams seemed to recognize that he had won the trick, for they ebbed back into their places again, and Gussie and I went into the inner room.
(PART) ebbed places
(272,0):
(FULL) At first poor old Gussie asked him to stop, but the chappie said, No, it was always done.
(PART) chappie said
(275,0):
(FULL) And the chappie said to Gussie, 'There you are!'
(PART) chappie said
(416,0):
(FULL) I do not like to say such things of any aunt of mine, or I would go further and put it on record that she was giggling.
(PART) like things aunt

RESULTS FOR THE MOVIE SCRIPTS:
main1(reps_threshold=100, draw=True)
(setting the reps_threshold to 4 will not generate graphical output!)
==================================
wall-e
{'(147,0)->(2031,0)': 107, '(147,0)->(2281,0)': 107, '(147,0)->(2295,0)': 107, '(147,0)->(878,0)': 101, '(147,0)->(1559,0)': 130, '(147,0)->(1645,0)': 140, '(147,0)->(2647,0)': 140, '(147,0)->(2375,0)': 108}

MAP LEGEND
title: wall-e
arrow repetitions threshold: 100
number of arrows shown: 8
number of arrows dropped: 23782
(147,0):
(FULL) The door shuts just as the storm hits.
(PART) door shuts
(878,0):
(FULL) EMERGENCY LINES appear on the floor.
(PART) appear floor
(1559,0):
(FULL) They HEAR something hover into the room.
(PART) Hear hover room
(1645,0):
(FULL) He opens his compactor door.
(PART) opens compactor door
(2031,0):
(FULL) Auto emits an ELECTRONIC COMMAND.
(PART) Auto emits
(2281,0):
(FULL) Auto activates every steward on the Axiom.
(PART) Auto activates steward
(2295,0):
(FULL) Auto...You hear me?...
(PART) Auto hear
(2375,0):
(FULL) Turns off the vid-com.
(PART) Turns vid com
(2647,0):
(FULL) Suddenly the junk jingles and clinks as the back door opens.
(PART) door opens
==================================
up
{'(403,0)->(550,0)': 112, '(403,0)->(558,0)': 112, '(403,0)->(625,0)': 112, '(403,0)->(484,0)': 102, '(403,0)->(2107,0)': 102, '(484,0)->(550,0)': 112, '(484,0)->(558,0)': 112, '(484,0)->(625,0)': 112, '(484,0)->(2107,0)': 102, '(253,0)->(550,0)': 126, '(253,0)->(558,0)': 126, '(253,0)->(625,0)': 126, '(253,0)->(403,0)': 107, '(253,0)->(484,0)': 107, '(253,0)->(2107,0)': 107, '(550,0)->(558,0)': 126, '(550,0)->(625,0)': 126, '(550,0)->(2107,0)': 107, '(558,0)->(625,0)': 126, '(558,0)->(2107,0)': 107, '(625,0)->(2107,0)': 107, '(253,0)->(1660,0)': 130, '(550,0)->(1660,0)': 130, '(558,0)->(1660,0)': 130, '(625,0)->(1660,0)': 130}

MAP LEGEND
title: up
arrow repetitions threshold: 100
number of arrows shown: 25
number of arrows dropped: 26351
(253,0):
(FULL) Carl looks out the window.
(PART) looks window
(403,0):
(FULL) Carl listens through the door.
(PART) listens door
(484,0):
(FULL) Carl shuts his door.
(PART) shuts door
(550,0):
(FULL) Carl looks out the window and LAUGHS.
(PART) looks window
(558,0):
(FULL) Carl looks out his window.
(PART) looks window
(625,0):
(FULL) Russell looks out the window in front of him.
(PART) looks window
(1660,0):
(FULL) A "Botanist" cataloguing plants.
(PART) cataloguing plants
(2107,0):
(FULL) Behind them Carl shuts the door.

(PART) sheep run
(79,0):
(FULL) Andy and Woody enter the room.
(PART) enter room
(168,0):
(FULL) I am going for fearsome here, but I just do not feel it.
(PART) going fearsome
(288,1):
(FULL) Woody jumps up onto Andy's bed and turns to the Sargent on the nightstand.
(PART) turns nightstand
(321,0):
(FULL) I have got Cool Ranch and Barbeque -- The door opens and Mrs. Davis' foot comes down hard on top of a soldier.
(PART) door opens
(402,0):
(FULL) Rex grabs a leg of the nightstand and shakes it, making the monitor drop to the floor.
(PART) grabs leg nightstand
(429,0):
(FULL) Outta my way!
(PART) Outta way
(566,0):
(FULL) Woody walks over to Bo Peep.
(PART) Woody walks
(751,0):
(FULL) Buzz closes his helmet.
(PART) closes helmet
(965,0):
(FULL) Andy enters the room and heads straight for the desk where Woody is lying.
(PART) enters room
(992,1):
(FULL) Buzz races out of the bush, and with a mighty leap, grabs the rear fender of the van as it pulls out of the driveway.
(PART) pulls driveway
(999,1):
(FULL) Rex throws the barrel down and runs back to the ledge.
(PART) runs ledge
(1166,0):
(FULL) Woody is gone.
(PART) Woody gone
(1173,0):
(FULL) We need a way to get inside.
(PART) need way inside
(1312,1):
(FULL) Sid leaps off the skateboard and walks up the front path towards the front door.
(PART) walks path door
(1330,0):
(FULL) The dog begrudgingly obeys.
(PART) dog begrudgingly obeys
(1385,0):
(FULL) He opens the door.
(PART) opens door
(1450,0):
(FULL) Rex turns off the flashlight.
(PART) turns flashlight
(1516,0):
(FULL) Shield your eyes!
(PART) Shield eyes
(1637,0):
(FULL) Hannah walks into the entryway, and not seeing Buzz, steps on him.
(PART) walks entryway
(1751,0):
(FULL) We are going to get outta here, Buzz -- Buzz?
(PART) going outta
(2028,0):
(FULL) Woody looks up to see...
(PART) Woody looks
(2066,0):
(FULL) Woody lifts himself out from under the rubble.
(PART) Woody lifts rubble
(2077,0):
(FULL) The dog barrels down the hallway straight for Woody.
(PART) dog barrels hallway straight
(2088,0):
(FULL) have appeared all around the room.
(PART) appeared room
(2096,0):
(FULL) I have got to savedag him!
(PART) got savedag
(2159,0):
(FULL) Hannah walks to the front door...
(PART) walks door
(2237,0):
(FULL) Sid walks over to Woody and picks him up.
(PART) Sid walks
(2280,0):
(FULL) Buzz extends his hand to Woody.
(PART) extends hand
(2299,0):
(FULL) Woody jumps down and runs back for Buzz.
(PART) Woody jumps
(2319,0):
(FULL) Woody looks behind him to see...
(PART) Woody looks
(2353,0):
(FULL) Woody pulls the RC Car and its remote out of the second box.
(PART) Woody pulls
(2453,0):
(FULL) The car slows even more.
(PART) car slows

These are the pickle files:
MM PICKLE FILES.zip
It seems that my DAGs are rather different then yours. They are more complex don't they?
Perhaps my Wordnet gives different results as we already saw when comparing the output of similarity.py !
Perhaps you have a different version of Wordnet as you already hinted to.
What is your version of Wordnet? I will look into my version and report it later.
Please comment on my results !
A question: how could one verify the resulting DAGs?

As to the vesion of nltk and Wordnet:
JB 29-04-2023
Dit is een cruciale toevoeging !!!!!
import nltk
nltk.download('wordnet')
[nltk_data] Downloading package wordnet to /root/nltk_data...
True
nltk.version
3.8.1
WHAT VERSION OF WORDNET ?
https://wordnet.princeton.edu/download/current-version
https://wordnet.princeton.edu/related-projects
Python
Natural Language Toolkit has taken over the development of pywordnet.
There is now a Python package, nltk_lite.wordnet,
which incorporates pywordnet and which supports WordNet 2.1. It is included in NLTK Lite.
import wordnet as wn
from nltk import wordnet as wn
wn # <module 'nltk.stem.wordnet' from '/usr/local/lib/python3.10/dist-packages/nltk/stem/wordnet.py'>

Thanks Jan
On Sat, Apr 29, 2023 at 8:49 AM Jan Bours @.***> wrote:

As to the vesion of nltk and Wordnet:
JB 29-04-2023
Dit is een cruciale toevoeging !!!!!
import nltk
nltk.download('wordnet')
[nltk_data] Downloading package wordnet to /root/nltk_data...
True
nltk.version
3.8.1
WHAT VERSION OF WORDNET ?
https://wordnet.princeton.edu/download/current-version
https://wordnet.princeton.edu/related-projects
Python
Natural Language Toolkit has taken over the development of pywordnet.
There is now a Python package, nltk_lite.wordnet,
which incorporates pywordnet and which supports WordNet 2.1. It is
included in NLTK Lite.
import wordnet as wn
from nltk import wordnet as wn
wn # <module 'nltk.stem.wordnet' from
'/usr/local/lib/python3.10/dist-packages/nltk/stem/wordnet.py'>
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1528780716,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYL6INREAUNYFEXJ7W3XDUE4XANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

I am sharing my preliminary Colab notebook for MM on short stories with you:
navigating_short_stories_Adapted by Jan Bours.ipynb
https://colab.research.google.com/drive/1JzgTPYScFjjOs-dc-Rp-a9dSwYb7-0Cr?authuser=1#scrollTo=b3bf4efd-985c-4ab1-be2d-2d99d4561b70
I have shared it with you using your email adress:
<EMAIL_ADDRESS>Also I have attached it to this post as an attachment, in navigating_short_stories_Adapted_by_Jan_Bours.zip
navigating_short_stories_Adapted_by_Jan_Bours.zip
I have done also some experiments with Cython in this notebook, in the hope of speeding things up in DagAtlas.py, because there is a heavy bottleneck with all the NLTK POS tagging going on. Cython helps a little bit, but I have not used all the tricks that are in Cython, so that could be optimized even further I think.

Thanks Jan.
As I explained in the white paper, I believe a dramatic improvement will
come when this is implemented with LLMs.
On Sun, Apr 30, 2023 at 3:06 PM Jan Bours @.***> wrote:

I am sharing my preliminary Colab notebook for MM on short stories with
you:
navigating_short_stories_Adapted by Jan Bours.ipynb
https://colab.research.google.com/drive/1JzgTPYScFjjOs-dc-Rp-a9dSwYb7-0Cr?authuser=1#scrollTo=b3bf4efd-985c-4ab1-be2d-2d99d4561b70
I have shared it with you using your email adress:
@.***
Also I have attached it to this post as an attachment, in
navigating_short_stories_Adapted_by_Jan_Bours.zip
navigating_short_stories_Adapted_by_Jan_Bours.zip
https://github.com/rrtucci/mappa_mundi/files/11362111/navigating_short_stories_Adapted_by_Jan_Bours.zip
I have done also some experiments with Cython in this notebook, in the
hope of speeding things up in DagAtlas.py, because there is a heavy
bottleneck with all the NLTK POS tagging going on. Cython helps a little
bit, but I have not used all the tricks that are in Cython, so that could
be optimized even further I think.
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1529117577,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYK6UPFF2QK43SHBDI3XD2Z3PANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

I think that is a good conclusion! Makes me wonder: what is your next step in developing this interesting idea of DEFT and MM? I am curious about that!

If I can't get any LLM company interested, plan B is to do it myself. Maybe
I can redo Mappa Mundi using a hacked version of nanoGPT.
https://github.com/karpathy/nanoGPT
On Mon, May 1, 2023 at 2:54 AM Jan Bours @.***> wrote:

I think that is a good conclusion! Makes me wonder: what is your next step
in developing this interesting idea of DEFT and MM? I am curious about that!
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1529412033,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYJ2HDKQI5SPR3ZYQG3XD5MZHANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

Perhaps then this is interesting for you? :
H2Oai releases Fully OpenSourced GPT | by Dr. Mandar Karhade, MD. PhD. | Apr, 2023 | Towards AI !!!!!
https://pub.towardsai.net/h2oai-releases-fully-opensourced-gpt-9bdcc2fc1f6d

Also I am looking into the tools that Hugging Face has for doing POS Tagging and sentence similarity, which seem very promising, also when it comes to speed!
For exmaple, see these links:
https://huggingface.co/tasks/sentence-similarity
https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
https://www.sbert.net/docs/usage/semantic_textual_similarity.html
https://www.sbert.net/examples/applications/paraphrase-mining/README.html
This last one seems very interesting:
"
Paraphrase Mining
Paraphrase mining is the task of finding paraphrases (texts with identical / similar meaning) in a large corpus of sentences. In Semantic Textual Similarity we saw a simplified version of finding paraphrases in a list of sentences. The approach presented there used a brute-force approach to score and rank all pairs.
However, as this has a quadratic runtime, it fails to scale to large (10,000 and more) collections of sentences.
For larger collections, util offers the paraphrase_mining function that can be used like this:
from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer('all-MiniLM-L6-v2')
Single list of sentences - Possible tens of thousands of sentences
sentences = ['The cat sits outside',
'A man is playing guitar',
'I love pasta',
'The new movie is awesome',
'The cat plays in the garden',
'A woman watches TV',
'The new movie is so great',
'Do you like pizza?']
paraphrases = util.paraphrase_mining(model, sentences)
for paraphrase in paraphrases[0:10]:
score, i, j = paraphrase
print("{} \t\t {} \t\t Score: {:.4f}".format(sentences[i], sentences[j], score))
The paraphrase_mining()-method accepts the following parameters:
sentence_transformers.util.paraphrase_mining(model, sentences: List[str], show_progress_bar: bool = False, batch_size: int = 32, *args, **kwargs)
Given a list of sentences / texts, this function performs paraphrase mining. It compares all sentences against all other sentences and returns a list with the pairs that have the highest cosine similarity score.

Parameters

        model – SentenceTransformer model for embedding computation

        sentences – A list of strings (texts or sentences)

        show_progress_bar – Plotting of a progress bar

        batch_size – Number of texts that are encoded simultaneously by the model

        query_chunk_size – Search for most similar pairs for #query_chunk_size at the same time. Decrease, to lower memory footprint (increases run-time).

        corpus_chunk_size – Compare a sentence simultaneously against #corpus_chunk_size other sentences. Decrease, to lower memory footprint (increases run-time).

        max_pairs – Maximal number of text pairs returned.

        top_k – For each sentence, we retrieve up to top_k other sentences

        score_function – Function for computing scores. By default, cosine similarity.

Returns

    Returns a list of triplets with the format [score, id1, id2]

Instead of computing all pairwise cosine scores and ranking all possible, combintations, the approach is a bit more complex (and hence efficient). We chunk our corpus into smaller pieces, which is defined by query_chunk_size and corpus_chunk_size. For example, if we set query_chunk_size=1000, we search paraphrases for 1,000 sentences at a time in the remaining corpus (all other sentences). However, the remaining corpus is also chunked, for example, if we set corpus_chunk_size=10000, we look for paraphrases in 10k sentences at a time.
If we pass a list of 20k sentences, we will chunk it to 20x1000 sentences, and each of the query is compared first against sentences 0-10k and then 10k-20k.
This is done to reduce the memory requirement. Increasing both values improves the speed, but increases also the memory requirement.
The next critical thing is finding the pairs with the highest similarities. Instead of getting and sorting all n^2 pairwise scores, we take for each query only the top_k scores. So with top_k=100, we find at most 100 paraphrases per sentence per chunk. You can play around with top_k to the ensure a certain behaviour.
So for example, with
paraphrases = util.paraphrase_mining(model, sentences, corpus_chunk_size=len(sentences), top_k=1)
You will get for each sentence only the one most other relevant sentence. Note, if B is the most similar sentence for A, A must not be the most similar sentence for B. So it can happen that the returned list contains entries like (A, B) and (B, C).
The final relevant parameter is max_pairs, which determines the maximum number of paraphrase pairs you like to get returned. If you set it to e.g. max_pairs=100, you will not get more than 100 paraphrase pairs returned. Usually, you get fewer pairs returned as the list is cleaned of duplicates, e.g., if it contains (A, B) and (B, A), then only one is returned.
"
I think this could speed up your similarity calculations in DagAtlas.py dramatically !
Tell me what you think about it !
I am curious ....

This is really helpful and relevant. Thanks for sharing it.
On Tue, May 2, 2023 at 10:08 AM Jan Bours @.***> wrote:

Also I am looking into the tools that Hugging Face has for doing POS
Tagging and sentence similarity, which seem very promising, also when it
comes to speed!
For exmaple, see these links:
https://huggingface.co/tasks/sentence-similarity
https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
https://www.sbert.net/docs/usage/semantic_textual_similarity.html
https://www.sbert.net/examples/applications/paraphrase-mining/README.html
This last one seems very interesting:
"
Paraphrase Mining
Paraphrase mining is the task of finding paraphrases (texts with identical
/ similar meaning) in a large corpus of sentences. In Semantic Textual
Similarity
https://www.sbert.net/docs/usage/semantic_textual_similarity.html we
saw a simplified version of finding paraphrases in a list of sentences. The
approach presented there used a brute-force approach to score and rank all
pairs.
However, as this has a quadratic runtime, it fails to scale to large
(10,000 and more) collections of sentences.
For larger collections, util offers the paraphrase_mining function that
can be used like this:
from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer('all-MiniLM-L6-v2')
Single list of sentences - Possible tens of thousands of sentences
sentences = ['The cat sits outside',
'A man is playing guitar',
'I love pasta',
'The new movie is awesome',
'The cat plays in the garden',
'A woman watches TV',
'The new movie is so great',
'Do you like pizza?']
paraphrases = util.paraphrase_mining(model, sentences)
for paraphrase in paraphrases[0:10]:
score, i, j = paraphrase
print("{} \t\t {} \t\t Score: {:.4f}".format(sentences[i], sentences[j],
score))
The paraphrase_mining()-method accepts the following parameters:
sentence_transformers.util.paraphrase_mining(model, sentences: List[str],
show_progress_bar: bool = False, batch_size: int = 32, *args, **kwargs)
Given a list of sentences / texts, this function performs paraphrase mining. It compares all sentences against all other sentences and returns a list with the pairs that have the highest cosine similarity score.
Parameters
    model – SentenceTransformer model for embedding computation

    sentences – A list of strings (texts or sentences)

    show_progress_bar – Plotting of a progress bar

    batch_size – Number of texts that are encoded simultaneously by the model

    query_chunk_size – Search for most similar pairs for #query_chunk_size at the same time. Decrease, to lower memory footprint (increases run-time).

    corpus_chunk_size – Compare a sentence simultaneously against #corpus_chunk_size other sentences. Decrease, to lower memory footprint (increases run-time).

    max_pairs – Maximal number of text pairs returned.

    top_k – For each sentence, we retrieve up to top_k other sentences

    score_function – Function for computing scores. By default, cosine similarity.

Returns
Returns a list of triplets with the format [score, id1, id2]

Instead of computing all pairwise cosine scores and ranking all possible,
combintations, the approach is a bit more complex (and hence efficient). We
chunk our corpus into smaller pieces, which is defined by query_chunk_size
and corpus_chunk_size. For example, if we set query_chunk_size=1000, we
search paraphrases for 1,000 sentences at a time in the remaining corpus
(all other sentences). However, the remaining corpus is also chunked, for
example, if we set corpus_chunk_size=10000, we look for paraphrases in 10k
sentences at a time.
If we pass a list of 20k sentences, we will chunk it to 20x1000 sentences,
and each of the query is compared first against sentences 0-10k and then
10k-20k.
This is done to reduce the memory requirement. Increasing both values
improves the speed, but increases also the memory requirement.
The next critical thing is finding the pairs with the highest
similarities. Instead of getting and sorting all n^2 pairwise scores, we
take for each query only the top_k scores. So with top_k=100, we find at
most 100 paraphrases per sentence per chunk. You can play around with top_k
to the ensure a certain behaviour.
So for example, with
paraphrases = util.paraphrase_mining(model, sentences,
corpus_chunk_size=len(sentences), top_k=1)
You will get for each sentence only the one most other relevant sentence.
Note, if B is the most similar sentence for A, A must not be the most
similar sentence for B. So it can happen that the returned list contains
entries like (A, B) and (B, C).
The final relevant parameter is max_pairs, which determines the maximum
number of paraphrase pairs you like to get returned. If you set it to e.g.
max_pairs=100, you will not get more than 100 paraphrase pairs returned.
Usually, you get fewer pairs returned as the list is cleaned of duplicates,
e.g., if it contains (A, B) and (B, A), then only one is returned.
"
I think this could speed up your similarity calculations in DagAtlas.py
dramatically !
Tell me what you think about it !
I am curious ....
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1531548871,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYNOHRXPQSQYTP23P4DXEEIM3ANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

Especially because it is free to use ! No API costs !
I have prepared a demo COLAB notebook in which I perform POS Tagging and determine similarities on the Short story texts form your github (short_stories_simp, 3 text files) and Movie scripts (https://github.com/rrtucci/mappa_mundi/tree/master/m_scripts_simp, 3 texts).
On all these instances I do similarities between all the entries in those files (which are a large number of comparisons as you poited out !). All this happens with a great speed with those free Huggingface tools!
I have attached a zip file with my COLAB Notebook that does this taht I share with you.
Name of the Notebook:
POS_TAGGING_HUGGINGFACE_BERT_base_multilingual_cased_finetuned_for_Part_of_Speech_tagging_COLAB_COU_PYTHON3_tvscitechtalk(3).ipynb
Name of ZIP file:
POS_TAGGING_HUGGINGFACE_BERT_base_multilingual_cased_finetuned_for_Part_of_Speech_tagging_COLAB_COU_PYTHON3_tvscitechtalk(3).zip
(attached to this post)
POS_TAGGING_HUGGINGFACE_BERT_base_multilingual_cased_finetuned_for_Part_of_Speech_tagging_COLAB_COU_PYTHON3_tvscitechtalk(3).zip
Tell me what you think about it !
I am curious ....

Wow, this is amazing. Good job! Thanks for sharing.
On Tue, May 2, 2023 at 6:35 PM Jan Bours @.***> wrote:

Especially because it is free to use ! No API costs !
I have prepared a demo COLAB notebook in which I perform POS Tagging and
determine similarities on the Short story texts form your github (
short_stories_simp
https://github.com/rrtucci/mappa_mundi/tree/master/short_stories_simp,
3 text files) and Movie scripts (
https://github.com/rrtucci/mappa_mundi/tree/master/m_scripts_simp, 3
texts).
On all these instances I do similarities between all the entries in those
files (which are a large number of comparisons as you poited out !). All
this happens with a great speed with those free Huggingface tools!
I have attached a zip file with my COLAB Notebook that does this taht I
share with you.
Name of the Notebook:
POS_TAGGING_HUGGINGFACE_BERT_base_multilingual_cased_finetuned_for_Part_of_Speech_tagging_COLAB_COU_PYTHON3_tvscitechtalk(3).ipynb
Name of ZIP file:
POS_TAGGING_HUGGINGFACE_BERT_base_multilingual_cased_finetuned_for_Part_of_Speech_tagging_COLAB_COU_PYTHON3_tvscitechtalk(3).zip
(attached to this post)
POS_TAGGING_HUGGINGFACE_BERT_base_multilingual_cased_finetuned_for_Part_of_Speech_tagging_COLAB_COU_PYTHON3_tvscitechtalk(3).zip
https://github.com/rrtucci/mappa_mundi/files/11378133/POS_TAGGING_HUGGINGFACE_BERT_base_multilingual_cased_finetuned_for_Part_of_Speech_tagging_COLAB_COU_PYTHON3_tvscitechtalk.3.zip
Tell me what you think about it !
I am curious ....
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1532240690,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYI3SNISE7DEQ2GSMW3XEGD3DANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

Very good, but needs to be cleaned up. I don't like to use jupyter
notebooks as an IDE. Therefore, I put most of my function definitions in
.py files. Ideally, the notebooks just call functions, not define them.
That is my philosophy.
Can I do all of this on Colab for free, or do I have to pay extra
for exceeding the free limits?

Do you work for a company interested in this, or are you in academia?
The reason I ask is because I don't think my computer is powerful enough to
do any of this (I only have 8GB RAM and no GPU)
On Tue, May 2, 2023 at 7:38 PM Robert Tucci @.***> wrote:

Wow, this is amazing. Good job! Thanks for sharing.
On Tue, May 2, 2023 at 6:35 PM Jan Bours @.***> wrote:

Especially because it is free to use ! No API costs !
I have prepared a demo COLAB notebook in which I perform POS Tagging and
determine similarities on the Short story texts form your github (
short_stories_simp
https://github.com/rrtucci/mappa_mundi/tree/master/short_stories_simp,
3 text files) and Movie scripts (
https://github.com/rrtucci/mappa_mundi/tree/master/m_scripts_simp, 3
texts).
On all these instances I do similarities between all the entries in those
files (which are a large number of comparisons as you poited out !). All
this happens with a great speed with those free Huggingface tools!
I have attached a zip file with my COLAB Notebook that does this taht I
share with you.
Name of the Notebook:
POS_TAGGING_HUGGINGFACE_BERT_base_multilingual_cased_finetuned_for_Part_of_Speech_tagging_COLAB_COU_PYTHON3_tvscitechtalk(3).ipynb
Name of ZIP file:
POS_TAGGING_HUGGINGFACE_BERT_base_multilingual_cased_finetuned_for_Part_of_Speech_tagging_COLAB_COU_PYTHON3_tvscitechtalk(3).zip
(attached to this post)
POS_TAGGING_HUGGINGFACE_BERT_base_multilingual_cased_finetuned_for_Part_of_Speech_tagging_COLAB_COU_PYTHON3_tvscitechtalk(3).zip
https://github.com/rrtucci/mappa_mundi/files/11378133/POS_TAGGING_HUGGINGFACE_BERT_base_multilingual_cased_finetuned_for_Part_of_Speech_tagging_COLAB_COU_PYTHON3_tvscitechtalk.3.zip
Tell me what you think about it !
I am curious ....
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1532240690,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYI3SNISE7DEQ2GSMW3XEGD3DANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

Yes, I did this all for free.
On Google COLAB the total running time is very short (the loading from Huggingface components itself takes some time, but the core calculations are fast!). So it can be carried out for free.
Also this can be carried out on slow computers, because COLAB notebooks run in the cloud in your browser! But my notebook could also run locally in a Jupyter Notebook, I will try that.
I am a Data Scientist working industry (almost retired) with a keen interest in al things causal (Causal Inference and Causal Discovery).
Your idea of extracting Causal information from texts is especially interesting, because it could help large language models (LLMs) become causally aware and become more robust. It is the implementation of the scientific method as you mentioned in your paper!
Just yesterday there was another paper about the emergence of Causal capabilities of LLMs:
[2305.00050] Causal Reasoning and Large Language Models: Opening a New Frontier for Causality
https://arxiv.org/abs/2305.00050
This paper is a strong conformation of the (emergent) causal capabilities of LLMs.
I think that it is plausible that LLMs get this capability because during training they have read about all kinds of causal information and using that knowledge at inference time.
Boris Sobolev says this about it:
https://twitter.com/soboleffspaces/status/1653495197597323264?s=46&t=8nmTR-4kmZaFFBCGObS6fA
“‘text about data’ IS written knowledge
I tried GPT4 for assembling a DAG from domain literature. Works great (refs hallucinated though). It even produced code to plot the DAG in any language asked. Haven't asked to find MSAS yet but don't think it'll be an issue. “ (!)
…
And there is this paper :
Are Emergent Abilities of Large Language Models a Mirage?
https://arxiv.org/abs/2304.15004
Which places a question mark ❓ to the emergent capabilities! At least, that is what I am getting from superficially reading this article.
I think and agree with you that the ability to ground the knowledge obtained by LLMs in Causal knowledge would be a groundbreaking step in the development of LLMs!

Just discovered this software on "sentence simplification". Sounds like it
might vastly improve the crude sentence simplifier that Mappa Mundi is
currently using
https://github.com/imohammad12/GRS
On Wed, May 3, 2023 at 2:02 AM Jan Bours @.***> wrote:

Yes, I did this all for free.
On Google COLAB the total running time is very short (the loading from
Huggingface components itself takes some time, but the core calculations
are fast!). So it can be carried out for free.
Also this can be carried out on slow computers, because COLAB notebooks
run in the cloud in your browser! But my notebook could also run locally in
a Jupyter Notebook, I will try that.
I am a Data Scientist working industry (almost retired) with a keen
interest in al things causal (Causal Inference and Causal Discovery).
Your idea of extracting Causal information from texts is especially
interesting, because it could help large language models (LLMs) become
causally aware and become more robust. It is the implementation of the
scientific method as you mentioned in your paper!
Just yesterday there was another paper about the emergence of Causal
capabilities of LLMs:
[2305.00050] Causal Reasoning and Large Language Models: Opening a New
Frontier for Causality
https://arxiv.org/abs/2305.00050
This paper is a strong conformation of the (emergent) causal capabilities
of LLMs.
I think that it is plausible that LLMs get this capability because during
training they have read about all kinds of causal information and using
that knowledge at inference time.
Boris Sobolev says this about it:
https://twitter.com/soboleffspaces/status/1653495197597323264?s=46&t=8nmTR-4kmZaFFBCGObS6fA
“‘text about data’ IS written knowledge
I tried GPT4 for assembling a DAG from domain literature. Works great
(refs hallucinated though). It even produced code to plot the DAG in any
language asked. Haven't asked to find MSAS yet but don't think it'll be an
issue. “ (!)
…
And there is this paper :
Are Emergent Abilities of Large Language Models a Mirage?
https://arxiv.org/abs/2304.15004
Which places a question mark ❓ to the emergent capabilities! At least,
that is what I am getting from superficially reading this article.
I think and agree with you that the ability to ground the knowledge
obtained by LLMs in Causal knowledge would be a groundbreaking step in the
development of LLMs!
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1532487664,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYIFDWK2CLUZ525RDFDXEHYGNANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

My dream is if we could start a project at github/huggingface, called Mappa
Mundi V2 , run by us two and others, with the goal of doing ALL 1100 movie
scripts from the IMSDb website.
We would have to replace the sentence simplification and sentence
similarity portions of the old Mappa Mundi by these speedier, more
precise versions.
On Wed, May 3, 2023 at 4:01 AM Jan Bours @.***> wrote:

Looks promising indeed. If you can get your DagAtlas algorithm more
speedy, then you can produce a lot more DAGs from texts and that will make
it more interesting fro practical use cases, would not it?
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1532604545,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYLUAPA7UWNMX2RGETTXEIGEVANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>

That certainly would be a nice project! Within a few weeks I hope to have more time to look into the exact working principles of the Mappa Mundi algorithm, because not everything is totally clear to me up till now.
We have to be sure that the working principles are sound of course!
I will come back to that!

Okay. Thanks.
On Wed, May 3, 2023 at 9:03 AM Jan Bours @.***> wrote:

That certainly would be a nice project! Within a few weeks I hope to have
more time to look into the exact working principles of the Mappa Mundi
algorithm, because not everything is totally clear to me up till now.
We have to be sure that the working principles are sound of course!
I will come back to that!
—
Reply to this email directly, view it on GitHub
https://github.com/rrtucci/mappa_mundi/issues/1#issuecomment-1532995674,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADZWTYM2FA5JZ6YHUMSGPS3XEJJSHANCNFSM6AAAAAAXMWE3C4
.
You are receiving this because you commented.Message ID:
@.***>


Fix: tests after update package-lock

Description
This PR fix tests after update the package-lock
Motivation and Context
Me and my friend are still using the v2 of this components, and my friend try to fix<EMAIL_ADDRESS>warning on this PR (https://github.com/YouCanBookMe/react-datetime/pull/674), but he tests are broken, that is because some dependency had updates that changes a little the assets of tests. So I create this PR to fix the tests first.
Checklist

[ ] I have not included any built dist files (us maintainers do that prior to a new release)
[ ] I have added tests covering my changes
[ ] All new and existing tests pass
[ ] My changes required the documentation to be updated
  [ ] I have updated the documentation accordingly
  [ ] I have updated the TypeScript 1.8 type definitions accordingly
  [ ] I have updated the TypeScript 2.0+ type definitions accordingly

Thanks for the PR!

Thank you for maintain this project.


Array field that includes a gallery with hasMany set to true crashes the app

Link to reproduction
No response
Describe the Bug
Having an upload filed with hasMany set to true in an array field causes the app to crash.
Please see the collection below:
export const Product: CollectionConfig = {
  slug: 'product',
  admin: {
    group: 'Products',
    useAsTitle: 'title',
  },
  fields: [
    {
      name: 'title',
      type: 'text',
      required: true,
    },
    {
      name: 'variants',
      type: 'array',
      fields: [
        {
          name: 'code',
          type: 'text',
        },
        {
          name: 'name',
          type: 'text',
        },
        {
          name: 'gallery',
          type: 'upload',
          relationTo: 'media',
          hasMany: true,
          required: true,
          maxDepth: 2,
        },
      ],
    },
  ],
}

This is how it looks in the admin app:

But as soon as I click "Add Variant" the app crashes with the following error:

Interestingly, if I add an empty record in the database for the "variant" field, I can add images with no issue:

To Reproduce
I have a minimal repo to show the issue: https://github.com/bmamouri/payload-bug
Payload Version
3.0.0-beta.91
Adapters and Plugins
No response

I have just reliased that this has nothing to do with array field. Even if the field does is not in array and is inside another collection this will happen:
Products
export const Product: CollectionConfig = {
  slug: 'product',
  fields: [
    {
      name: 'title',
      type: 'text',
      required: true,
    },
    {
      name: 'variants',
      type: 'relationship',
      relationTo: 'variant' as CollectionSlug,
      hasMany: true,
    },
  ],
}

Variants
import { CollectionConfig, CollectionSlug } from 'payload'

export const Variant: CollectionConfig = {
  slug: 'variant',
  fields: [
    {
      name: 'code',
      type: 'text',
    },
    {
      name: 'name',
      type: 'text',
    },
    {
      name: 'gallery',
      type: 'upload',
      relationTo: 'media',
      hasMany: true, // <= Adding this line will crash the app
    },
  ],
}

I am closing as this issue is fixed in 3.0.0-beta.95. Thanks Payload CMS team. 🙏

I've noticed that the bug is reproducing if you have Global that includes a field with type 'array', that consists of a field with type 'relation' that refers to collection.
The error:
ERROR: Error: cannot connect to Postgres. Details: Cannot read properties of undefined (reading 'Symbol(drizzle:Name)')

My simplified config:
const Navigation: GlobalConfig = {
  slug: 'navigation',
  fields: [
    {
      name: 'areas',
      type: 'array',
      fields: [
        {
          name: 'nav',
          type: 'relationship',
          relationTo: 'navigationItems', // my collection slug
          localized: true,
          hasMany: true,
        },
      ],
    },
  ],
};

The versions of packages:
 "payload": "3.0.0-beta.98",

Could this issue be reopened?


Reduce run-time for Sequence Encoder/Generator unit test

Code Pull Requests
Summary of changes:

Add Sequence encoder test that confirm correctness of generated output
Add Generator decoder test that confirm correctness of generated output
Reduce parameter combinations when generating test cases for Sequence Encoder/Sequence Generator decoder that execute experiment_cli()

Before and after results on testing this PR locally:

Metrics For Entire Test Suite
Before PR
After PR

Generated Tests
895
718

Elapsed Time (mm:ss)
33:15
27:38

Overall Test Coverage
81%
81%

Note:

Horvord test (test_horovod.py) fails because there is no local Horovod setup
After the change one of the graph execution test fails (test_graph_exection.py::test_sequence_generator). The error occurs in the TFA dynamic_decode function.  The stack trace indicates the issue is around use of TFA custom ops.  Still investigating this failure.

Here is consolidate log file from testing the PR.
_pytest_entire_tests_after_test_sequence_generator_change.txt

For the record, resolution for the tf-nightly issue was the tf-nightly version.  Error does not occur with version tf-nightly==2.4.0.dev20201020.


About the speed of data parallel training on different machines

I have read the speed given on the page of mxnet-speed.
I know that when the GPU is different, the speed and speed up rate will be different as well. But I also read this paper. I think the result of this paper has a large gap with the speed I mentioned above. Will P40 make such a huge difference?
Looking forward for an answer. Thanks a lot.

Looking into the configuration for AlexNet, mxnet uses batch size per GPU of 512, and paper uses 1024 (Table IV). Moreover paper says they exclude first epoch training times to avoid the impact of I/O overheads from hard disks.
This two considerations could also have significant impact in throughput, as far as I know.

I think if the data save in machine and don't transfer on the network, the bigger batch size the better speed up rate? Cause the rate of computation and communication can be larger when have a larger batch size.
If I have some mistake understanding, please correct me : )

You're right. In general larger batch sizes reduce communication overhead and improve throughput. But also cause optimization difficulties. Though there are workarounds. +info

Yes, I totally agree with the optimization difficulties large batch size bring. But the speed evaluation in both mxnet's web page and the article is the speed of image processing per second.

proposed labels: "Question", "Performance", "Discussion"

Can you please ask usage related questions at - https://discuss.mxnet.io/
Please feel free to reopen if you encounter any issues.


[Integration] If to switch between models on Talk to form for New conversation, UI is not user-friendly. Please see the video.

https://github.com/epam/ai-dial-chat/assets/111180458/dced3162-5374-4d7e-8cfa-f1758e9fbf58

Verified on staging and UAT successfully


Make Model Preview Extension work with hidden folders

When using folder for previews, the folders also appear in the Extra Networks cards, and having lots of models, Extra networks get bloated with folders that have no use there.
I was able to hide the folder in Extra Networks making the folder to start with a period ".", but then Model Previews won´t find the folder.
Can you make this extension to work with hidden folders? (starting with ".").

Do you need it to be a hidden folder or do you just not want to see it in the subfolder list on the Extra Networks page? As far as I can tell A1111 doesnt let you link to hidden folders through the gradio front end. But I can add something that would go through and hide all the subfolders in the Extra Network tab that start with a specific string. Like say "Previews_"

I added some JavaScript that will hide subfolders from the extra networks subfolder button list if the folder starts with an underscore (example _previews) in commit 54a63489addc0dad48dc43ecbb1b7f45642b46f2


A little advice

Thank you for the tool,but, Can you add this small feature? incr.zsh

auto-fu.zsh is essentially the successor to incr.zsh. And you can use both it and zsh-autosuggestions at the same time. My config does so, although my config is a rather interesting beast that I'm not sure people should be reading.

@Alessandro-Barbieri you might want to leave an issue on my config, so I won't forget to get to it.

ok this works for me (I had to change colour because with solarized dark theme was invisible)
ZSH_AUTOSUGGEST_HIGHLIGHT_STYLE='fg=grey'
ZSH_AUTOSUGGEST_CLEAR_WIDGETS+=(
    "expand-or-complete"
    "pcomplete"
    "copy-earlier-word"
)
bindkey '^ ' autosuggest-accept


[jest-each] Add pretty format

Summary
Use pretty format to interpolate test titles with given data.
Closes #6321
/cc @aaronabramov @SimenB
Do you guys think we need to trim serialising large datasets?
Test plan

Unit tests
e2e tests

I've landed the CI fix PR, if you wanna rebase

Codecov Report

Merging #6357 into master will increase coverage by 0.04%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #6357      +/-   ##
==========================================
+ Coverage   63.47%   63.52%   +0.04%
==========================================
  Files         227      227
  Lines        8677     8688      +11
  Branches        4        4
==========================================
+ Hits         5508     5519      +11
  Misses       3168     3168
  Partials        1        1

Impacted Files
Coverage Δ

packages/jest-each/src/bind.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aff9681...dafcb5b. Read the comment docs.