How to get the chain of thought from the original data from 2WikiMultiHopQA or musique

Hi-
It's not clear what your question is, but-
Those chain-of-thought answers were written by us, for those 4 examples in the prompt. All of the chain-of-thought examples we wrote are in the appendix.

Thx, your answer is helpful to me.
Will you consider release the GPT results on dataset 2WikiMultiHopQA and musique?

BTW, will u release the prompts your model outputs on 2WikiMultiHopQA or musique dataset?


Can't delete phylogeny from phylogeny view

At least in the currently deployed version (refs/tags/v0.3.1), there doesn't seem to be a UI element for deleting a phylogeny. One has to know to go back to the "Summary" option under the Phyloreference section, then scroll down to Phylogenies. That's quite unintuitive.

This issue is still present in deployment release/v0.4.0.

Also true for phyloreferences.


Extracting URL doesn't work if the page is non-html or invalid

@alex000kim If you provide a raw Github README link, it doesn't work since the page is a plain string (extract() doesn't work).
Also, if the URL is invalid, the Slackbot responds "I can't provide a response. Encountered an error: 'NoneType' object has no attribute 'lower'.
I was able to fix these issues by using the following updated code:
def is_html(content):
    content_start = content.lower().strip()[:15]
    return content_start.startswith("<!doctype html>") or content_start.startswith("<html>")

def augment_user_message(user_message, url_list):
    all_url_content = ''
    for url in url_list:
        downloaded = fetch_url(url)
        if downloaded is None:
            return user_message
        # Check if the content is HTML, then use extract() to clean and extract the main text content
        if is_html(downloaded):
            url_content = extract(downloaded, config=newconfig)
        else:
            url_content = downloaded
        user_message = user_message.replace(f'<{url}>', '')
        all_url_content = all_url_content + f' Contents of {url} : \n """ {url_content} """'
    user_message = user_message + "\n" + all_url_content
    return user_message

Please consider applying these changes to your code, or feel free to use a better solution if you know of any. I just wanted to share this with you.
Thank you so much for your great work by the way! :)

@cuinjune thanks, please create a PR with this change


Release 0.9.0

Are we happy with the current list of issues/PRs for 0.9.0?
https://github.com/audreyr/cookiecutter/milestones/0.9.0
@audreyr @pydanny @pfmoore

I completed this yesterday. Closing out this ticket and the milestone.


Add support for ACLs

Sorry I wasn't aware that Freyskeyd had already started some dev on ACL,
but well, I needed it to work asap so here's my PR.

I have completely separate the credentials from the ACL,
as Mashape do in their documentation
http://screencast.com/t/WG29AAnIfBQ3
Maybe I've done too much compared to Freyskeyd ..
Comments are welcome!

Looks great, thanks! I'll check it out and release a new version tomorrow.

:+1: excited for this.


Django version update

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.80%. Comparing base (738af77) to head (e33a7d1).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #198   +/-   ##
=======================================
  Coverage   90.80%   90.80%
=======================================
  Files          70       70
  Lines        2078     2078
=======================================
  Hits         1887     1887
  Misses        191      191

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


refactor: dynconfig without Unmarshal

Signed-off-by: Gaius<EMAIL_ADDRESS>
Description

Refactor dynconfig without Unmarshal.

Related Issue

Motivation and Context

Screenshots (if appropriate)
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation Update (if none of the other choices apply)

Checklist

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.

Codecov Report

Merging #1926 (aecdcb2) into main (5fffd7c) will increase coverage by 0.03%.
The diff coverage is 76.19%.

@@            Coverage Diff             @@
##             main    #1926      +/-   ##
==========================================
+ Coverage   43.07%   43.10%   +0.03%
==========================================
  Files         132      132
  Lines       16965    16952      -13
==========================================
  Hits         7308     7308
+ Misses       9104     9092      -12
+ Partials      553      552       -1

Flag
Coverage Δ

Object-compatibility-e2etests
?

e2etests
?

unittests
43.10% <76.19%> (+0.03%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

internal/dynconfig/dynconfig.go
75.40% <73.68%> (+2.82%)
:arrow_up:

client/config/dynconfig_manager.go
58.94% <100.00%> (+0.34%)
:arrow_up:

scheduler/config/dynconfig.go
25.26% <100.00%> (-0.76%)
:arrow_down:

...ent/daemon/peer/peertask_piecetask_synchronizer.go
49.54% <0.00%> (-3.67%)
:arrow_down:

client/daemon/peer/peertask_conductor.go
56.99% <0.00%> (-0.65%)
:arrow_down:

client/daemon/peer/peertask_piecetask_poller.go
48.63% <0.00%> (+14.20%)
:arrow_up:


renaming metrics

Description
[renaming metrics]
Issues Resolved
[List any issues this PR will resolve]
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

are we removing all NODE keywords in the metric naming? like changing ML_NODE_TOTAL_MODEL_COUNT to ML_TOTAL_MODEL_COUNT. What's the reason for that?

are we removing all NODE keywords in the metric naming? like changing ML_NODE_TOTAL_MODEL_COUNT to ML_TOTAL_MODEL_COUNT. What's the reason for that?

Before we had only data note. But in model serving framework we also initiated ML Node. So having this ML_NODE prefix might confuse users.

are we removing all NODE keywords in the metric naming? like changing ML_NODE_TOTAL_MODEL_COUNT to ML_TOTAL_MODEL_COUNT. What's the reason for that?

Before we had only data note. But in model serving framework we also initiated ML Node. So having this ML_NODE prefix might confuse users.

LGTM, just could you check why all the builds are failing? Will approve after CI is passing

LGTM, just could you check why all the builds are failing? Will approve after CI is passing

It's failing here, we applied some node level vs cluster level distinction in the stats. Now it's treating all of them as Cluster level stats.


Select and Execute Specific Chunks of SQL Queries Within Stored Procedures

It would be great to have a feature for executing specific chunks of SQL queries within stored procedures. This would make debugging and testing more convenient by allowing selective execution of query blocks rather than running the entire procedure.

Doesn't it already work exactly like your title says: select a fragment of script editor's text you want to execute as a unit and then use Run query command?


Only files from the most recent commit in a push are rendered

If two commits are made before a push, only the files modified in the most recent are modified.
https://github.com/agurodriguez/github-texify/blob/a5b703f1889ed1c8e397996e2bc6365ed04d7271/src/texify.js#L41
This should be solvable by replacing the above line with:
let commitFiles = [];
for (let commit of this.push.commits) {
    commitFiles.push(...commit.added, ...commit.modified);
}

Awesome, thanks for all your work today

Is the server running at the moment? I can't see any builds coming through at https://github.com/xsanda/SPS-notes/commits/master

Yes, it is.
As far as I can see your request can’t be processed because we are receiving this error when trying to use Github API:
HTTP/1.1 403 Forbidden
Content-Type: application/json; charset=utf-8
Connection: close
{
"message": "You have triggered an abuse detection mechanism and have been temporarily blocked from content creation. Please retry your request again later.",
"documentation_url": "https://developer.github.com/v3/#abuse-rate-limits"
}
https://developer.github.com/v3/#abuse-rate-limits
It’s probably because I generated a lot of request in the past hours when I was testing.
I’ll see if there is a better way to handle the requests in order to go through this limitation.
Sorry for the inconvenience
Regards,
Agustín Rodríguez
http://agurodriguez.net

On May 9, 2018, at 12:27 PM, Charlie Harding<EMAIL_ADDRESS>wrote:
Is the server running at the moment? I can't see any builds coming through at https://github.com/xsanda/SPS-notes/commits/master https://github.com/xsanda/SPS-notes/commits/master
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub https://github.com/agurodriguez/github-texify/issues/2#issuecomment-387778216, or mute the thread https://github.com/notifications/unsubscribe-auth/ANMlyWzsCdsgcLkGlyuxb7dDM5vdlS5Xks5twwrugaJpZM4T4FSj.


Enable CI binary build for the vertical federated learning branch.

Help others who want to test the latest development on that branch without building a binary wheel.

cc @hcho3

@trivialfis You should backport this PR to the vertical federated branch to obtain the binary build.


[Event Request] Table 39 "Purchase Line" - OnUpdateDirectUnitCostByFieldOnBeforeUpdateItemReference

We are migrating our code from NAV 2018 to BC16.4. In the table 39 Purchase Line, in the local procedure UpdateDirectUnitCostByField, we need to add our code at a specific moment. To do so, we need an event publisher, like this :

And the event integration :

Do you think it could be possible ?
Thanks for your work.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


Cannot use link component inside rich text field

Link to reproduction
No response
Describe the Bug
Link button not working in rich text component with SlateEditor.
We are trying to use link component in rich text field but it does not open when I click on it.
Instead this error is displayed on the console and nothing else happens. We recently updated to the "payload": "^2.30.3", version  and also updated this postgres plugin to the following version. "@payloadcms/db-postgres": "^0.8.9",
When I click the link component in RichText field I expect to open a modal but instead I get this error on the console.
I have noticed that you are using this library to open such modal @faceless-ui/modal
We are also using this adapter , but it seems to be updated to the latest version @payloadcms/richtext-slate
THIS IS THE ERROR MESSAGE IM GETTING:

To Reproduce
Just try to open the link component on RichText field with the latest version and nothing will happen.
Instead i get this error on the console

Payload Version
^2.30.3
Adapters and Plugins
@payloadcms/db-postgres @payloadcms/richtext-slate

+1, also seing this issue with v2.30.3

This issue is a duplicate of #8576. Make sure to follow it and check out the recommended workaround.


Dashes

Currently the string -- in source is converted into an em dash. The RFC Editor converts it back into --, so this is unnecessary diff in AUTH48.
I believe kramdown gives some control of this.

Actually, -- is an en dash; --- would be an em-dash.
I'm not giving up on the hope that RFCs can get basic typography right sometime this decade.
But, like smart-quotes, there indeed can be an option.
This is a continuation of
https://github.com/cabo/kramdown-rfc2629/issues/16
and I plan to handle this in the same way.

Oh, I share your hope; it's just that I'm battling unintended changes in RFCs due to the impenetrability of their diffs...

https://mailarchive.ietf.org/arch/msg/rfc-markdown/TlY-yCb58mADnQy6KxrD8RNWjFo/

Works for me (in v3) - thank you so much for the amazing turnaround!


Rancher 2.x UI allows creation of multiple projects with same name

What kind of request is this (question/bug/enhancement/feature request):
Bug
Steps to reproduce (least amount of steps as possible):

Install Rancher 2.x
Create Project .

Create project with same name as in Step 2.

Result:
Rancher UI allows creation of multiple projects with same names.
Other details that may be helpful:
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI):
Installation option (single install/HA):
Rancher version : v2.1.7

Screenshot : https://imgur.com/a/SMVZCV9

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported):
Machine type (cloud/VM/metal) and specifications (CPU/memory):
Kubernetes version (use kubectl version):

(paste the output here)

Docker version (use docker version):
1.13.1

(paste the output here)

This is intentional. It’s designed to be multi-tenant. Tenant X shouldn’t be restricted to picking a project name that hasn’t been used by Tenants A-W.

Ok. But should we allow to create in same cluster? Won't that be causing issues.
Even in multi tenant environment, no two teams have same project names.

As Craig said, it is not unique and does not need to be; is just a display name.  Two unrelated groups can be sharing the same cluster and know nothing about each other.  Since projects don't "cost" anything, enforcing uniqueness just leads to early users claiming all the "good" names.


Fab disappear

Avoid FAB cover content:

on the topic tabs
In the (vertical) scrolling views for the Behavior/Action picker
of expanding cards in Behavior/Action picker views

@bradmontgomery looks good :+1:

Thanks @damianurbanczyk !


performance improvement for SearchIntersect()

SearchIntersect used to allocate new array and resize it at every level of recursion. This lead to a performance bottleneck for searches that return many results. This change allocates array only once and then appends to this single array. In my tests this resulted in ~2x performance improvement on searches with many hits (100000 random points in 10000x10000 square, do 1000 lookups).
All tests pass.

Thanks for the patch, Leonid!


Force grab focus of application

Hi, new here. Would it be possible to use revery to create a desktop application that is "locked-down" / force focus? Think of a quiz-app where you want to prevent cheating

Hi @einarpersson !
There are a couple of SDL2 flags that could facilitate this:

SDL_SetWindowGrab
SDL_WINDOW_ALWAYS_ON_TOP

We'd need to plumb the API through into Revery, so that they are exposed on our Window API

Also new here and following up on this. Are you open to pull requests on reason-sdl2? I am up for working on this. =)

Also new here and following up on this. Are you open to pull requests on reason-sdl2? I am up for working on this. =)

Hey @ifazk, definitely open for PR's and we'll happily try to assist if you decide to give it a go. 🙂


Update scalatest to 3.2.6

Updates org.scalatest:scalatest from 3.2.4 to 3.2.6.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scalatest", artifactId = "scalatest" } ]

labels: library-update, semver-patch

Part of https://github.com/chipsalliance/treadle/pull/314 now.


Error while uploading a file in multipart/form-data

Here is the test code (I test a WebService that takes a file and some data, then upload it to a FTP server):
let chakram = require('chakram'),
    expect = chakram.expect,
    wait = chakram.wait;
let fs = require('fs')
let path = require("path");
let g = require('./lib/globals.js'); /* I define some globals, like g.portfolioUrl: the WebService url I'll test  */
let assert = require('assert');

...

it("should put a file in root folder", function () {
        let $body = {
            action: 'putFile',
            parentPath: '/',
            filenameToStore: 'sample_picture.png'
        };
        /* "options['headers']" is already defined and a "before" step adds authentication cookie */
        options['headers']['Content-Type'] = 'multipart/form-data; charset=utf-8; boundary='
            + Math.random().toString().substr(2);
        options['formData'] = {
            pkgFile: fs.createReadStream(path.resolve(__dirname, '../test/res/sample_picture.png'))
        };
        return chakram.post(g.portfolioUrl, $body, options).then(function (response) {
            expect(response).to.have.status(200);
            return response;
        });
    });

I get this error (seems related to fs.createReadStream):
Error:
write after end
    at write_ (_http_outgoing.js:635:15)
    at ClientRequest.write (_http_outgoing.js:630:10)
    at Request.write (D:\wamp64\www\***\test\***\node_modules\request\request.js:1514:27)
    at FormData.ondata (internal/streams/legacy.js:16:26)
    at emitOne (events.js:115:13)
    at FormData.emit (events.js:210:7)
    at FormData.CombinedStream.write (D:\wamp64\www\***\test\***\node_modules\combined-stream\lib\combined_stream.js:118:8)
    at DelayedStream.ondata (internal/streams/legacy.js:16:26)
    at emitOne (events.js:115:13)
    at DelayedStream.emit (events.js:210:7)
    at DelayedStream._handleEmit (D:\wamp64\www\***\test\***\node_modules\delayed-stream\lib\delayed_stream.js:82:15)
    at ReadStream.source.emit (D:\wamp64\www\***\test\***\node_modules\delayed-stream\lib\delayed_stream.js:29:19)
    at addChunk (_stream_readable.js:252:12)
    at readableAddChunk (_stream_readable.js:239:11)
    at ReadStream.Readable.push (_stream_readable.js:197:10)
    at onread (fs.js:2004:12)
    at FSReqWrap.wrapper [as oncomplete] (fs.js:663:17) {stack: Error: write after end
    at write_ (_http_outgoi… FSReqWrap.wrapper [as oncomplete] (fs.js:663:17),
message: write after end}

Tested on Windows10 with NodeJS 6.11.0 and 8.1.3.
I found some possible solutions with Mocha (see https://stackoverflow.com/questions/10120866/how-to-unit-test-with-a-file-upload-in-mocha/19148688), but I don't know how to adapt it to Chakram.

no comment until now? 👎

This library hasn't been maintained for about four years, @mikila85, so I don't expect there'll be a response soon.  It looks like the readme was updated to say development had stopped in 2018 (8349da3534b350c284fa51b64581894a572d830f)


Composebox cleanup

The first two commits here a kinda big and messy, sorry about that :( but the rest are reasonably small and nice.

I've done some of these changes, will do the rest when I'm back at my computer. Specifically, I think I've done everything except:

[ ] Turn Pull sending logic out of ComposeBox into a NFC commit
[ ] Use useCallback in src/chat/ChatScreen.js

Let me know if that's incorrect, GitHub's UI is quite unwieldy for me for finding things in a review with this many comments :)

(Also, I've not tested all of these changes on-device yet, since the yarn update that I rebased onto seems to have broken the Android build for me, will also do that when I get back)

Ok, I beleive I've addressed all of your comments @chrisbobbe :)

Just rebased this and fixed up a merge conflict. Would love to get this in soon :)

Thanks for the review @gnprice! I've addressed everything except the onLayout comment — that's the last commit here, so I'd appreciate a re-review and merging the beginning of the series while I look into that.

I've rebased this and fixed several merge conflicts. @gnprice / @chrisbobbe I'd appreciate another review on this, since I think it's blocking several other changes people are interested in making on the composebox.

@chrisbobbe I addressed all of those comments, should be ready for another look!


Drag-and-drop upload of a file with "&" in its name fails

Drag-and-drop upload of a file with "&" in its name fails. Though files selected via system file selection dialog upload fine.
This is due to lack of escaping in XML response from the server.
I first tried to work around it via this change:
--- a/UploadServlet.java (Oct 6, 2016 11:49:06 AM)
+++ b/Current File
@@ -770,7 +770,8 @@
     Map<String, String> item = new HashMap<String, String>();
     item.put(TAG_CTYPE, i.getContentType() !=null ? i.getContentType() : "unknown");
     item.put(TAG_SIZE, "" + i.getSize());
-    item.put(TAG_NAME, "" + i.getName());
+    //item.put(TAG_NAME, "" + i.getName());
+    item.put(TAG_NAME, "<![CDATA[" + i.getName() + "]]>");
     item.put(TAG_FIELD, "" + i.getFieldName());
     if (i instanceof HasKey) {
       String k = ((HasKey)i).getKeyString();

It fixed upload of DnD selected files but spoiled file names by replacing "&" with "&" for files selected via system dialog.
To work around that second issue I applied this fix:
--- a/Uploader.java (Oct 6, 2016 11:49:06 AM)
+++ b/Current File
@@ -435,7 +435,7 @@
       serverRawResponse = event.getResults();
       if (serverRawResponse != null) {
         serverRawResponse = serverRawResponse.replaceFirst(".*" + TAG_MSG_START + "([\\s\\S]*?)" + TAG_MSG_END + ".*", "$1");
-        serverRawResponse = serverRawResponse.replace(TAG_MSG_LT, "<").replace(TAG_MSG_GT, ">").replace("<", "<").replaceAll(">", ">").replaceAll(" ", " ");
+        serverRawResponse = serverRawResponse.replace(TAG_MSG_LT, "<").replace(TAG_MSG_GT, ">").replace("<", "<").replaceAll(">", ">").replaceAll(" ", " ").replace("&", "&");
       }
       try {
         // Parse the xml and extract UploadedInfos

Now it works as expected though that second piece of code looks strange unescaping XML char references before passing the XML to the parser. I actually see similar concern has been raised before here: https://github.com/manolo/gwtupload/issues/22 .

Can be closed


Allow to customize or change default nginx gzip settings

When running nginx behind elb, cloudfront or other proxies the gzip_proxied setting must be set to serve compressed files. Also nginx is compressing only text/html by default, so the gzip_types needs to be extended to enable gzip for other types like css, json, etc..
Today gzip is turned on, but without the two settings above, compression is disabled in most cases. It would awesome if gzip would have some other defaults or is configurable via etcd.
Here are for example the default settings used by AWS OpsWorks https://github.com/aws/opsworks-cookbooks/blob/release-chef-11.4/nginx/attributes/nginx.rb#L37 but they also didn't include application/json, which I need for our json-api.

What do you suggest for turning on the 'gunzip' option for nginx?
http://nginx.com/resources/admin-guide/compression-and-decompression/
I would have expected that setting the key 'deis/router/gunzip' to 'on' would do it.

@davidlmorton is on by default http://docs.deis.io/en/latest/customizing_deis/router_settings/#settings-used-by-router

@davidlmorton the correct setting is /deis/router/gzip. use deisctl config router set gzip=on :)

@aledbf and @bacongobbler Thanks for the prompt responses, but I'm asking for the decompression settings to be turned on for nginx, not the compression.  When I send a request to my app I want to be able to send gzip compressed requests without doing the decompression in the app itself.  Unless I'm doing things wrong, I haven't seen nginx do the decompression, it will compress large responses though.

Ah. That's something which is currently not implemented. Could you please open a different issue for that?

I opened an issue for that, thanks again for the responses.  It is issue #3600 for reference.


Python crashes with "Check failed" error when fitting TensorForestEstimator

Problem
I am experiencing an issue trying to use TensorForestEstimator. I am trying to predict 7 output labels by inputting 7 features. That is, my num_outputs=7 and num_features=7 in my hyperparameters. The shape of features and labels is (484876, 7).
Here is an example of the format of my input data:
f1       f2     f3    f4      f5    f6    f7   l1       l2       l3       l4       l5       l6       l7
39000.0  120.0  65.0  1000.0  25.0  0.69  3.94 39000.0  39959.0  42099.0  46153.0  49969.0  54127.0  55911.0
32000.0  185.0  65.0  1000.0  75.0  0.46  2.19 32000.0  37813.0  43074.0  48528.0  54273.0  60885.0  63810.0
30000.0  185.0  65.0  1000.0  25.0  0.41  1.80 30000.0  32481.0  35409.0  39145.0  42750.0  46678.0  48595.0

When calling fit(),  each of the trees get properly trained (see logs below), but when it goes to theCheckpointSaverHook phase, Python crashes with the following error:
2017-08-02 22:55:00.690746: F tensorflow/contrib/tensor_forest/kernels/count_extremely_random_stats_op.cc:404]
Check failed: column < num_classes_ (39001 vs. 2)

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

I might be wrong, but this seems like a bug since TensorFlow is training each the TensorForestEstimator trees, but crashing upon saving. Also the number 39001 is not related to my data since the  shape for both feature and label is (484876, 7).
Here is the code to reproduce:
import tensorflow as tf
import os

from tensorflow.contrib.tensor_forest.client.random_forest import TensorForestEstimator, TensorForestLossHook
from tensorflow.contrib.tensor_forest.python.tensor_forest import ForestHParams, RandomForestGraphs

tf.logging.set_verbosity('INFO')
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1'

def get_training_data():
    training_file = "data.txt"
    data = pd.read_csv(training_file, sep='\t')

    X = np.array(data.drop('Result', axis=1), dtype=np.float32)

    y = []
    # All 7 labels are stored in a string initially, so we parse each label to float and store in an array
    for e in data.ResultStr:
        y.append(list(np.array(str(e).replace('[', '').replace(']', '').split(','))))

    y = np.array(y, dtype=np.float32)

    features = tf.constant(X) # Also tried with tf.Variable
    labels = tf.constant(y) # Also tried with tf.Variable

    return features, labels

# Main
hyperparameters = ForestHParams(
    num_trees=100,
    max_nodes=10000,
    bagging_fraction=1.0,
    num_splits_to_consider=7,
    feature_bagging_fraction=1.0,
    max_fertile_nodes=0,
    split_after_samples=250,
    min_split_samples=5,
    valid_leaf_threshold=1,
    dominate_method='bootstrap',
    dominate_fraction=0.99,
    num_classes=6, # num_outputs - 1 for some reason
    num_features=7
)

estimator = TensorForestEstimator(
    params=hyperparameters,
    device_assigner=None,
    model_dir=None,
    graph_builder_class=RandomForestGraphs,
    config=None,
    weights_name=None,
    keys_name=None,
    feature_engineering_fn=None,
    early_stopping_rounds=100,
    num_trainers=1,
    trainer_id=0,
    report_feature_importances=False,
    local_eval=False
)

estimator.fit(
    input_fn=lambda: get_training_data(),
    max_steps=100,
    monitors=[
        TensorForestLossHook(
            early_stopping_rounds=30
        )
    ]
)

Here is the verbose logging output:
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /var/folders/s3/k08tc0v94j10xn6fk3wz79zh0000gn/T/tmp2RnVSx
INFO:tensorflow:Using config: {'_save_checkpoints_secs': 600, '_num_ps_replicas': 0, '_keep_checkpoint_max': 5, '_task_type': None, '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x1121fe390>, '_model_dir': '/var/folders/s3/k08tc0v94j10xn6fk3wz79zh0000gn/T/tmp2RnVSx', '_save_checkpoints_steps': None, '_keep_checkpoint_every_n_hours': 10000, '_session_config': None, '_tf_random_seed': None, '_environment': 'local', '_num_worker_replicas': 0, '_task_id': 0, '_save_summary_steps': 100, '_tf_config': gpu_options {
  per_process_gpu_memory_fraction: 1
}
, '_evaluation_master': '', '_master': ''}
INFO:tensorflow:Constructing forest with params =
INFO:tensorflow:{'valid_leaf_threshold': 1, 'split_after_samples': 250, 'num_output_columns': 7, 'feature_bagging_fraction': 1.0, 'split_initializations_per_input': 1, 'bagged_features': None, 'min_split_samples': 5, 'max_nodes': 10000, 'num_features': 7, 'num_trees': 100, 'num_splits_to_consider': 7, 'base_random_seed': 0, 'num_outputs': 1, 'dominate_fraction': 0.99, 'max_fertile_nodes': 5000, 'bagged_num_features': 7, 'dominate_method': 'bootstrap', 'bagging_fraction': 1.0, 'regression': False, 'num_classes': 6}
INFO:tensorflow:training graph for tree: 0
INFO:tensorflow:training graph for tree: 1
...
INFO:tensorflow:training graph for tree: 98
INFO:tensorflow:training graph for tree: 99
INFO:tensorflow:Create CheckpointSaverHook.
2017-08-02 22:53:25.218258: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-02 22:53:25.218298: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-08-02 22:53:25.218304: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-02 22:53:25.218308: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-08-02 22:55:00.690746: F tensorflow/contrib/tensor_forest/kernels/count_extremely_random_stats_op.cc:404] Check failed: column < num_classes_ (39001 vs. 7)

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

System information: tf_env.txt

== cat /etc/issue ===============================================
Darwin 127001.local 16.6.0 Darwin Kernel Version 16.6.0: Fri Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64
Mac OS X 10.12.5

== are we in docker =============================================
No

== compiler =====================================================
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

== uname -a =====================================================
Darwin 127001.local 16.6.0 Darwin Kernel Version 16.6.0: Fri Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64

== check pips ===================================================
numpy (1.12.1)
protobuf (3.3.0)
tensorflow (1.2.1)

== check for virtualenv =========================================
False

== tensorflow import ============================================
tf.VERSION = 1.2.1
tf.GIT_VERSION = v1.2.0-5-g435cdfc
tf.COMPILER_VERSION = v1.2.0-5-g435cdfc
Sanity check: array([1], dtype=int32)

== env ==========================================================
LD_LIBRARY_PATH is unset
DYLD_LIBRARY_PATH is unset

== nvidia-smi ===================================================
./tf.sh: line 105: nvidia-smi: command not found

== cuda libs  ===================================================

regression=True needed to be specified in the ForestHParams because TensorForestEstimator by default assumes a classification problem. There is an implicit num_outputs variable created upon initialization of the estimator and it is set to False if regression was not specified. If regression is specified, then num_outputs = num_classes and checkpoints are saved normally.


Remove debug instructions on dead temporaries in CopyForwarding

In CopyForwarding, we forward copy into a dead temp.
Example:
copy_addr %src to [init] %temp
copy_addr %temp [take] to %dest
becomes
copy_addr %src to %dest
Any debug_value_addr instructions on the dead temp between the source to temp copy and temp to dest copy refers to uninitialized memory. This PR deletes these debug_value_addr instructions.

I found this issue while doing the OSSA conversion for CopyForwarding. In OSSA MemoryLifetime verifier asserts on these debug_value_addr instructions on dead temp.

@atrick Thanks, that is certainly more consistent. I have updated my changes.

@swift-ci test

Build failed
Swift Test Linux Platform
Git Sha - f05262855033115ab4d4ffdb201b60bb00a36136

Build failed
Swift Test OS X Platform
Git Sha - f05262855033115ab4d4ffdb201b60bb00a36136

Great catch! This is exactly the type of stuff to look for as we port!


Github Action Workflows for Scheduled Algorithm Deployment

Develop a workflow using Github Actions to apply processing functions to a stream of data, for example:

Simple preprocessing
Calculate total power spectrum
Classify sounds (apply a pre-trained ML model to generate predictions)
Collect results and upload somewhere
Log process.

Github Actions can run sporadically at about 30 min rate but are free, so when data is also publicly available (which is the case with the OOI and Orcasound streams), this approach allows anybody to test their models one the long streams and then compare the results.
Required skills: Python, Github, basic signal processing
Bonus skills: Docker, Cloud computing, deep learning
Possible mentors: Valentina @valentina-s
References:
An example of a github action applied to the Ocean Observatories Initiative Hydrophone Stream:
https://github.com/valentina-s/cron_action
A package to read data from the Ocean Observatories Archive:
https://ooipy.readthedocs.io/en/latest/
Getting Started:

Currently the script.py is only reading the file and creating a spectrogram, but it is not doing anything with it.
Calculate some properties based on it and decide how to organize the results.
Log properly when there the piniging does not return anything
Experiment with longer periods of time, what are the limitations

Points to consider in the proposal:
What are the inputs of the system, what are the outputs? Where will the inputs and outputs live?
What are the simple operations that can be done within the github actions?
How can they be streamlined using Docker containers?
How can they be extended using Cloud computing resources?
How can the results be organized for easy access?
Can this be extended for multiple processes?

@valentina-s , I see your CronPy inside your main.yml script which will call your script.py each 5 minutes after it is started.  Where are you setting the conditions that will start CronPy running? Is it always running?  Github actions are completely new to me.

CronPy is always running (based on the schedule). You can look at the progress here:
https://github.com/valentina-s/cron_action/actions
There is another workflow called Manual workflow which you can trigger manually, or make it work based on some event.

More followup questions:

"What are the inputs of the system, what are the outputs? Where will the inputs and outputs live?"
a) "What are the inputs of the system?"
As I don't fully understand the intent of this project I can only guess that inputs could be the argument to choose which datastream to process (OOI/Orcasound/etc). Possibly to choose what processing algorithm to apply to data?
Maybe input would be dependent on the chosen algorithm, but I feel like then it would be best to create separate workflows for different algorithms. For example, if we have one (oversimplified) workflow that simply pulls new data, applies bandpass filter [2000, 6000] and posts results as artifacts/sends them somewhere, one might want to manually trigger a workflow for given day N and use bandpass filter [1000, 5000]. Then we would add appropriate inputs to the workflow but it all depends on type of algorithm used. Though one similar between them all would be to choose datasource/timeframe to analyze.
b) "what are the outputs?"
Like I mentioned above my understanding is that an alert about detected killer whale/some other interesting activity is the expected end output. Would probably be in the form of a POST request to some of the Orcasound services. Another possibility is just processed audio files/spectrogram/etc. If this tool is the endpoint then just upload as artifacts, otherwise send it further in the pipeline.
c) "Where will the inputs live?"
As inputs are just run arguments they are not stored anywhere explicitly, could be just workflow inputs entered during manual start, comment contents if workflow starts from comments on issues/PRs, arguments set by another GH Action and possibly a few other ways.
d) "Where will the outputs live?"
Can upload them as artifacts to the workflow run, can commit them through PR to the repository if needed, can send them somewhere else.
It should be noted that artifacts and run logs are retained for 90 days (400 for Enterprise) so they should be saved somewhere else if desired.

"What are the simple operations that can be done within the github actions?"
I'm not sure what's considered simple, would love to know :) GitHub Actions are basically free on-demand virtual machines, think Google Colab but on different OSes and with tight GitHub integration.

"How can they be streamlined using Docker containers?"
Using Docker images is really simple, you only need to specify container, it will be pulled for you and workflow step will run inside that container.

"How can they be extended using Cloud computing resources?"
Not sure about this one, GitHub doesn't provide GPUs or other heavy machinery, running inference is fine, but training ML models or doing something compute intensive would probably be a bad idea. This only applies to GitHub-hosted runners, but GitHub allows self-hosted runners, meaning it's possible to configure runner on your own hardware/in the cloud.

"How can the results be organized for easy access?"
See 9b if I understood the question correctly.

"Can this be extended for multiple processes?"
If multiple processes means concurrent execution, then yes, separate jobs in a workflow can run in parallel.
If multiple processes means different use cases then also yes, just create different workflows.
There are limitations to both, mainly on the number of concurrent jobs (20 for free accounts, other accounts get more) and API requests/hour.

In the second set of comments, you are on the right path: there can be different inputs: the streams, or the way the streams are accessed: per file?, per unit time?, and the outputs will differ based on the functions. The point of that question is for students to describe the details in the proposal (like a few scenarios).
There are already some Docker images specifically for the Orcasound data. You can look at the orcaal-research repo, or ask @kunakl07 to direct you to them.
We have some extra cloud resources, so if the github actions resources are not sufficient, so having a way to utilize the cloud ones would be great!

@Molkree you are welcome to submit a PR with the spectrogram artifact. The images will most probably quickly start accumulating and taking up a lot of space. You can add change the retention policy to delete them after a few days:
retention-days: 1

The images will most probably quickly start accumulating and taking up a lot of space.

@valentina-s
Everything should be free for public repos so it's of no concern.

Storing artifacts uses storage space on GitHub. GitHub Actions usage is free for both public repositories and self-hosted runners. For private repositories, each GitHub account receives a certain amount of free minutes and storage, depending on the product used with the account.

From docs.
Alright, I'll send PR shortly, might also change it to process more than one file. And process the previous day by default, there were no new files today...

@valentina-s, quick question, have you ever seen error 404?
if r == 'Response [404]':

from script

@Molkree you are welcome to submit a PR with the spectrogram artifact.

PR opened, it has a bit more than artifact uploading though 😅


Fix incorrect multipart Form Boundaries for upload binary images

According to https://tools.ietf.org/html/rfc7578#section-4.2
Multipart Form Boundaries should include "name" inside the request.
Latest sample from documentation of uploading images also include this field https://developer.ebay.com/Devzone/XML/docs/Reference/eBay/UploadSiteHostedPictures.html
If we try to make an "Image Upload" request in the sandbox environment (production environment is still working) it will fail with an error like "XML Parse error" or something like "No XML RequestPassword or RequestToken was found in XML Request.".
After applying this fix, image uploading is not failed on sandbox.
Please notice soon ebay will roll out a more strict check for Multipart Form data in production and all upload will start failing in both production and sandbox.

@kernio seems like you got the same info from ebay like me 😄 many thanks for this PR ;)

@kernio seems like this file hat to be changed too to pass tests

@michabbb Thank you. Also, notice that :) Should be fixed now.

The format current used by the SDK for binary attachements XML-binary Optimized Packaging, or simply XOP, which allows you to mix binary with XML without the need to base64 encode the binary.  This is used when sending zip files for LMS.
Does anyone now if eBay are changing how attachments for LMS are sent? Will they also be using multipart form-data in the future, because as it stands this PR will break the SDK for LMS.

@davidtsadler i am in contact with the developer support, i will ask them.

@michabbb cheers. Depending on what they say it may just mean that the SDK needs to handle calls to UploadSiteHostedPictures a bit differently.

@davidtsadler jfyi... still waiting for feedback.... 🤔

@davidtsadler I got feedback from ebay:
The LMS documentation is up to date and if you've implemented
your app per specification it should be fine.

The reason we are contacting users of UploadSiteHostedPictures is
 because we had a glitch in our production previously  that allows
for non-standard request format to successfully pass.
We are rolling out code that fixed this glitch which affects people
who previously using the call in the incorrect way.

Sounds like it has nothing to do with LMS

@davidtsadler so ebay fixed something, that should have been already RFC conform.
If the LMS part of your framework isn't 100% RFC conform, I guess we need a change there, too.

A possible fix has been committed to the develop branch. I've used it to upload an image to the sandbox and it appears to work.
It people don't mind testing the changes I can look at getting them into production once everyone is happy.

Just a heads up that I may put a release out this evening that will include this commit. I'm pretty confident the changes will resolve the issue but any feedback once it's live will be helpful.

Thank you for your work @davidtsadler


chore(CODEOWNERS): add Daniel Murphy as codeowner

@danielmurphy I this still in use? I'm asking because it looks like you were the last one to commit to it. If not I can archive it, otherwise I am looking for a code owner. Feel free to suggestion someone or a team, or 👍 to approve this PR.

Pretty sure it's not. Feel free to archive it!


Update docker image to latest and greatest

SMPL PR : https://github.com/CCorporation/smpl/pull/3080

@ascmp Linked in description


Add TYPO3 GFX configuration for ImageMagick6

In order to configure system to process images. Otherwise possible
existing project configuration within LocalConfiguration.php might
prevent processing, due to configured GraphicsMagick or different paths.
Resolves: #1710
Automated Testing Overview:
I've didn't find instructions how to execute existing tests, therefore none were executed or adjusted or added.
Related Issue Link(s):
#1710

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Yes, the use case we encounter are agencies migrating existing project to ddev. Therefore an LocalConfiguration.php already exists on production system, or within VCS already. Whether it's versioned depends on the agencies and people involved. We, as most of our customers, always track this file.

So although we'd expect LocalConfiguration.php to have "local configuration", it actually is general config, and stuff like credentials is always put into AdditionalConfiguration? I assume they're not checking credentials into LocalConfiguration?
Anyway, thanks for helping me understand.

That depends on the users ;) Sometimes the AdditionalConfiguration is used to make use of a different loader, e.g. env or yaml loader.
But in context of ddev, this is the place where ddev can configure TYPO3 to work. We could also add something to the docs instead.

I wonder if there are other things that need to be overwritten by the AdditionalConfiguration for clients in this situation.
Anyway, I'm fine with this, I just have to manually test it.


unit-test.py: Problems with analyzes

All BMC downstream repos should generate UT coverage report using upstream script, but there are problems with analyzes. There is no coverage report if the previous analyzes fail.

clang-tidy
With clang 17 there are compiler errors for nested tuple construction with flag -std=c++23. This is resolved in clang 18, but we are still waiting for clang update: https://gerrit.openbmc.org/c/openbmc/openbmc-build-scripts/+/69586
sanitizers:
linker error with -Db_lundef=false and -Db_sanitize=address,undefined

I can propose 2 solutions:
1) changing the order of the analyzes https://gerrit.openbmc.org/c/openbmc/openbmc-build-scripts/+/72247
2) adding coverage report to 'TEST_ONLY' option
--- a/scripts/unit-test.py
+++ b/scripts/unit-test.py
@@ -921,6 +921,7 @@ class Meson(BuildSystem):
             # is supported by meson's build types. Configure it manually.
             meson_flags.append("-Ddebug=true")
             meson_flags.append("-Doptimization=g")
+            meson_flags.append("-Db_coverage=true")
         else:
             meson_flags.append("--buildtype=debugoptimized")
         if OptionKey("tests") in meson_options:
@@ -1186,6 +1187,8 @@ class Package(object):
         system.test()
         if not TEST_ONLY:
             system.analyze()
+        else:
+            check_call_cmd("ninja", "-C", "build", "coverage-html")

     def test(self):
         for system in self.build_systems():

Any other ideas? 🙂

Perhaps add a coverage method to the BuildSystem class in https://github.com/openbmc/openbmc-build-scripts/blob/master/scripts/unit-test.py#L526 ? That way it can be run independent of other analysis.

https://gerrit.openbmc.org/c/openbmc/openbmc-build-scripts/+/72727
Add an optional parameter ANALYSES to enable the ability to run only the listed analyses in the given order.
I think this solution has more potential.


Port the essential services from Docker to Kubernetes

This PR is mainly concerned with porting the essentials over to Kubernetes.
Fixes #3
Fixes #10
Changes:

The first change is that I removed all of the docker related files.
Updated the README to reflect the Kubernetes deployment
Added a new directory, deploy/ that contains the K8 Helm Chart
Refactored the Docker Image builds so that the commands are now in the K8 config files
Removed the data/ folder since K8 works with PersistentVolume objects

Kubernetes Details:
The most controversial thing that I can think of is that copying files from the host into a pod isn't a Kubernetes pattern. This amounts to some refactoring done around the Docker COPY directive. The best way to do this is to bundle the requirements into a docker image, hosted on dockerhub, and pulled by K8. This allows for a streamlined deployment process that's agnostic of the hosts' filesystem. For the moment, I'm using my personal dockerhub to hold the scheduler & worker images. These should be moved over to the DataONE dockerhub at some point.
K8 still has a 'master' role for services, it looks like the issue is currently unresolved (it's apparently easier to close an issue that sparks debate than actually deal with it). I'll keep my eyes open for ways to come up with a better name.
To Test:

Install & start minikube
Clone this repository and navigate to deploy/
Run helm install ./ --generate-name
Check that the pods and services are up
Run minikube service virtuoso
See that the virtuoso UI opens
Run a sparql query to check that data from DataONE has been inserted

Bonus Testing
Play around with setting replicas to 0 for various pods and check the logs of other pods to make sure they fail. Then bring the replica count back up and see that the other pods successfully find the pod. For example setting the replica count for the worker pod to 0 will cause the scheduler to fail.
Reviewing
The commits are fairly isolated and can be reviewed individually. 456bb45 concerns Kubernetes management and 683d625 concerns adding the Dockerfiles so that the Kubernetes stack can be updated with code changes.

@ThomasThelen I retargeted this PR at develop so that it merges into the right branch. We shouldn't be merging from feature branches directly into main if we follow our release guidelines.

Also, I am not going to have time to dive into this deeply in the next few days, so please have @amoeba review it and hopefully you two can merge to develop and keep moving forward.

Oops I always forget to retarget the branch there. Thanks!

Thanks @ThomasThelen, I'll have a go here in a bit.

Looks good to me @ThomasThelen. Nice work. I was able to get it running without any issues and confirm that triples were being inserted. I might have comments once merged or once we start developing/deploying but I think this is good to go as is.


OPT-1.3b performance tracker

For OPT-1.3b (fp32) we would like to burn down performance at the dispatch level.
Here is a tracy profile for the model executed end-to-end.
To reproduce this trace:

Download opt-1_3b-causallm_cpu_torch.mlir
Run iree-compile (or download opt_untuned.vmfb):

iree-compile ./opt-1_3b-causallm_cpu_torch.mlir --iree-hal-target-backends=llvm-cpu -o opt_untuned.vmfb

Benchmark opt_untuned.vmfb:

TRACY_NO_EXIT=1 iree-benchmark-module --module=opt_untuned.vmfb --function="forward" --input=1x8xi64 --input=1x8xi64 --benchmark_repetitions=50 --task_topology_max_group_count=16

capture and profile the trace.

Here is a screenshot of the dispatches from the profile statistics, ordered by total runtime:

I also noticed for larger sequence lengths, a significant performance delta. Is this expected? e2e execution for sequence length 128 version of OPT takes about 9 times longer than the tiny seqlen8 version from above.
Here is a tracy profile for the model executed end-to-end.
To reproduce this trace:

Download opt-1_3b-causallm_128_torch.mlir
Run iree-compile (or download opt-1_3b_causallm_128_torch_cpu-task.vmfb):

iree-compile ./opt-1_3b-causallm_128_torch.mlir --iree-hal-target-backends=llvm-cpu -o opt-1_3b_causallm_128_torch_cpu-task.vmfb

Benchmark opt-1_3b_causallm_128_torch_cpu-task.vmfb:

TRACY_NO_EXIT=1 iree-benchmark-module --module=opt-1_3b_causallm_128_torch_cpu-task.vmfb --function="forward" --input=1x128xi64 --input=1x128xi64 --benchmark_repetitions=50 --task_topology_max_group_count=16

capture and profile the trace.

Here is a screenshot of the dispatches from the profile statistics, ordered by total runtime:

Do you have tuned results for dispatch 17 and 18? Looks like we should first boil down these ones.

Do you have tuned results for dispatch 17 and 18? Looks like we should first boil down these ones.

Tuning these dispatches for tile/workgroup sizes does not yield a significant performance delta, unfotunately.

cc @MaheshRavishankar @hanhanW
Current status:
The sequence length = 8 case should see significant improvements from @bjacob 's work; see this discord thread
We are discussing the matmul cases where M > 16, as the associated optimization space is drastically different from the narrow matmul cases.

thanks @monorimet for the summary; FTR, the above-linked discord thread also contains suggestions of flags to use for the M>=16 case. looking forward to hearing how that performs; we can discuss next steps from there.

Here is an updated set of artifacts and tracy profile for the case where sequence length (M dimension) = 128, at fp32 precision.
The .mlir can be found here: opt-1_3b_causallm_128_torch.mlir
From here there are two cases that produce drastically different results (with and without --iree-llvmcpu-enable-microkernels):
Case 1 - without microkernels:
In this case, we are faster than I was able to achieve earlier but still lagging behind pytorch performance.

The .vmfb is located here: opt-1_3b_causallm_128_torch_cpu-task.vmfb
The tracy profile is also provided here: opt128_noukernels.tracy

Case 2 - with microkernels enabled:
In this case, I was surprised to see a sharp decrease in performance. I assume that we are simply not in a case where this flag helps, but in case it seems wrong I wanted to share my results:

The .vmfb is located here: opt-1_3b_causallm_128_torch_cpu-task_ukernels.vmfb
The tracy profile is also provided here: opt128_ukernels.tracy

The likely reason why microkernels decrease performance here is the f16 element type --- I have not yet added the optimized code paths for f16 microkernels, so it's currently very slow generic code. I plan to do so this week though, so we can retry then.
Just checking - is it intentional that f16 was the element type here? The benchmarks discussed above were f32 and the last comment even says:

Here is an updated set of artifacts and tracy profile for the case where sequence length (M dimension) = 128, at fp32 precision.

I am also confused to see the f16 precision. Let me make sure this .mlir is fp32 precision and I will update accordingly.

Running the same flags as above with the fp32 OPT.mlir results in a segfault in iree-benchmark-module.
I will be removing flags to see if any specific ones are the culprit.
Here is the commands I'm using where the segfaults occur:
/home/ean/iree-build/tools/iree-compile ./opt-1_3b_causallm_128_torch.mlir --iree-hal-target-backends=llvm-cpu --iree-llvmcpu-target-cpu-features=host --iree-flow-enable-data-tiling --iree-llvmcpu-target-cpu=cascadelake --iree-llvmcpu-stack-allocation-limit=256000 --iree-llvmcpu-enable-microkernels -o opt-1_3b_causallm_128_torch_cpu-task_ukernels.vmfb

TRACY_NO_EXIT=1 /home/ean/iree-build/tools/iree-benchmark-module --module=opt-1_3b_causallm_128_torch_cpu-task.vmfb --function="forward" --input=1x128xi64 --input=1x128xi64 --benchmark_repetitions=10 --task_topology_max_group_count=16 --device=local-task

2023-07-11T18:34:15+00:00
Running /home/ean/iree-build/tools/iree-benchmark-module
Run on (16 X 2800.27 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 33792 KiB (x1)
Load Average: 21.13, 25.39, 20.56
Segmentation fault (core dumped)

It seems --iree-flow-enable-data-tiling was the flag causing the segfault in f32. I will upload results without this flag shortly.

That is the flag that triggers everything. Without that flag, the other flags dont do anything. I will take a look, but it will take me sometime (a day or so) to get around to it. We need to track down the stack allocation issue as well.

That is the flag that triggers everything. Without that flag, the other flags dont do anything. I will take a look, but it will take me sometime (a day or so) to get around to it. We need to track down the stack allocation issue as well.

OK. In the meantime I will find out if the sequence length (M dimension in matmuls) is relevant to the segfaults-- 128 is a somewhat arbitrary choice so if we can find some other M>16 that works with data tiling then we can be at least temporarily unblocked.

It seems the data tiling flag now causes this segfault for all sequence lengths. I will see what I can do to bisect the problem / isolate the problem dispatch(es) today.

@monorimet I can take a look at this this week. I have to finish a few things before I can get to it. So I'll post here when I get to this.

OK. I have tracked the segfault down to dispatch 25 (correct me if the the numbering could be wrong, I'm not sure if the number in the function signature is the same as the index, but the segfault occurs with data tiling starting at index 25 as shown:)

I was able to compile and run dispatch 25 with the following input:
(shark.venv) ean@ean-highmem:~/SHARK/tank/examples/opt$ /home/ean/iree-build/tools/iree-compile ./opt-1_3b-causallm_cpu_dispatches/module_forward_dispatch_25_embedded_elf_x86_64_benchmark.mlir --iree-hal-target-backends=llvm-cpu --iree-llvmcpu-target-cpu-features=host --iree-llvmcpu-target-cpu=cascadelake --iree-llvmcpu-stack-allocation-limit=131072 --iree-flow-enable-data-tiling -o opt-1_3b_causallm_128_dispatch_25_tiled.vmfb

(shark.venv) ean@ean-highmem:~/SHARK/tank/examples/opt$ /home/ean/iree-build/tools/iree-benchmark-module --module=./opt-1_3b_causallm_128_dispatch_25_tiled.vmfb

Yielding the following results:
2023-07-12T23:56:22+00:00
Running /home/ean/iree-build/tools/iree-benchmark-module
Run on (16 X 2800.27 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 33792 KiB (x1)
Load Average: 0.00, 0.00, 0.00
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                                Time             CPU   Iterations UserCounters...
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
BM_forward_dispatch_25_embedded_elf_x86_64_forward_dispatch_25_matmul_32x8192x2048_f32/process_time/real_time      10150 us        76696 us           52 items_per_second=98.5238/s

This behavior is replicated on sequence lengths 8, 16, 32, and 128 -- It isn't very surprising as the data tiling is happening at the flow level.
I will try a few other cases tomorrow in f16 / compare with pytorch in the meantime -- please let me know if I can help more with the segfaults here @MaheshRavishankar

I was able to compile and run dispatch 25 with the following input:
(shark.venv) ean@ean-highmem:~/SHARK/tank/examples/opt$ /home/ean/iree-build/tools/iree-compile ./opt-1_3b-causallm_cpu_dispatches/module_forward_dispatch_25_embedded_elf_x86_64_benchmark.mlir --iree-hal-target-backends=llvm-cpu --iree-llvmcpu-target-cpu-features=host --iree-llvmcpu-target-cpu=cascadelake --iree-llvmcpu-stack-allocation-limit=131072 --iree-flow-enable-data-tiling -o opt-1_3b_causallm_128_dispatch_25_tiled.vmfb

(shark.venv) ean@ean-highmem:~/SHARK/tank/examples/opt$ /home/ean/iree-build/tools/iree-benchmark-module --module=./opt-1_3b_causallm_128_dispatch_25_tiled.vmfb

Yielding the following results:
2023-07-12T23:56:22+00:00
Running /home/ean/iree-build/tools/iree-benchmark-module
Run on (16 X 2800.27 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 33792 KiB (x1)
Load Average: 0.00, 0.00, 0.00
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                                Time             CPU   Iterations UserCounters...
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
BM_forward_dispatch_25_embedded_elf_x86_64_forward_dispatch_25_matmul_32x8192x2048_f32/process_time/real_time      10150 us        76696 us           52 items_per_second=98.5238/s

This behavior is replicated on sequence lengths 8, 16, 32, and 128 -- It isn't very surprising as the data tiling is happening at the flow level.
I will back up to the flow level and do some more poking around, and try a few other cases tomorrow in f16 / compare with pytorch in the meantime -- please let me know if I can focus my efforts anywhere to help @MaheshRavishankar

Could you just create an issue with the dispatch itself on IREE.  You can take the func.func that is within the dispatch that is the offending dispatch and pretty easily recreate a small repro. If you post even just the func.func within the flow.executable, that should be a good enough starting point. I think the issue is with respect to pack and unpack fusion. We might be fusing too aggressively.

OK, I have filed the issue in IREE. For the segfaults we can move discussion to that issue until resolved.

I have included a script for running a perf comparison (SHARK/IREE vs. PyTorch) for opt1.3b causallm inference.
Without data-tiling (with default SHARK flags -- I can dig these out if anyone is interested) we achieve the following:
(shark.venv) ean@ean-highmem:~/SHARK/tank/examples/opt$ python opt_perf_comparison.py
Loading flatbuffer at opt_1-3b_causallm_8_torch_cpu.vmfb as a mmapped file
[DEBUG] setting iree runtime flags for cpu:
--task_topology_max_group_count=14
--- Took 0.48285388946533203 seconds to load Shark.
prompt: What is the meaning of life?
prompt: Tell me something you don't know.
prompt: What does Xilinx do?
prompt: What is the mass of earth?
prompt: What is a poem?
prompt: What is recursion?
prompt: Tell me a one line joke.
prompt: Who is Gilgamesh?
prompt: Tell me something about cryptocurrency.
prompt: How did it all begin?
--- Took 4.863748073577881 seconds to run Shark.

--- Took 2.3916406631469727 seconds to run Huggingface.

@monorimet - With https://github.com/openxla/iree/issues/14398 now fixed, here are some benchmark results to give a flavor of performance to expect. Note - testing on a Intel Skylake-XEON CPU with AVX-512. Compiling with --iree-llvmcpu-target-cpu=skylake-avx512. Command lines as in the original PR description above.

Without data-tiling and ukernels: 515 ms
With data-tiling but not ukernels: 72 ms
With data-tiling and ukernels: 3100 ms

So, data-tiling alone is a ~ 8x speedup. Ukernels alone are not yet good. But I'll get to that now, and it will be at least as fast as non-ukernels and in some cases faster. What's almost certainly happening here is that this particular model is f32, and f32 matmuls on ISAs like AVX-512 are what default codegen is good at. As soon as we depart from that, e.g. f16, things are more challenging for default codegen and the ukernels become more of a win.

Nice! Are these the narrow shapes case? E.g., mostly I've seen ukernels doing rather well :)

Nice! Are these the narrow shapes case? E.g., mostly I've seen ukernels doing rather well :)

Oh I get it now - no it's not narrow (it's opt-1_3b_causallm_128_torch.mlir which has sequence length 128) but contrary to what I assumed in my previous comment, it is a f16 model.   So neither the default codegen nor the ukernels are good at the moment, but the ukernels are even worse, as they are running completely slow generic code.  OK, didn't remember we were already looking at f16 --- i'll reorder my queue so f16 ukernels come sooner.

Nice! Are these the narrow shapes case? E.g., mostly I've seen ukernels doing rather well :)

Oh I get it now - no it's not narrow (it's opt-1_3b_causallm_128_torch.mlir which has sequence length 128) but contrary to what I assumed in my previous comment, it is a f16 model. So neither the default codegen nor the ukernels are good at the moment, but the ukernels are even worse, as they are running completely slow generic code. OK, didn't remember we were already looking at f16 --- i'll reorder my queue so f16 ukernels come sooner.

Whats the target here? Without ukernels but data tiling, this is at 72 us (maybe 515us is just so bad, that this is unfair comparison)

I'll let Nod decide if there's a specific target; all I know is we still have plenty of room to run :-)

Nice! Are these the narrow shapes case? E.g., mostly I've seen ukernels doing rather well :)

Oh I get it now - no it's not narrow (it's opt-1_3b_causallm_128_torch.mlir which has sequence length 128) but contrary to what I assumed in my previous comment, it is a f16 model. So neither the default codegen nor the ukernels are good at the moment, but the ukernels are even worse, as they are running completely slow generic code. OK, didn't remember we were already looking at f16 --- i'll reorder my queue so f16 ukernels come sooner.

The first case we wanted to meet/beat pytorch performance on was OPT1.3b in fp32 precision. With data tiling, I do see a significant improvement in e2e execution time (tested on this iree SHA)
The following are e2e benchmarks on OPT-1.3b in fp32 precision, at sequence length 128, with avx512 instructions enabled. I will preface each result with the reproduction commands.
Link to opt_1-3b_causallm_128_torch.mlir
Case 1: No data tiling, no microkernels
iree-compile ./opt_1-3b_causallm_128_torch.mlir --iree-hal-target-backends=llvm-cpu --iree-llvmcpu-target-cpu-features=host --iree-llvmcpu-stack-allocation-limit=140000 -o opt_1-3b_causallm_128_torch_cpu_base.vmfb

iree-benchmark-module --module=./opt_1-3b_causallm_128_torch_cpu_base.vmfb --function="forward" --input=1x128xi64 --input=1x128xi64 --benchmark_repetitions=10 --task_topology_max_group_count=16 --device=local-task

2023-07-18T11:06:19-07:00
Running /home/ean/SHARK/shark.venv/lib/python3.11/site-packages/iree/runtime/scripts/iree_benchmark_module/../../iree-benchmark-module
Run on (16 X 4000 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 16384 KiB (x1)
Load Average: 0.33, 0.38, 0.46
---------------------------------------------------------------------------------------------------
Benchmark                                         Time             CPU   Iterations UserCounters...
---------------------------------------------------------------------------------------------------
BM_forward/process_time/real_time              2072 ms        15921 ms            1 items_per_second=0.482637/s
BM_forward/process_time/real_time              2060 ms        15901 ms            1 items_per_second=0.485382/s
BM_forward/process_time/real_time              2068 ms        16012 ms            1 items_per_second=0.483572/s
BM_forward/process_time/real_time              2052 ms        15923 ms            1 items_per_second=0.4873/s
BM_forward/process_time/real_time              2089 ms        16011 ms            1 items_per_second=0.478638/s
BM_forward/process_time/real_time              2177 ms        15987 ms            1 items_per_second=0.459379/s
BM_forward/process_time/real_time              2058 ms        15968 ms            1 items_per_second=0.485893/s
BM_forward/process_time/real_time              2042 ms        15825 ms            1 items_per_second=0.489682/s
BM_forward/process_time/real_time              2049 ms        15889 ms            1 items_per_second=0.488093/s
BM_forward/process_time/real_time              2053 ms        15936 ms            1 items_per_second=0.487093/s
BM_forward/process_time/real_time_mean         2072 ms        15937 ms           10 items_per_second=0.482767/s
BM_forward/process_time/real_time_median       2059 ms        15930 ms           10 items_per_second=0.485638/s
BM_forward/process_time/real_time_stddev       39.2 ms         58.9 ms           10 items_per_second=8.79873m/s
BM_forward/process_time/real_time_cv           1.89 %          0.37 %            10 items_per_second=1.82%

Case 2: Data tiling, no ukernels:
iree-compile ./opt_1-3b_causallm_128_torch.mlir --iree-hal-target-backends=llvm-cpu --iree-llvmcpu-target-cpu-features=host --iree-flow-enable-data-tiling --iree-llvmcpu-stack-allocation-limit=140000 -o opt_1-3b_causallm_128_to
rch_cpu_tiled.vmfb

iree-benchmark-module --module=./opt_1-3b_causallm_128_torch_cpu_tiled.vmfb --function="forward" --input=1x128xi64 --input=1x128xi64 --benchmark_repetitions=10 --task_topology_max_group_count=14 --device=local-task

2023-07-18T12:29:20-07:00
Running /home/ean/SHARK/shark.venv/lib/python3.11/site-packages/iree/runtime/scripts/iree_benchmark_module/../../iree-benchmark-module
Run on (16 X 4000 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 16384 KiB (x1)
Load Average: 2.18, 1.57, 0.80
---------------------------------------------------------------------------------------------------
Benchmark                                         Time             CPU   Iterations UserCounters...
---------------------------------------------------------------------------------------------------
BM_forward/process_time/real_time               762 ms         5892 ms            1 items_per_second=1.31257/s
BM_forward/process_time/real_time               764 ms         5918 ms            1 items_per_second=1.30817/s
BM_forward/process_time/real_time               764 ms         5911 ms            1 items_per_second=1.3083/s
BM_forward/process_time/real_time               778 ms         5893 ms            1 items_per_second=1.28507/s
BM_forward/process_time/real_time               766 ms         5936 ms            1 items_per_second=1.30487/s
BM_forward/process_time/real_time               764 ms         5934 ms            1 items_per_second=1.30868/s
BM_forward/process_time/real_time               763 ms         5926 ms            1 items_per_second=1.30978/s
BM_forward/process_time/real_time               765 ms         5929 ms            1 items_per_second=1.30763/s
BM_forward/process_time/real_time               766 ms         5924 ms            1 items_per_second=1.30607/s
BM_forward/process_time/real_time               762 ms         5895 ms            1 items_per_second=1.31302/s
BM_forward/process_time/real_time_mean          765 ms         5916 ms           10 items_per_second=1.30642/s
BM_forward/process_time/real_time_median        764 ms         5921 ms           10 items_per_second=1.30824/s
BM_forward/process_time/real_time_stddev       4.70 ms         17.1 ms           10 items_per_second=7.91746m/s
BM_forward/process_time/real_time_cv           0.61 %          0.29 %            10 items_per_second=0.61%

Case 3: Data tiling and ukernels:
iree-compile ./opt_1-3b_causallm_128_torch.mlir --iree-hal-target-backends=llvm-cpu --iree-llvmcpu-target-cpu-features=host --iree-flow-enable-data-tiling --iree-llvmcpu-stack-allocation-limit=140000 --iree-llvmcpu-enable-microkernels -o opt_1-3b_causallm_128_torch_cpu_tiled_ukernels.vmfb

iree-benchmark-module --module=./opt_1-3b_causallm_128_torch_cpu_tiled_ukernels.vmfb --function="forward" --input=1x128xi64 --input=1x128xi64 --benchmark_repetitions=10 --task_topology_max_group_count=14 --device=local-task

2023-07-18T12:33:04-07:00
Running /home/ean/SHARK/shark.venv/lib/python3.11/site-packages/iree/runtime/scripts/iree_benchmark_module/../../iree-benchmark-module
Run on (16 X 2093.89 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 16384 KiB (x1)
Load Average: 0.85, 1.23, 0.84
---------------------------------------------------------------------------------------------------
Benchmark                                         Time             CPU   Iterations UserCounters...
---------------------------------------------------------------------------------------------------
BM_forward/process_time/real_time               751 ms         5786 ms            1 items_per_second=1.33228/s
BM_forward/process_time/real_time               750 ms         5808 ms            1 items_per_second=1.33304/s
BM_forward/process_time/real_time               750 ms         5814 ms            1 items_per_second=1.33299/s
BM_forward/process_time/real_time               748 ms         5775 ms            1 items_per_second=1.33752/s
BM_forward/process_time/real_time               753 ms         5824 ms            1 items_per_second=1.32875/s
BM_forward/process_time/real_time               753 ms         5773 ms            1 items_per_second=1.32838/s
BM_forward/process_time/real_time               747 ms         5768 ms            1 items_per_second=1.33878/s
BM_forward/process_time/real_time               750 ms         5791 ms            1 items_per_second=1.33385/s
BM_forward/process_time/real_time               752 ms         5801 ms            1 items_per_second=1.32931/s
BM_forward/process_time/real_time               750 ms         5803 ms            1 items_per_second=1.33333/s
BM_forward/process_time/real_time_mean          750 ms         5794 ms           10 items_per_second=1.33282/s
BM_forward/process_time/real_time_median        750 ms         5796 ms           10 items_per_second=1.33301/s
BM_forward/process_time/real_time_stddev       1.95 ms         18.8 ms           10 items_per_second=3.46296m/s
BM_forward/process_time/real_time_cv           0.26 %          0.32 %            10 items_per_second=0.26%

So in my case the tiled + ukernels mode seems to produce the best results.
Examining performance delta vs. pytorch is a bit tricky -- we have to feed an input of 127 tokens for the pytorch model to be comparable to our sequence length 128 model. Evidently padding with the tokenizer doesn't seem to stop PyTorch from using the smallest possible model for optimal performance. This is generally equivalent in behavior to the dynamic path in torch-mlir/IREE stack.
Since we are looking at sequence length 128, I've just run the performance comparison with excerpts of the declaration of independence with 127 words each, for 5 iterations:
(shark.venv) ean@sharkbox:~/SHARK/tank/examples/opt$ python opt_perf_comparison.py
Loading flatbuffer at opt_1-3b_causallm_128_torch_cpu.vmfb as a mmapped file
[DEBUG] setting iree runtime flags for cpu:
--task_topology_max_group_count=14
--- Took 0.23573708534240723 seconds to load Shark.
prompt: We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness.--That to secure these rights, Governments are instituted among Men, deriving their just powers from the consent of the governed, --That whenever any Form of Government becomes destructive of these ends, it is the Right of the People to alter or to abolish it, and to institute new Government, laying its foundation on such principles and organizing its powers in such form, as to them shall seem most likely to effect their Safety and Happiness. Prudence, indeed, will dictate that Governments long established should not be changed for light and transient causes
prompt: <truncated>
prompt: <truncated>
prompt: <truncated>
prompt: <truncated>
--- Took 4.245878458023071 seconds to run Shark.
--- Took 5.405649662017822 seconds to load Huggingface.
prompt: <truncated>
prompt: <truncated>
prompt: <truncated>
prompt: <truncated>
prompt: <truncated>
--- Took 3.9855310916900635 seconds to run Huggingface.

4.24 vs. 3.9 is really quite good! Is there anything I've missed in latest IREE, from these reproducers, that could push us past pytorch performance?

https://github.com/openxla/iree/pull/13822 might help a bit

So in my case the tiled + ukernels mode seems to produce the best results.

This is because this is a f32 model, and the sequence length is high enough that this doesn't depend on microkernels handling of narrow cases. So there, as long as you're data-tiling, both pure codegen and microkernels perform well, with microkernels being even a little faster --- that's consistent with what we've observed on other models.
When the data type is not f32, pure codegen tends to struggle a bit, and microkernels can shine if they have a dedicated code path for that case, or be extremely slow if they don't --- that must sound scary, but it's not, because it's a problem that we fix once and for all per element type and then it applies to all models.
When the shapes are narrow (when the sequence length is small), codegen tends to adapt gracefully, but microkernels don't currently have fast code for narrow cases, so that's the other thing I want to fix very soon in microkernels.
Further performance gains beyond that point will come from:

Switching to other element types as appropriate for each workload and hardware target. For example, some recent x86 CPUs have support for bf16 arithmetic, so they would benefit from switching to that (even more cutting-edge is support for f16 arithmetic, currently limited to Intel Sapphire Rapids).
Better distribution heuristics (cache friendliness and multi-threading) and more fusions as appropriate (both topics tracked by @MaheshRavishankar).
Looking at profiles to see if there's something else than matmuls that is being slow.

Examining performance delta vs. pytorch is a bit tricky

So I understand correctly this log: Huggingface is the pytorch value ?

openxla/iree#13822 might help a bit

Thanks. I am building with tracy on latest IREE  to get a trace of the tiled ukernels case, so I will share results with iree-llvmcpu-reassociate-fp-reductions soon.

openxla/iree#13822 might help a bit

Thanks. I am building with tracy on latest IREE to get a trace of the tiled ukernels case, so I will share results with iree-llvmcpu-reassociate-fp-reductions soon.

WIth latest ToT that flag is on by default.

So I understand correctly this log: Huggingface is the pytorch value ?

Yes, that is the value we get from pytorch runtime.

Sorry to say I seem to get better performance with  --iree-llvmcpu-reassociate-fp-reductions=False:
 --iree-llvmcpu-reassociate-fp-reductions=True (default)
(shark.venv) ean@sharkbox:~/SHARK/tank/examples/opt$ /home/ean/iree-build/tools/iree-compile ./opt_1-3b_causallm_128_torch.mlir --iree-hal-target-backends=llvm-cpu --iree-llvmcpu-target-cpu-features=host --iree-flow-enable-data-tiling --iree-llvmcpu-stack-allocation-limit=140000
--iree-llvmcpu-enable-microkernels -o opt_1-3b_causallm_128_torch_cpu_tiled_ukernels_fp_reassoc.vmfb

(shark.venv) ean@sharkbox:~/SHARK/tank/examples/opt$ iree-benchmark-module --module=./opt_1-3b_causallm_128_torch_cpu_tiled_ukernels_fp_reassoc.vmfb --function="forward" --input=1x128xi64 --input=1x128xi64 --benchmark_repetitions=10 --task_topology_max_group_count=14 --device=local-task

2023-07-18T13:32:23-07:00
Running /home/ean/SHARK/shark.venv/lib/python3.11/site-packages/iree/runtime/scripts/iree_benchmark_module/../../iree-benchmark-module
Run on (16 X 4000 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 16384 KiB (x1)
Load Average: 0.67, 3.14, 7.56
---------------------------------------------------------------------------------------------------
Benchmark                                         Time             CPU   Iterations UserCounters...
---------------------------------------------------------------------------------------------------
BM_forward/process_time/real_time               798 ms         5921 ms            1 items_per_second=1.25344/s
BM_forward/process_time/real_time               759 ms         5879 ms            1 items_per_second=1.31727/s
BM_forward/process_time/real_time               762 ms         5892 ms            1 items_per_second=1.31168/s
BM_forward/process_time/real_time               754 ms         5837 ms            1 items_per_second=1.32597/s
BM_forward/process_time/real_time               759 ms         5862 ms            1 items_per_second=1.31826/s
BM_forward/process_time/real_time               760 ms         5873 ms            1 items_per_second=1.31539/s
BM_forward/process_time/real_time               761 ms         5879 ms            1 items_per_second=1.31371/s
BM_forward/process_time/real_time               762 ms         5895 ms            1 items_per_second=1.3116/s
BM_forward/process_time/real_time               762 ms         5897 ms            1 items_per_second=1.3127/s
BM_forward/process_time/real_time               763 ms         5898 ms            1 items_per_second=1.3103/s
BM_forward/process_time/real_time_mean          764 ms         5883 ms           10 items_per_second=1.30903/s
BM_forward/process_time/real_time_median        761 ms         5886 ms           10 items_per_second=1.31321/s
BM_forward/process_time/real_time_stddev       12.1 ms         22.9 ms           10 items_per_second=0.0200596/s
BM_forward/process_time/real_time_cv           1.59 %          0.39 %            10 items_per_second=1.53%

--iree-llvmcpu-reassociate-fp-reductions=False:
(shark.venv) ean@sharkbox:~/SHARK/tank/examples/opt$ iree-compile ./opt_1-3b_causallm_128_torch.mlir --iree-hal-target-backends=llvm-cpu --iree-llvmcpu-target-cpu-features=host --iree-flow-enable-data-tiling --iree-llvmcpu-stack-allocation-limit=140000 --iree-llvmcpu-enable-microkernels --iree-llvmcpu-reassociate-fp-reductions=False -o opt_1-3b_causallm_128_torch_cpu_tiled_ukernels.vmfb

(shark.venv) ean@sharkbox:~/SHARK/tank/examples/opt$ iree-benchmark-module --module=./opt_1-3b_causallm_128_torch_cpu_tiled_ukernels.vmfb --function="forward" --input=1x128xi64 --input=1x128xi64 --benchmark_repetitions=10 --task_topology_max_group_count=14 --device=local-task
2023-07-18T13:27:49-07:00

Running /home/ean/SHARK/shark.venv/lib/python3.11/site-packages/iree/runtime/scripts/iree_benchmark_module/../../iree-benchmark-module
Run on (16 X 4000 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 16384 KiB (x1)
Load Average: 0.92, 6.57, 9.84
---------------------------------------------------------------------------------------------------
Benchmark                                         Time             CPU   Iterations UserCounters...
---------------------------------------------------------------------------------------------------
BM_forward/process_time/real_time               751 ms         5790 ms            1 items_per_second=1.33219/s
BM_forward/process_time/real_time               754 ms         5833 ms            1 items_per_second=1.32702/s
BM_forward/process_time/real_time               756 ms         5843 ms            1 items_per_second=1.32273/s
BM_forward/process_time/real_time               758 ms         5836 ms            1 items_per_second=1.31952/s
BM_forward/process_time/real_time               752 ms         5799 ms            1 items_per_second=1.32904/s
BM_forward/process_time/real_time               754 ms         5821 ms            1 items_per_second=1.32564/s
BM_forward/process_time/real_time               754 ms         5808 ms            1 items_per_second=1.32647/s
BM_forward/process_time/real_time               755 ms         5821 ms            1 items_per_second=1.32446/s
BM_forward/process_time/real_time               756 ms         5832 ms            1 items_per_second=1.32258/s
BM_forward/process_time/real_time               757 ms         5846 ms            1 items_per_second=1.32073/s
BM_forward/process_time/real_time_mean          755 ms         5823 ms           10 items_per_second=1.32504/s
BM_forward/process_time/real_time_median        755 ms         5826 ms           10 items_per_second=1.32505/s
BM_forward/process_time/real_time_stddev       2.20 ms         18.8 ms           10 items_per_second=3.86125m/s
BM_forward/process_time/real_time_cv           0.29 %          0.32 %            10 items_per_second=0.29%

With debug iree-runtime builds, the results are a bit more sporadic so the two cases seem quite similar (I can share those numbers if desired)
Tracy profile for the fastest configuration (latest IREE, seqlen 128, fp32, avx512) (link)

Are the matrix multiplications in this model involving a matrix of constant weights (as is the case in many NN inference workloads) as opposed to runtime values being multiplied by runtime values (as is the case in some recent NN architectures like transformers) ?
If some matmul operands are constant data, then the corresponding set_encoding dispatches are running on constant data and are prime candidates for being constant-evaluated (--iree-opt-const-eval).

Hmm yes it really is lots of big constant matrices.
  func.func @forward(%arg0: tensor<1x128xi64>, %arg1: tensor<1x128xi64>) -> tensor<1x128x50272xf16> {
    %cst = arith.constant dense_resource<__elided__> : tensor<2048xf16>
    %cst_0 = arith.constant dense_resource<__elided__> : tensor<2048xf16>
    %cst_1 = arith.constant dense_resource<__elided__> : tensor<2048x8192xf16>
    %cst_2 = arith.constant dense_resource<__elided__> : tensor<8192xf16>
    %cst_3 = arith.constant dense_resource<__elided__> : tensor<8192x2048xf16>
    %cst_4 = arith.constant dense_resource<__elided__> : tensor<2048xf16>
    %cst_5 = arith.constant dense_resource<__elided__> : tensor<2048xf16>

Are the matrix multiplications in this model involving a matrix of constant weights (as is the case in many NN inference workloads) as opposed to runtime values being multiplied by runtime values (as is the case in some recent NN architectures like transformers) ?
If some matmul operands are constant data, then the corresponding set_encoding dispatches are running on constant data and are prime candidates for being constant-evaluated (--iree-opt-const-eval).

Is the flag all that's necessary to constant-evaluate the set_encoding dispatches?
I got slightly less performant results:
Running /home/ean/SHARK/shark.venv/lib/python3.11/site-packages/iree/runtime/scripts/iree_benchmark_module/../../iree-benchmark-module
Run on (16 X 4000 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 16384 KiB (x1)
Load Average: 0.67, 0.45, 0.71
---------------------------------------------------------------------------------------------------
Benchmark                                         Time             CPU   Iterations UserCounters...
---------------------------------------------------------------------------------------------------
BM_forward/process_time/real_time               848 ms         6093 ms            1 items_per_second=1.17943/s
BM_forward/process_time/real_time               764 ms         5906 ms            1 items_per_second=1.3097/s
BM_forward/process_time/real_time               760 ms         5880 ms            1 items_per_second=1.31503/s
BM_forward/process_time/real_time               763 ms         5907 ms            1 items_per_second=1.30981/s
BM_forward/process_time/real_time               902 ms         6168 ms            1 items_per_second=1.10823/s
BM_forward/process_time/real_time               777 ms         5864 ms            1 items_per_second=1.28711/s
BM_forward/process_time/real_time               786 ms         5956 ms            1 items_per_second=1.27155/s
BM_forward/process_time/real_time               807 ms         6021 ms            1 items_per_second=1.23861/s
BM_forward/process_time/real_time               773 ms         5957 ms            1 items_per_second=1.29322/s
BM_forward/process_time/real_time               770 ms         5961 ms            1 items_per_second=1.29821/s
BM_forward/process_time/real_time_mean          795 ms         5971 ms           10 items_per_second=1.26109/s
BM_forward/process_time/real_time_median        775 ms         5956 ms           10 items_per_second=1.29016/s
BM_forward/process_time/real_time_stddev       46.1 ms         97.1 ms           10 items_per_second=0.0678102/s
BM_forward/process_time/real_time_cv           5.80 %          1.63 %            10 items_per_second=5.38%

hum, nothing out of the top of my head. need to look into this.

There are a couple of things we need to do to get the const eval to work here. Its not a simple flag flip. One we need a way for the MaterializeEncodingPass to use the target information other than what is specified on the dispatch (basically have an "override" attribute somewhere and set that attribute during const eval so that the const eval can run those dispatches.
Once that is done, we just need to make const eval hoisting to hoist out the const -> set_encoding .

@MaheshRavishankar thanks for the explanation, if you file an issue with a ~ 4x expanded version of that to get me started, i might be able to try.
I realized meanwhile that we also needed --iree-opt-const-expr-hoisting for part of what you're saying here, but I was missing that there would be something specific to set_encoding here.

@MaheshRavishankar thanks for the explanation, if you file an issue with a ~ 4x expanded version of that to get me started, i might be able to try.

I dont know what the full solution is, but definitely worth starting an issue and describing what I know, and getting Ben/Stella's help on the remaining. Stay tuned.

I realized meanwhile that we also needed --iree-opt-const-expr-hoisting for part of what you're saying here, but I was missing that there would be something specific to set_encoding here.

There is already an issue for this https://github.com/openxla/iree/issues/11360. Ill add some things there


Disk seeking on A1020 (5.25" inch drive) broken

I have recently purchased a A1020 for fun and tried to run it through its courses in ATK.
ATK detects the device type (device Id, number of tracks) perfectly, but fails seeking.
I.e. it cannot seek back to track 0 and thus cannot perform some of the tests.
I believe this might be due to ATK using the incorrect wait time between steps when issuing multiple step commands. They need to be longer than for 3.5" drives. I did not look up how long, but could find out if necessary.

Please can you try the fixed build linked below (you will need to be logged in to GitHub to see the Artifact download link):
https://github.com/keirf/amiga-stuff/actions/runs/12295522475

This definitely helped!
I can run the read test now and the calibration tests also move the head correctly.
One thing is a bit weird, though: while this version does seek to track 0 in the Signal Test, it seems to try slam the head past its limits - is this intended?

There is a pointless max-cylinders probe at the start of the signal test. It doesn't belong in the signals test, and it's not very good or useful. So I'm removing it. I will have a new build for you shortly.

Here is a fixed build for you to try:
https://github.com/keirf/amiga-stuff/actions/runs/12328565141

Thank you! The head banging stopped now. I can perform signal test, calibration test and read test either the A1020.

The write test, though, doesn't seem to respect the 40 track limit:

The write test is correct, but the final message is wrong. And it would be clearer as cyl.head perhaps rather than a track number. I will fix that and make another build.

Here's another build for you to try. It fixes the track-number reporting in the write test, so that's the only thing you need to try. Let me know how it goes: https://github.com/keirf/amiga-stuff/actions/runs/12372782259

Works like a charm!


[Hotfix] Update ecs.tf nginx log group to also use project name

Reviewer: @lickem22
Estimate: 5 minutes

Ticket
Hot fix
Description, Motivation and Context
Include project name in log group name for nginx like backend and frontend
How has this been tested?
Tested by destroying and recreating resources using Terraform
Checklist
Fill with x for completed. Delete any lines that are not relevant

[x] My code follows the style guidelines of this project
[x] I have reviewed my own code to ensure good quality
[x] I have tested the functionality of my code to ensure it works as intended
[x] I have resolved merge conflicts
[ ] I have updated the automated tests (if applicable)
[ ] I have updated the requirements (if applicable)
[ ] I have updated the README file (if applicable)
[ ] I have updated affected documentation (if applicable)

I am ok approving this though this will probably change when we switch to caddy?


Initial attempt to remove dep injection

lgtm, although I wish there were a way to still enable/disable plugins that didn't require a bot rebuild.
:+1:

Yeah, I don't think there's any way around that for now without exposing everything in a scripting language. :cry:


story link is expired

The link to view the story to demonstrate this "dropin" simply says the link is expired, so its not possible to see it in action.  thanks

Thanks for raising this!
For clarity, the expired link is to one of our payment flows. The reason it shows as expired is because the Storybook sets the drop-in to use a fake Billing Request Flow ID (BRF123).
This was originally by design because the intention of this Storybook was to show the button itself (and the styles of the modal) rather than show the full payment flow. But you're right; it's not showing useful context.
While we figure out a more sensible approach than linking to a dummy payment flow that doesn't exist, there's a couple of other things you can try if this is blocking you from progressing with the drop-in solution.
Firstly, if you're looking to see what the payment flow steps look like, the best place to look is our Anatomy of a Billing Request Flow document.
Secondly, if you're specifically wanting to see the modal with a payment flow inside, here's a workaround for making the Storybook work:-

Open the sandbox demo payment URL
Wait for it to redirect you to a payment flow
Copy the Billing Request Flow ID from the URL (it starts with BRF)
Open the Storybook
Click into the Controls tab
Paste the Billing Request Flow ID into the relevant control field
Try the button in the canvas and see the modal showing a payment flow

Hey @markwilson, thank you for providing the above info to solve the "expired" story.
It would probably help everyone coming across the repo to see the above workaround in the readme as this issue might be easy to miss. Alternatively (or additionally), adding it to the story docs would also improve the UX.
I am happy to raise a PR if you want me to 😃.

Thanks for the suggestion, @MinasG! I've updated the storybook now to include the instructions above.

@markwilson perfect, thank you for sorting that out.


Rewrote http.py to be faster

…, mainly by using select() instead of timeouts
Split from PR #66, part 1

I did that so the PR checks would run through and to show, that the change doesn't affect the tests.

Are the tests failing when upgrading dependencies?

Yes because, partly because the domains used in the tests are unreliable.

Well, then we should change the description on this PR to match whatever is doing. Also, if someone needs to revert it will erase both the http.py rewriting AND the tests update.
I think that the easiest way to solve this is to create a new PR for updating the tests and the reason of reliability. We merge that one. Then rebase this one so it doesn't update tests also. We merge this one.
Then rebase the following and keep going :)

That PR already exists :) Your point about reverting is a good one. I'll update the PR's and indicate the proper order in which to review them.

@fzipi @fgsch Final one.

Merging.


Allow users to select their banner map

Possibility to change/set feature map for the public dashboard.
Organizations:

Possibility to select any public map from any organization member.
If it is not set, we should set the most viewed one (public of course).
If the selected one turns into private/link/password, we should set the most viewed one (public of course).

Users:

Possibility to select any of your public maps.
If it is not set, we should set the most viewed one (public of course).
If the selected one turns into private/link/password, we should set the most viewed one (public of course).

Right now, in the public dashboard, feature map is defined with:
https://github.com/CartoDB/cartodb/blob/master/app/controllers/admin/pages_controller.rb#L245
To be done by @kartones and @xavijam.
Branch: highlighted-map

We have to decide the interaction for this

Easy, for the time being, I'd just add an option on the options dropdown on the map editor

@CartoDB/design so, option within cartodb editor?

A preview map in profile page would be easy to be implemented.

@xavijam what is your plan for api and model changes for this?

@kartones decides, I need him for this. But we talked yesterday to have (another) attribute in user/organization data, storing a map_id.

before start, define first:

changes in frontend model
changes in backend model
changes in the api

how it should behave

my idea is to just add a new User model field to store that map. Changes in the back I think it will need are:

setting it manually (access via the property)
resetting it manually to nil if affected visualization's privacy changes to anything other than public. This requires small code digging to see from where we trigger privacy changes to vis.

Ok, org stuff adds a bit more of complexity but the main point remains the same... Detecting all points from when a Visualization changes privacy OR adding logic to priv change of the vis so it grabs its owner and gets most viewed.
Doesn't looks complex, but we're just starting the Visualization refactor and we already are going to break the rule of "no new functionality" exactly with the classes being refactored :trollface:
Is not a big deal as only reads are being changed ATM but this should really be the last new feature at least regarding visualizations. @javisantana confirm please

uff, reverting to the most visited one I don't like it, because checking the code that does it currently at public dashboard, it's Visualization collection call, and Visualization Members shouldn't be aware of their parent Collections (control should be the inverse, Collections handle and know about Members), but handling privacy changes is done always inside the Member.
Can we not default to the most visited one? just "null the field" and make the frontend (or Rails template logic) render the default one? That way we keep logic minimal inside the new field and leave to the "presentation" acting if the field is empty or not

For me :+1:

if this takes changes in core model we need to wait

requires changes in both User and Visualization Member models, yes... not big, but at both.