Can't install Casperjs during Provision

The same error, below, happens each time I try. Would appreciate any pointers!
C:\dev\Ghost-Vagrant>vagrant provision
==> default: [vagrant-hostsupdater] Checking for host entries
==> default: [vagrant-hostsupdater]   found entry for: <IP_ADDRESS> local.tryghost.org
==> default: Installing Puppet modules in "." with Librarian-Puppet...
==> default: Configuring cache buckets...
==> default: Running provisioner: puppet...
==> default: Running Puppet with base.pp...
==> default: stdin: is not a tty
==> default: Warning: Config file /etc/puppet/hiera.yaml not found, using Hiera defaults
==> default: Notice: Compiled catalog for local.tryghost.org in environment production in 7.41 seconds
==> default: Notice: /Stage[main]/Ghost/Exec[apt-update]/returns: executed successfully
==> default: Notice<EMAIL_ADDRESS>created
==> default: Error: Download file checksum mismatch
==> default: Error: /Stage[main]/Casperjs/Archive[phantomjs]/ensure: change from absent to present failed: Download file checksum mismatch
==> default: Notice: /Stage[main]/Nvm/Exec[set-node-version]/returns: executed successfully
==> default: Notice: /Stage[main]/Casperjs/Exec[link-phantomjs]: Dependency Archive[phantomjs] has failures: true
==> default: Warning: /Stage[main]/Casperjs/Exec[link-phantomjs]: Skipping because of failed dependencies
==> default: Error: Could not start Service[app]: Execution of '/sbin/start app' returned 1:
==> default: Error: /Stage[main]/Upstart/Service[app]/ensure: change from stopped to running failed: Could not start Service[app]: Execution of '/sbin/start app' returned 1:
==> default: Notice: Finished catalog run in 20.66 seconds
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.

Hm, that error is saying that the downloaded PhantomJS archive does not have the same checksum as what is specified in the manifest. I've just downloaded the archive and run
sha1sum phantomjs-1.9.8-linux-x86_64.tar.bz2

and got
d29487b2701bcbe3c0a52bc176247ceda4d09d2d  phantomjs-1.9.8-linux-x86_64.tar.bz2

which is the correct checksum. If you ssh into the VM are you able to verify that /tmp/phantomjs-1.9.8-linux-x86_64.tar.bz2 has the correct checksum as above?

I thought it might be a corrupted file downloaded, but wasn't sure how to get rid of it, so destroyed the box, started again, and didn't encounter this error


[DNM]s3-test: add a mock test-case for presign-url, verify skew-time error

Signed-off-by: liuchang0812<EMAIL_ADDRESS>

This case was added for https://github.com/ceph/ceph/pull/13354
It depends on https://github.com/boto/boto/pull/3686


Support Protocols

Notably, https vs. http.

Hi, is this going to be implemented? I'm interested in this feature too.

Hi @ShacharHarshuv!  This project has long laid dormant, so I don't know when or if it would happen.  Happy to accept PR's though!


Support multi chain deployment

As a coordinator, I'd like to use one command to deploy to the selected networks.

For coordinator side: there is only one task update. It's mostly for UI because UI needs support for showing all the networks and allow to switch between them for voters.


Get.back remove all the same type Controller even though it has different tag

Get.back remove all the same type Controller even though it has different tag
**Reproduction code
example:

void main() {
  runApp(GetMaterialApp(
    debugShowCheckedModeBanner: false,
    getPages: [
      GetPage(
        name: '/page',
        page: () => PageView(),
        binding: PageBinding(),
      ),
      GetPage(
        name: '/page-detail',
        page: () => PageDetailView(),
        binding: PageDetailBinding(),
      ),
    ],
    initialRoute: '/page?tag=random1',
  ));
}

class PageController extends GetxController {
  PageController();

  final _ready = false.obs;
  bool get isReady => _ready.value;

  @override
  void onReady() async {
    super.onReady();
    _ready.value = true;
  }

  String saveFuntion() {
    return "Working";
  }
}

class PageView extends GetView<PageController> {
  @override
  String get tag => Get.parameters['tag'];

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('Page $tag'),
      ),
      body: Obx(
        () => controller.isReady
            ? Center(
                child: RaisedButton(
                  child: Text('Go to Detail'),
                  onPressed: () {
                    Get.toNamed('/page-detail?tag=$tag');
                  },
                ),
              )
            : Container(),
      ),
    );
  }
}

class PageBinding implements Bindings {
  @override
  void dependencies() {
    assert(Get.parameters['tag']?.isNotEmpty ?? false);
    Get.lazyPut(
      () => PageController(),
      tag: Get.parameters['tag'],
    );
  }
}

//page detail
class PageDetailController extends GetxController {
  final String previousTag;
  PageDetailController(this.previousTag);

  final _ready = false.obs;
  bool get isReady => _ready.value;
  //need to acesss its parent controller
  PageController tc;
  @override
  void onReady() async {
    super.onReady();
    tc = Get.find(tag: previousTag);
    _ready.value = true;
  }
}

class PageDetailView extends GetView<PageDetailController> {
  @override
  String get tag => Get.parameters['tag'];

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('Detail of $tag'),
      ),
      body: Obx(
        () => controller.isReady
            ? Center(
                child: Column(
                  mainAxisAlignment: MainAxisAlignment.center,
                  children: [
                    Text(controller.tc.saveFuntion()),
                    RaisedButton(
                      child: Text('Go to Another Page'),
                      onPressed: () {
                        var tag = randomAlphaNumeric(5);
                        Get.toNamed('/page?tag=$tag');
                      },
                    ),
                  ],
                ),
              )
            : Container(),
      ),
    );
  }
}

class PageDetailBinding implements Bindings {
  @override
  void dependencies() {
    assert(Get.parameters['tag']?.isNotEmpty ?? false);
    Get.lazyPut(
      () => PageDetailController(Get.parameters['tag']),
      tag: Get.parameters['tag'],
    );
  }
}

To Reproduce

Click Go to Detail => Detail Page of the tag pop up. Detail page need to access some function in PageController.
Click Go to Another Page => New Page with different tag pop up, another PageController created
Go back => Back to Detail page of previous tag. Both PageController got removed.
Go back => Back to Page
Click on Go to Detail => Error show up

Expected behavior
Expect only one PageController to get removed when back to Detail page.
Log
Connecting to VM Service at ws://<IP_ADDRESS>:65158/xHe3b7A9oDY=/ws
[GETX] "GetMaterialController" has been initialized
[GETX] GOING TO ROUTE /page?tag=random1
[GETX] "PageControllerrandom1" has been initialized
[GETX] GOING TO ROUTE /page-detail?tag=random1
[GETX] "PageDetailControllerrandom1" has been initialized
[GETX] GOING TO ROUTE /page?tag=r35b5
[GETX] "PageControllerr35b5" has been initialized
[GETX] CLOSE TO ROUTE /page?tag=r35b5 <- Problem here****
[GETX] "PageControllerrandom1" onClose() called
[GETX] "PageControllerrandom1" deleted from memory
[GETX] "PageControllerr35b5" onClose() called
[GETX] "PageControllerr35b5" deleted from memory
[GETX] CLOSE TO ROUTE /page-detail?tag=random1
[GETX] "PageDetailControllerrandom1" onClose() called
[GETX] "PageDetailControllerrandom1" deleted from memory
[GETX] GOING TO ROUTE /page-detail?tag=r35b5
[GETX] "PageDetailControllerr35b5" has been initialized
Error: "PageController" not found. You need to call "Get.put(PageController())" or "Get.lazyPut(()=>PageController())"
    at Object.throw_ [as throw] (http://localhost:65099/dart_sdk.js:4361:11)

Flutter Version:
[√] Flutter (Channel beta, 1.25.0-8.3.pre, on Microsoft Windows [Version 10.0.19041.746], locale en-US)
[√] Android toolchain - develop for Android devices (Android SDK version 30.0.1)
[√] Chrome - develop for the web
[√] Android Studio (version 4.0)
[√] VS Code (version 1.52.1)
[√] Connected device (2 available)
Getx Version: 3.24.0

similar issue #895 , im going ahead in referencing it, so once one of these issues resolved, you can also close the other one.

I bumped into this issue too. Both Navigator.pop and Get.back remove all binded dependencies of the given type. Is there any progress on the issue?

Currently workaround is Get.put(XXXController(), permanent: true,); and manually delete the instance

Currently workaround is Get.put(XXXController(), permanent: true,); and manually delete the instance. However, I think it is not acceptable.

for new get x version set fenix to true in dependency injection


[ASK] transformers.abstractive_summarization_bertsum.py not importing transformers

Description
I run in Google Colab the following code
!pip install --upgrade
!pip install -q git+https://github.com/microsoft/nlp-recipes.git
!pip install jsonlines
!pip install pyrouge
!pip install scrapbook

import os
import shutil
import sys
from tempfile import TemporaryDirectory
import torch
import nltk
from nltk import tokenize
import pandas as pd
import pprint
import scrapbook as sb

nlp_path = os.path.abspath("../../")
if nlp_path not in sys.path:
    sys.path.insert(0, nlp_path)

from utils_nlp import models
from utils_nlp.models import transformers
from utils_nlp.models.transformers.abstractive_summarization_bertsum \
     import BertSumAbs, BertSumAbsProcessor

It breaks on the last line and I get the following error
/usr/local/lib/python3.7/dist-packages/utils_nlp/models/transformers/abstractive_summarization_bertsum.py in <module>()
     15 from torch.utils.data.distributed import DistributedSampler
     16 from tqdm import tqdm
---> 17 from transformers import AutoTokenizer, BertModel
     18
     19 from utils_nlp.common.pytorch_utils import (

ModuleNotFoundError: No module named 'transformers'

In summary, the code in  abstractive_summarization_bertsum.py doesn't resolve transformers where it is located in the transformer folder. Is it something to be fixed on your side?

I am getting the same issue after doing a pip install of the repo into google colab.


No result for some address

Not getting any result for some address when I used "Explorer->getAddress()". PHP shows "php memory exhausted error" and I added the code below to fix this.
ini_set('memory_limit', '-1');
Now it shows "Internal server error". What I understand is, some address is taking too much time and not returning any result.
Is there any solution ?
Can I add a maximum execution time to solve this issue to either Blockchain API or in PHP ?

What address causes this errors?

19ZZ8DZsb5qgchuKPZWET7Uj8rDoj4KgmB
17RTTUAiiPqUTKtEggJPec8RxLMi2n9EZ9

By using this code
<?php
require __DIR__ . '/vendor/autoload.php';

$Blockchain = new \Blockchain\Blockchain();

$address = "...";
$limit = 1000;
$offset = 0;
$address_info = $Blockchain->Explorer->getAddress($address, $limit, $offset);
echo memory_get_peak_usage() / (1024 * 1024.0) . "\n";

with the two addresses you provided, I get peak memory uses of ~31MiB and ~9.8MiB respectively.
Raising memory_limit to 50MiB should be enough to process these addresses.
I would recommend you to use the pagination provided by $Blockchain->Explorer->getAddress($address, $limit, $offset) to lower the memory usage of your script. It enables fetching transactions by packets of $limit transactions starting at the $offset nth transaction.

Such addresses shows "Fatal Error: Allowed Memory Size of xx Bytes Exhausted"
To resolve this, I added two lines:
set_time_limit(0);
ini_set('memory_limit', '-1');

Regarding pagination, I have already set $limit to 50

Maybe lowering $limit to something like 5 or 10 would remove the need to remove memory and time limitations as the transactions involving the addresses are quite big.

Yes, I have noted that long address list on blockchain official site. But your suggestion to lowering the limit is a good suggestion and I will try with that.
Thank you.

The site is running on GoDaddy. If I change to another hosting or upgrade to higher level, will this resolve the issue permanently ?

It should resolve it, shared hosting providers usually lower the limits on memory use and runtime for PHP scripts.

Okay. Thank you.


refactor(navbar): rename ListDropdownMenus in ListNavDropdown

Based on our team's decision, we no longer require approval of the refactoring pull request. Instead, we will review and approve the refactoring once all changes related to a specific component have been completed.


Rustfmt does not format some redundant imports

Describe the bug

Rustfmt does not normalize use std::rc::Weak as Weak; to use std::rc::Weak; in some cases.
To Reproduce

x.py fmt on https://github.com/rust-lang/rust/blob/c3ed6681ff8d446e68ce272be4bf66f4145f6e29/compiler/rustc_data_structures/src/sync.rs#L224
Expected behavior
It should turn that line into use std::rc::Weak;, the same way it does on playground: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=2fbcb920f10e1ad6b4883e3970eba5cc. I'm not sure why this only breaks on rustc.
Meta

rustfmt version: rustfmt: nightly-2020-11-19
From where did you install rustfmt?: however x.py downloads it, probably CI artifacts directly?
How do you run rustfmt: x.py fmt

This might be because it's inside a cfg_if!?

Ok yes, this was because of the macro: https://play.rust-lang.org/?version=beta&mode=debug&edition=2018&gist=b39c431c5fba50549d424848e86a009d

Ok yes, this was because of the macro:

Correct. Mac call formatting is intentionally pretty limited for various reasons (complexity, failure to be semantics preserving, high potential for butchering, etc.) and that definitely holds true for items within cfg_if. rustfmt will recurse into and format any mods defined or imported within cfg_if, but not the items themselves

Going to go ahead and close as there's various other open issues like #8 around macro and mac call formatting


"Export variants to CSV" button not working for some proteins

Hi,
I am trying to download gnomad v4.0.0 data for KCNQ1 protein. I did the research from the search bar and I found the following page:
https://gnomad.broadinstitute.org/gene/ENSG00000053918?dataset=gnomad_r4
I went down and clicked on "Export variants to CSV" button but it doesn't work. It doesn't download anything.
I tested it with Brave v159.124 and Firefox v119 on Ubuntu 22.04.
For some other proteins, "Export variants to CSV" button worked when I deselected "Other" or "pLof" options just above the button but it doesn't work for this particular case.
I was wondering if you can take a look to this bug.
Thanks,
Mustafa

I just tried downloading the CSV for this gene and it worked.  I think it was resolved by #1274 (cc @rileyhgrant).
@tekpinar could you please let us know if it is working now?

Hi,
I was able to download the file.
Thanks,
Mustafa

Awesome, happy to hear it was resolved Mustafa!
Closing this as completed by #1274


aws: Split configuration to ensure early failure

When checking whether credentials are valid, we do it on a basic shared
config, and the retrier configuration should get added later. This
ensures that whenever a request is done without valid credentials fails
early with an error instead of retrying.

/cc @jharrington22


Log entries

With vermouth v0.9.0 we can now have log-entries in ff-files. This implements such entries to polyply gen_params. One nice thing to use these for is to raise a warning whenever people are not applying termini to polymers or to print ff parameters in case they are not part of the default force-field.
@pckroon do you have any idea why in the current setup info messages wouldn't be printed? Warnings and Errors are info.

@pckroon do you have any idea why in the current setup info messages wouldn't be printed? Warnings and Errors are info.

What do you mean by this? Do Warnings and Errors get logged as Info? Do Info entries not show up at all?

What's your loglevel and do you set that before parsing the ff files?

the default logleve is zero, which translates to logging.INFO. I thought that would be enough

Have you tried printing the molecule.logentries before trying to log them?

@pckroon so it seems that the info lines are not parsed because the log level evaluation does not evaluate to True. But I don't really understand why, since it also don't respond to adjustment of the Loglevel. Could it be that it imports the wrong Logger from vermouth and not polyply?

@pckroon regarding adding those tests for vermouth: the only problem is that in this case I can call the itp workflow from the library since it is a function. But for the vermouth tests we mostly run integration tests, for which the context capturing of the logs doesn't work.

I'm reasonably sure we have selective tests for ffinput.py in vermouth, but meh. Maybe later. Good intro project for new people that want to get involved maybe.

@pckroon sure for ffinput but this test here also checks that the whole thing propagates through he program and gets actually printed. The ffinput tests I can add no problem. But also good to get started indeed.


ASSERT_PARAM(0 1), in ld_sched.c at line 402 Guru Meditation Error: Core  0 panic'ed (IllegalInstruction)  (IDFGH-7495)

ASSERT_PARAM(0 1), in ld_sched.c at line 402
Guru Meditation Error: Core  0 panic'ed (IllegalInstruction). Exception was unhandled.
Memory dump at 0x40090b34: 000000f0 00004136 f01d0000
Core  0 register dump:
PC      : 0x40090b3b  PS      : 0x00060934  A0      : 0x800856ad  A1      : 0x3ffdc7a0
A2      : 0x00000000  A3      : 0x00000000  A4      : 0x00000001  A5      : 0x3ff9de2f
A6      : 0x00000192  A7      : 0xfffffffc  A8      : 0x8000814b  A9      : 0x3ffdc710
A10     : 0x00000000  A11     : 0x3ffdc733  A12     : 0x3ffdc6df  A13     : 0x00000032
A14     : 0x00000000  A15     : 0x3ffdc6e4  SAR     : 0x00000004  EXCCAUSE: 0x00000000
EXCVADDR: 0x00000000  LBEG    : 0x400855b5  LEND    : 0x400855bd  LCOUNT  : 0x00000000
Backtrace:0x40090b38:0x3ffdc7a00x400856aa:0x3ffdc7c0 0x4003fa41:0x3ffdc7e0 0x40031336:0x3ffdc800 0x4003624a:0x3ffdc820 0x400207e9:0x3ffdc840 0x40108f7d:0x3ffdc860 0x401093d2:0x3ffdc880 0x40086530:0x3ffdc8a0 0x40019d11:0x3ffdc8d0 0x40055b4d:0x3ffdc8f0 0x40104827:0x3ffdc910 0x40104ea4:0x3ffdc930
--=================================
Running on 0 core 4 threads. From the periphery, I use the Classic Bluedroid and SPI.

I use more than 200 devices. This error occurs periodically.
To avoid memory leaks, I made everything I could static. But it doesn't help.

Hi @Make4soul ,
This issue should have been resolved in Release 4.4.1.
Related Link https://github.com/espressif/esp-idf/commit/169008616b623539038f0d58524301d2abd10e2e
Thanks

Ok, Thanks!
I will be testing the 4.4.1 release .


Bug: type 'Null' is not a subtype of type 'LoginProvider' in type cast

Here is the Login Provider:
class LoginProvider extends InheritedWidget {
final bloc = new FormBloc();
LoginProvider({required Key key, required Widget child}) : super(key: key, child: child);
// constructor, forward to Parent InheritedWidget
u/override
bool updateShouldNotify(InheritedWidget _) {
// Todo implantation
return true;
}
static FormBloc of(BuildContext context) {
return (context.dependOnInheritedWidgetOfExactType() as LoginProvider).bloc;
}
}
And on my Login Page, I call that as:
Widget build(BuildContext context) {
final FormBloc formBloc = LoginProvider.of(context);
...
What do I do wrong? Thank you in advance.

Hello, I don't see any issue of your implementations.
It was best to mention the error logs.
Something you have to aware of that
 (context.dependOnInheritedWidgetOfExactType() as LoginProvider).bloc; is deprecated since then I guess now it got removed.
I don't know if that error is valid. or you still working on the project. sorry for late answer.


SP points to last-1 instead of last push

initial SP = 0000
run:
MVI A,F0H
INR A
PUSH PSW
HLT

SP = FFFD, it should be FFFE instead.
SP isn't pointing to the address containing the last PUSHed byte but rather the next available one. This is an error, according to Intel's specification (see 3-44, 3-45) it should be pointing to the address of the last one.

Nice catch! In order to adhere to that, I'm probably going to start the SP at address FFFE. It could still start at FFFF, but since every PUSH is pushing 16 bits, I'd like to keep those aligned nicely in the memory.
That way:
If SP=FFFE, the stack is empty
SP will always pointsto the last added data.
If SP > FFFE, you've popped an empty stack (which will, in the future, notify you and give you a warning).
Does that sound okay?

Sounds perfect, and FFFE is the starting address of some old, respected but outdated simulators.

Fixed as of edc7bc007c253a4d4482d7cf745677e50b7ce663.
Thank you!


getCountForPagination and Oracle

There is an issue with the getCountForPagination (framework/src/Illuminate/Database/Query/Builder.php) when using Oracle driver.
In MySQL it is possible to do:
SELECT COUNT(DISTINCT column1, column2, columnN) FROM table;
However, it is not possible to do the same in Oracle. It is necessary to do something like:
SELECT COUNT(*) FROM (SELECT DISTINCT column1, column2, columnN from table);
For these reason, the call to getCountForPagination fails when setting multiple columns. (e.g.: getCountForPagination(['column1', 'column2', 'columnN'])), as it try to do:
SELECT COUNT(DISTINCT column1, column2, columnN) as aggregate...
Is there any simple solution to this issue?
Thanks.

Does https://github.com/laravel/framework/pull/9055 fix this?

@GrahamCampbell, Don't think my PR would fix this for the normal count, but it would fix for when doing this with a groupBy Call

It does not fix the problem for this particular case.

Closing due to lack of activity. We're open to PRs though.


Impl std::fmt::Display and std::error::Error for RfcErrorInfo

With the implementations for Display and Error, RfcErrorInfo can be used with anyhow to make error handling easier.

Sorry for the late reaction and thank you very much for the pull request!


Can't access UIMenuController iOS native API properly

here is an sample code:
html:
<StackLayout (loaded)="onStackLoaded($event)">
    <Label
        #label
        (tap)="onTap()"
        [text]="text"
    ></Label>
</StackLayout>

ts:
@Component(...)
export class MessageComponent {

    @Input() text;
    @ViewChild('label', { static: false }) labelRef: ElementRef;
    private wrapperView: UIView;

    onStackLoaded(args: EventData) {
        const stackLayout = args.object as StackLayout;
        this.wrapperView = stackLayout.ios;
    }

    onTap() {
        const labelElem: Label = this.labelRef.nativeElement;
        labelElem.focus(); // don't know if i need this

        const menu = UIMenuController.sharedMenuController;
        menu.update(); // don't know if i need this

        menu.menuItems = NSArray.arrayWithArray([
            UIMenuItem.alloc().initWithTitleAction('copy', null),
            UIMenuItem.alloc().initWithTitleAction('delete', null),
        ]);

        const nativeLabel = labelElem.ios as UILabel;
        menu.setTargetRectInView(this.wrapperView.frame, nativeLabel);
        menu.setMenuVisibleAnimated(true, true);
    }
}

After some hours of "go horse" the result was the same, nothing appears on the screen.
I just started trying to use native API recently so, maybe something is wrong on that component. i tried to use the 'page' injected by the constructor but the line this.page.ios.frame returns undefined. so i tried the wrapper frame that returns something like <struct CGRect: 0x600003aa8bc0>.
I also already know the nativescript-tooltip and nativescript-popup plugins but none of these have a good UI by default.

@yassern try something like this - you will still need to create the logic for your custom actions.

@NickIliev The playground code is working on TextField. But it still doesn't work on Label


Interaction constraints fail with monotone_constraints_method=intermediate

Description
Interaction constraints are not enforced with monotone_constraints and monotone_constraints_method=intermediate or monotone_constraints_method=advanced. With monotone_constraints_method=basic interaction constraints appear to be enforced. I suspect this is a bug (see more below), but it might be by design. In the latter case, updating the documentation would be nice.
Reproducible example
# %%
import lightgbm as lgb
import numpy as np
import pandas as pd

# %%
def simple_pd(X, estimator, feature, values):
    """Calculate simple partial dependency."""
    Xc = X.copy()
    yps = np.zeros_like(values)
    for i, x in enumerate(values):
        Xc[feature] = x
        yps[i] = np.mean(estimator.predict(Xc.values))
    return values, yps

def simple_pds(df, estimator, features):
    """Calculate simple partial dependency for all features."""
    pds = {}
    for feat in features:
        values, yps = simple_pd(
            df[features], estimator, feat, np.sort(df[feat].unique())
        )
        pds[feat] = pd.DataFrame(data={feat: values, "y": yps})
    return pds

def make_data():
    """Make toy data."""
    np.random.seed(1)
    n = 10000
    d = 3
    X = np.random.normal(size=(n, d))
    # round to speed things up
    X = np.round(X)
    eps = np.random.normal() * 0.1
    y = -1 * X[:, 0] + 3 * X[:, 1] + X[:, 0] * X[:, 1] + np.cos(X[:, 2]) + eps
    df = pd.DataFrame(data={"y": y, "x0": X[:, 0], "x1": X[:, 1], "x2": X[:, 2]})
    features = ["x0", "x1", "x2"]
    outcome = "y"

    return df, features, outcome

def find_interactions(gbm, feature_sets):
    """Find interactions in tree.

    Parameters
    ---------
    gbm: booster
    feature_sets: list of list
        set of features across which to check for interactions

    Returns
    -------
    tree_features: pandas.DataFrame
        boolean flag for every tree of has interaction across feature sets.
    """
    df_trees = gbm.trees_to_dataframe()
    tree_features = (
        df_trees.groupby("tree_index")
        .apply(lambda x: set(x["split_feature"]) - set([None]))
        .reset_index()
        .rename(columns={0: "features"})
    )

    def has_interaction(tree):
        n = 0
        for fs in feature_sets:
            if len(tree["features"].intersection(fs)) > 0:
                n += 1
        if n > 1:
            return True
        else:
            return False

    tree_features["has_interaction"] = tree_features.apply(has_interaction, axis=1)

    return tree_features

# %%

boosting_params = {
    "boosting_type": "gbdt",
    "objective": "regression",
    "num_leaves": 5,
    "learning_rate": 0.1,
    "num_boost_round": 100,
    # disallow all interactions
    "interaction_constraints": [[0], [1], [2]],
    "monotone_constraints": [1, 1, 0],
    # fails for intermediate, advanced. works for basic.
    "monotone_constraints_method": "intermediate",
}

df, features, outcome = make_data()
data = lgb.Dataset(df[features], df[outcome])
gbm = lgb.train(boosting_params, data)
# %%
feature_sets = [[0], [1], [2]]
feature_sets = [[f"x{f}" for f in fs] for fs in feature_sets]
tree_features = find_interactions(gbm, feature_sets)

# Should not find any co-occurances in a given tree, since above we're disallowing all interactions.
if tree_features["has_interaction"].any():
    print(tree_features.loc[tree_features["has_interaction"] == True].head())

assert tree_features["has_interaction"].any() == False

# %%
# Check monotonicity
pds = simple_pds(df, gbm, features)
cnt = 0
for feat, df_pd in pds.items():
    df_pd = df_pd.sort_values(by=feat, ascending=True)
    y_pred_diff = df_pd["y"].diff().values[1:]
    if boosting_params["monotone_constraints"][cnt] == 1:
        assert (y_pred_diff >= 0).all()
    elif boosting_params["monotone_constraints"][cnt] == -1:
        assert (y_pred_diff <= 0).all()
    else:
        pass
    cnt += 1
# %%

Environment info
LightGBM version or commit hash: 87c37bf04f12cdf15498adfec6bf0baba5498ab2
Command(s) you used to install LightGBM
mkdir -p build
cd build
cmake -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
    -GNinja \
    -DCMAKE_CXX_FLAGS="-ggdb -fno-omit-frame-pointer ${CXXFLAGS}" \
    -DCMAKE_C_FLAGS="-ggdb -fno-omit-frame-pointer ${CFLAGS}" \
    -DUSE_DEBUG=ON ..
ninja -j2

Additional Comments
I suspect that the underlying issue is here: https://github.com/microsoft/LightGBM/blob/87c37bf04f12cdf15498adfec6bf0baba5498ab2/src/treelearner/serial_tree_learner.cpp#L807. To enforce interaction constraints when leaf splits are recomputed due to monotonicity constraints, one would also have to restrict the set of permitted features here (rather than setting is_feature_used=true by default). I have attempted a solution that manages to resolve the issue in #4043.

Fixed via #4043.


Consider interactive questioning?

Thanks for this great project. But it's a bit tedious to type iwhat at the command line every time, is it considered possible to make it an interactive question?
For example:
$ iwhat --interactive
> MySQL
<answer>
> Python
<answer>

maybe we can


This closes #2 and #3

Hello,
First of all thanks for your contribution.
I think that adding those packages into runtime dependencies is not an ideal solution, because we will force everyone to load those in their application, even if they don't need them.
Ideal solution would be probably to create a separate file for example directory or at least add them to devDependencies. I would prefer the first solution.
What's your opinion on that?

You alright, we can create a package for the example only.


Add missing n_batch attribute to BasePolicy

We store other properties such as n_env and n_steps but not n_batch.
Useful to know n_batch in many settings. In my case, I'm making a wrapper around a BasePolicy object so need to know this to satisfy the constructor. In general since it also specifies the shape of the placeholders, callers might need to know it to feed in appropriately sized batches.

I don't know if an entry should be added in the changelog for that...

Added, think you're right it's worth mentioning, although I don't like it when the changelog is 50% of my diff ;)

Ok, we need @hill-a here, so he can by-pass travis (when using ci-skip, then the tests are not run and github does not let us merge)
another solution is a dummy commit but I don't like that too much.


test: Why is TS workflow failing

yarn typecheck:tests is failing in CI. Looks like it's using TS 5.0 but no idea if that assumption is correct or why it would use 5.0

See #14028

My guess is that ae77cc8 (#14055) fixes it?
We just used to restore the yarn.lock but then we need to re-install from my experience.

Looks right to me. Thanks.


Partials fallback to lookup by filename

I wish we could have this cemented in the spec, specifically:
When the lookup of a partial named 'sidebar.mustache' fails, lookup would fallback to the file named 'sidebar.mustache'. If rendering is being done on the client, then this simply doesn't occur.
Of course we need to support full file paths like../../layout.mustachebut I think this is trivial for most languages. In PHP the path string is simply placed into a file_get_contents call. However, I'd like to hear the opinions of some implementors on this. When partials reference more partials, there's potential to mixup paths, for example:
/partials/one.mustache:
Here's one.mustache

/partials/two.mustache:
Here's two.mustache
{{> one.mustache}}

/final.mustache:
Here's final.mustache
{{> partials/two.mustache}}

Since references would be relative, implementations need to take care to look for the file relative to the file including it.

Closing as this does not seem to be a spec issue.


In case of discount as percentage showing incorrect discount amount but grand total is correct.

Framework Version: 0.1.6

Created a rule having discount as percentage 10, added a product having price 240 and its showing discount amount as 23.04 .

@jyoti-webkul Not an issue as exchange rates were applied.


Update code of conduct #5090

Summary
As set out in Issue #5090 update email, the email address in the code of conduct is incorrect.
Full change-log

Update support email in code of conduct

Issues resolved
Fix #5090

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Running TLAPM in a specification which extends Randomization results in a crash

I get the following TLAPM Warning when I attempt to run TLAPM on any theorem in a specification which extends Randomization, even if the said theorem does not use any definitions from the module.
File "/foo/bar/dir/TLA+/Specification.tla", line 2, characters 38-50
Unknown module "Randomization"

I get the following Problem Occurred prompt right afterwards:
'Prover Launch' has encountered a problem.
Error running tlapm. Report a bug with the error code to the developers at https://github.com/tlaplus/tlapm/issues.

 Error code: 3

Here is the corresponding line 2 of the specification:
EXTENDS Integers, TLAPS, FiniteSets, Randomization
Upon removing all references to Randomization and related expressions, TLAPM works as expected without raising any errors.

Randomization is part of TLC.  You can simply copy&paste it into your spec directory.
Is the approach described in https://lamport.azurewebsites.net/tla/inductive-invariant.pdf the reason why Randomization appears in your spec?

Randomization is part of TLC.  You can simply copy&paste it into your spec directory.
Is the approach described in https://lamport.azurewebsites.net/tla/inductive-invariant.pdf the reason why Randomization appears in your spec?

You may want to try using tlapm -I /path/to/Randomization.tla for extending the search path of tlapm. (From the Toolbox, use C-G C-P and enter the option at the bottom of the window.)

On 23 Jan 2021, at 00:28, Ugur Yavuz<EMAIL_ADDRESS>wrote:
I get the following TLAPM Warning when I attempt to run TLAPM on any theorem in a specification which extends Randomization, even if the said theorem does not use any definitions from the module.
File "/foo/bar/dir/TLA+/Specification.tla", line 2, characters 38-50
Unknown module "Randomization"
I get the following Problem Occurred prompt right afterwards:
'Prover Launch' has encountered a problem.
Error running tlapm. Report a bug with the error code to the developers at https://github.com/tlaplus/tlapm/issues.
Error code: 3
Here is the corresponding line 2 of the specification:
EXTENDS Integers, TLAPS, FiniteSets, Randomization
Upon removing all references to Randomization and related expressions, TLAPM works as expected without raising any errors.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/tlaplus/tlapm/issues/28, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZCW5OFYNUKIKKZVCET7ZTS3ICZJANCNFSM4WPFURDA.

You may want to try using tlapm -I /path/to/Randomization.tla for extending the search path of tlapm. (From the Toolbox, use C-G C-P and enter the option at the bottom of the window.)

On 23 Jan 2021, at 00:28, Ugur Yavuz<EMAIL_ADDRESS>wrote:
I get the following TLAPM Warning when I attempt to run TLAPM on any theorem in a specification which extends Randomization, even if the said theorem does not use any definitions from the module.
File "/foo/bar/dir/TLA+/Specification.tla", line 2, characters 38-50
Unknown module "Randomization"
I get the following Problem Occurred prompt right afterwards:
'Prover Launch' has encountered a problem.
Error running tlapm. Report a bug with the error code to the developers at https://github.com/tlaplus/tlapm/issues.
Error code: 3
Here is the corresponding line 2 of the specification:
EXTENDS Integers, TLAPS, FiniteSets, Randomization
Upon removing all references to Randomization and related expressions, TLAPM works as expected without raising any errors.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/tlaplus/tlapm/issues/28, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZCW5OFYNUKIKKZVCET7ZTS3ICZJANCNFSM4WPFURDA.

@muenchnerkindl, that does not seem to solve the problem. In my macOS installation of the TLA+ Toolbox, Randomization seems to be in the /Applications/TLA+ Toolbox.app/Contents/Eclipse/plugins/org.lamport.tlatools_<IP_ADDRESS>004251858/tla2sany/StandardModules/ directory along with the other standard modules. Adding this directory using tlapm -I yields the following error:
Oops, this seems to be a bug in TLAPM.
Please give feedback to developers.

 Failure("Module.Parser.parse_file")
Raised at file "stdlib.ml", line 29, characters 17-33
Called from file "m_save.ml", line 168, characters 28-50
Called from file "set.ml", line 381, characters 34-55
Called from file "set.ml", line 381, characters 39-54
Called from file "m_save.ml", line 149, characters 20-1023
Called from file "m_save.ml", line 27, characters 16-20
Called from file "tlapm.ml", line 328, characters 14-65
Called from file "tlapm.ml", line 348, characters 8-33

version == "1.4.5 (build 33809)"
built_with == "OCaml 4.09.0"
tlapm_executable == "/usr/local/bin/tlapm"
max_threads == 4
library_path == "/usr/local/lib/tlaps"
search_path == << "/Applications/TLA+ Toolbox.app/Contents/Eclipse/plugins/org.lamport.tlatools_<IP_ADDRESS>004251858/tla2sany/StandardModules/"
                , "/usr/local/lib/tlaps/" >>
zenon == "PATH='/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/lib/tlaps/bin'; zenon -p0 -x tla -oisar -max-time 1d \"$file\""
zenon version == "zenon version 0.8.4 [a268] 2017-11-14"
flatten_obligations == TRUE
normalize == TRUE

And these two warnings upon closing the error window:
File "/Applications/TLA+ Toolbox.app/Contents/Eclipse/plugins/org.lamport.tlatools_<IP_ADDRESS>004251858/tla2sany/StandardModules/Randomization.tla", line 48, character 31
Unexpected EOF

File "/Applications/TLA+ Toolbox.app/Contents/Eclipse/plugins/org.lamport.tlatools_<IP_ADDRESS>004251858/tla2sany/StandardModules/Randomization.tla", line 49, character 1
Unexpected TAB character. TLAPS does not handle TAB characters in source files.

@lemmy, That is correct -- I am attempting to verify an inductive invariant for a large model and I saw the Randomization approach in that document. Your suggestion seems to work, as I have been able to run TLAPM without any issues once I copied Randomization.tla into my spec directory. But I'm still curious as to why this error happens, given that it doesn't happen with any of the other standard modules.

@muenchnerkindl, that does not seem to solve the problem. In my macOS installation of the TLA+ Toolbox, Randomization seems to be in the /Applications/TLA+ Toolbox.app/Contents/Eclipse/plugins/org.lamport.tlatools_<IP_ADDRESS>004251858/tla2sany/StandardModules/ directory along with the other standard modules. Adding this directory using tlapm -I yields the following error:
Oops, this seems to be a bug in TLAPM.
Please give feedback to developers.

 Failure("Module.Parser.parse_file")
Raised at file "stdlib.ml", line 29, characters 17-33
Called from file "m_save.ml", line 168, characters 28-50
Called from file "set.ml", line 381, characters 34-55
Called from file "set.ml", line 381, characters 39-54
Called from file "m_save.ml", line 149, characters 20-1023
Called from file "m_save.ml", line 27, characters 16-20
Called from file "tlapm.ml", line 328, characters 14-65
Called from file "tlapm.ml", line 348, characters 8-33

version == "1.4.5 (build 33809)"
built_with == "OCaml 4.09.0"
tlapm_executable == "/usr/local/bin/tlapm"
max_threads == 4
library_path == "/usr/local/lib/tlaps"
search_path == << "/Applications/TLA+ Toolbox.app/Contents/Eclipse/plugins/org.lamport.tlatools_<IP_ADDRESS>004251858/tla2sany/StandardModules/"
                , "/usr/local/lib/tlaps/" >>
zenon == "PATH='/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/lib/tlaps/bin'; zenon -p0 -x tla -oisar -max-time 1d \"$file\""
zenon version == "zenon version 0.8.4 [a268] 2017-11-14"
flatten_obligations == TRUE
normalize == TRUE

And these two warnings upon closing the error window:
File "/Applications/TLA+ Toolbox.app/Contents/Eclipse/plugins/org.lamport.tlatools_<IP_ADDRESS>004251858/tla2sany/StandardModules/Randomization.tla", line 48, character 31
Unexpected EOF

File "/Applications/TLA+ Toolbox.app/Contents/Eclipse/plugins/org.lamport.tlatools_<IP_ADDRESS>004251858/tla2sany/StandardModules/Randomization.tla", line 49, character 1
Unexpected TAB character. TLAPS does not handle TAB characters in source files.

@lemmy, That is correct -- I am attempting to verify an inductive invariant for a large model and I saw the Randomization approach in that document. Your suggestion seems to work, as I have been able to run TLAPM without any issues once I copied Randomization.tla into my spec directory. But I'm still curious as to why this error happens, given that it doesn't happen with any of the other standard modules.

@uguryavuz Consider moving the Randomization-related definitions out of your spec into a dedicated MC.tla module that is used for model-checking.

@uguryavuz Consider moving the Randomization-related definitions out of your spec into a dedicated MC.tla module that is used for model-checking.

The second error reported above (https://github.com/tlaplus/tlapm/issues/28#issuecomment-766099236) was caused due to tlapm not parsing tab characters. It has been addressed by a change to the module Randomization in commit https://github.com/tlaplus/tlaplus/commit/6ed7d29a47ec25fe8c96d86a7e55b62112f3a2cf.


Usage

Hi Shaun
Sorry for raising this as an issue but I couldn't find a way to contact you.
I've had homebridge running for a while and I was really pleased to find your Sky HD implementation. I've installed your code and amended my config file but despite ranting at Siri like a mad man, I cannot get him to understand what I'm trying to do.
Do you have some example sentences that work with your plugin?
I've tried every variation on "What channel is the Sky Box set to" but Siri complains that he doesn't understand "channel".
EDIT: Just to add, I'm running the Elgato Eve app and can see;
Power (On)
Channel Name (Blank)
Channel Number (101)
Channel Show (Blank)

Hi there,
What you are seeing is the expected behaviour. Siri only knows about a few different types of command and so it doesn't allow you to set arbitrary properties via voice. HomeKit however does allow devices to have lots of different properties.
So you should be able to use elgato to set those things as part of a scene and have Siri set that scene. But you won't be able to have Siri set to any arbitrary channel number via voice. Sadly this is a restriction of the SIri/HomeKit system

Had to format my mac but got it working now, its great plugin. Have you had trouble setting numbers in scenes in the elgato app ? I can only select certain channels like 177 but not 178

Hi Shaun
Thanks for the reply. Is it also correct that the channel number shows as 101 but it was actually on 614 at the time?

That's a little bit of a bug that I'm going to try look into. Right now it catches the channel change events and updates accordingly. But I'd like to modify it slightly so that it updates more frequently. I'll update this issue when I get a chance to look into that one.
On 28 November 2016 at 18:05:12, DJBenson (notifications@github.com(mailto:notifications@github.com)) wrote:

Hi Shaun
Thanks for the reply. Is it also correct that the channel number shows as 101 but it was actually on 614 at the time?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub(https://github.com/shauncampbell/homebridge-sky-plus-hd/issues/3#issuecomment-263423512), or mute the thread(https://github.com/notifications/unsubscribe-auth/AAw1My58eJZpuKREGYu-cMcsehE-HVvUks5rC14ogaJpZM4K9-m-).


Not working from clean install?

From a clean install of OS X, I installed Node v0.12, Atom, atom-typescript, and autocomplete plus. I made sure all of them are completely updated to their latest versions. I also restarted Atom after it prompted me to, following the atom-typescript installation.
However, atom-typescript refuses to:

Build typescript files I have open in Atom, even when I force it with cmd+shift+b.
Provide autocomplete suggestions.

Is there any way to debug why atom-typescript isn't doing anything? Is there a prerequisite piece of software that I might be missing?

I figured it out. My project does not have a tsconfig.json file, and for some reason some tool I invoked at one point generated an invalid one.
Thus, this is related to #138. Maybe when you are working on a file not mentioned in a tsconfig.json in a parent directory, then you don't use that tsconfig.json?

Is there any way to debug why atom-typescript isn't doing anything

some tips https://github.com/TypeStrong/atom-typescript/blob/master/CONTRIBUTING.md#workflow

Maybe when you are working on a file not mentioned in a tsconfig.json in a parent directory, then you don't use that tsconfig.json?

duplicate of https://github.com/TypeStrong/atom-typescript/issues/45 its not a high priority task as it is really an error on the users behalf


node 0.12 and iojs on travis

This replaces 0.10 with 0.12 and iojs. Should we keep 0.10 or carry on?
/cc @jcrugzz @juliangruber @mcollina @rvagg

keep 0.10 for the time being, it is not going away anytime soon :/
Il giorno sab 21 mar 2015 alle 15:41 Lars-Magnus Skog <
<EMAIL_ADDRESS>ha scritto:

This replaces 0.10 with 0.12 and iojs. Should we keep 0.10 or carry on?
/cc @jcrugzz https://github.com/jcrugzz @juliangruber
https://github.com/juliangruber @mcollina https://github.com/mcollina
@rvagg https://github.com/rvagg
You can view, comment on, or merge this pull request online at:
https://github.com/rvagg/node-levelup/pull/317
Commit Summary

node 0.12 and iojs on travis

File Changes

M .travis.yml
https://github.com/rvagg/node-levelup/pull/317/files#diff-0 (3)

Patch Links:

https://github.com/rvagg/node-levelup/pull/317.patch
https://github.com/rvagg/node-levelup/pull/317.diff

—
Reply to this email directly or view it on GitHub
https://github.com/rvagg/node-levelup/pull/317.

@mcollina Roger that. Yeap it makes sense! Updating.

So then this is basically a non issue now since we didn't remove anything from testing, just added more environments.

Given it's all green, LGTM.
Il giorno sab 21 mar 2015 alle 16:16 Lars-Magnus Skog <
<EMAIL_ADDRESS>ha scritto:

So then this is basically a non issue now since we didn't remove anything
from testing, just added more environments.
—
Reply to this email directly or view it on GitHub
https://github.com/rvagg/node-levelup/pull/317#issuecomment-84370364.


Fix for stack and stacks command

cf version 6.11.0-bba7fcf-2015-04-14T00:39:44+00:00
For  stack  command when we type
cf stack -h
NAME:
stack - Show information for a stack (a stack is a pre-built file system, including an operating system, that can run apps)
USAGE:
cf stack
OPTIONS:
--guid       Retrieve and display the given stack's guid. All other output for the stack is suppressed.
It is not showing how to use the  stack  command and if we follow the current usage instruction of  stack  command we are getting
cf stack
FAILED
Incorrect Usage.
NAME:
stack - Show information for a stack (a stack is a pre-built file system, including an operating system, that can run apps)
USAGE:
cf stack
OPTIONS:
--guid       Retrieve and display the given stack's guid. All other output for the stack is suppressed.
for  stacks  command when we type
cf stacks hhh
Getting stacks in org shweOrg / space secspace as admin...
OK
name         description
lucid64      Ubuntu 10.04 on x86-64
cflinuxfs2   Cloud Foundry Linux-based filesystem
It should through error as per the USAGE of  stacks  command.
After fix :
./out/cf stack -h
NAME:
stack - Show information for a stack (a stack is a pre-built file system, including an operating system, that can run apps)
USAGE:
cf stack STACK_NAME
OPTIONS:
--guid       Retrieve and display the given stack's guid. All other output for the stack is suppressed.
When we type
./out/cf stacks hhhh
FAILED
Incorrect Usage.
NAME:
stacks - List all stacks (a stack is a pre-built file system, including an operating system, that can run apps)
USAGE:
cf stacks

Hi @maximilien , Fixed all the comments. We are ready to fix ,If you have any more comments.
Thanks & Regards,
SriniCH


Attempted rollback of ArchiveFile operations results in Exception

BUG DESCRIPTION
Problem

Any exception that causes an attempted rollback of an ArchiveFile operation results in a Exception attempting to delete a file for an existing record.
Steps to reproduce
Any exception encountered when attempting to load an mzXML file will result in this exception, however one way to reproduce it relatively simply is to attempt to load an mzXML file that does not match any existing sample.

Login to tracebase-devssh tracebase-dev.princeton.edu

sudo as the tracebase usersudo -iu tracebase

Source the tracebase conda envsource /usr/local/tracebase/bin/activate

Attempt to load an mzXML file that does not correspond to an existing samplecd "/tracebase-staging/loaded/exp027f_long_kli_timecourse"
python /var/www/tracebase/manage.py load_msruns \
  --mzxml-files './220909_exp027f4_free plasma and tissues/scan2/skipped_samples/exp027f4_free_M01_01_plasma.mzXML' \
  --instrument "Exploris480" \
  --date "2022-09-09" \
  --lc-protocol-name "polar-HILIC-25-min" \
  --operator "Michael Neinast"

Current behavior

  File "/var/www/tracebase/manage.py", line 22, in <module>
    main()
  File "/var/www/tracebase/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 382, in handle_wrapper
    retval = fn(self, *args, **options)
  File "/var/www/tracebase/DataRepo/management/commands/load_msruns.py", line 136, in handle
    self.load_data()
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 423, in load_data
    return self.loader.load_data(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/table_loader.py", line 1951, in load_wrapper
    retval = fn(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 385, in load_data
    sample = self.get_sample_by_name(sample_name, from_mzxml=True)
  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 786, in get_sample_by_name
    self.aggregated_errors_object.buffer_error(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 1415, in buffer_error
    return self.buffer_exception(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 1369, in buffer_exception
    buffered_tb_str = self.get_buffered_traceback_string()
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 1349, in get_buffered_traceback_string
    for step in traceback.format_stack()

The above caught exception had a partial traceback:

  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 778, in get_sample_by_name
    rec = Sample.objects.get(name=sample_name)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/db/models/query.py", line 637, in get
    raise self.model.DoesNotExist(
EXCEPTION2(ERROR): RecordDoesNotExist: Sample record matching the mzXML file's basename [exp027f4_free_M01_01_plasma] does not exist.  Please identify the associated sample and add a row with it, the matching mzXML file name(s), and the Sequence Name to the Peak Annotation Details sheet/file.
  File "/var/www/tracebase/manage.py", line 22, in <module>
    main()
  File "/var/www/tracebase/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 382, in handle_wrapper
    retval = fn(self, *args, **options)
  File "/var/www/tracebase/DataRepo/management/commands/load_msruns.py", line 136, in handle
    self.load_data()
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 423, in load_data
    return self.loader.load_data(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/table_loader.py", line 1951, in load_wrapper
    retval = fn(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 400, in load_data
    self.clean_up_created_mzxmls_in_archive()
  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 1507, in clean_up_created_mzxmls_in_archive
    self.aggregated_errors_object.buffer_error(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 1415, in buffer_error
    return self.buffer_exception(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 1369, in buffer_exception
    buffered_tb_str = self.get_buffered_traceback_string()
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 1349, in get_buffered_traceback_string
    for step in traceback.format_stack()
EXCEPTION3(ERROR): ProgrammingError: Cannot delete a file [/tracebasedev-archive/archive/archive_files/2024-06/ms_data/exp027f4_free_M01_01_plasma_wP9ious.mzXML] from the os that is associated with existing ArchiveFile database record: {'id': 652, 'filename': 'exp027f4_free_M01_01_plasma.mzXML', 'checksum': 'f9d2074dff00cf659a0888617bc92163725a45e4', 'file_location': <FieldFile: archive_files/2024-06/ms_data/exp027f4_free_M01_01_plasma_wP9ious.mzXML>, 'data_type': 2, 'data_format': 4}.
mzXML file rollback disk archive clean up stats: 0 deleted, 0 failed to be deleted, and 0 expected files did not exist.

Expected behavior

The RecordDoesNotExist exception should be reported and the copy of mzXML file should no longer be in the archive directory.
Suggested Change

Comment

ISSUE OWNER SECTION
Assumptions

List of assumptions made WRT the code
E.g. We will assume input is correct (explaining why there is no validation)

Limitations

A list of things this work will specifically not do
E.g. This feature will only handle the most frequent use case X

Affected Components

change: File path or DB table ...
add: Environment variable or server setting
delete: External executable or cron job

Requirements

[ ] 1. List of numbered conditions to be met for the feature
[ ] 2. E.g. Every column/row must display a value, i.e. cannot be empty
[ ] 3. Numbers for reference & checkboxes for progress tracking

DESIGN
GUI Change description

None provided
Code Change Description

None provided
Tests

[ ] 1. A description of at least one test for each requirement above.
[ ] 2. E.g. Test for req 2 that there's an exception when display value is ''
[ ] 3. Numbers for reference & checkboxes for progress tracking

Just a few notes...
Not important for this specific issue, but the sample record indeed does not exist:
In [1]: from DataRepo.models import Sample

In [2]: Sample.objects.get(name="exp027f4_free_M01_01_plasma")
---------------------------------------------------------------------------
DoesNotExist                              Traceback (most recent call last)
<ipython-input-2-c7f022d0b3cd> in <module>
----> 1 Sample.objects.get(name="exp027f4_free_M01_01_plasma")

/usr/local/tracebase/lib/python3.9/site-packages/django/db/models/manager.py in manager_method(self, *args, **kwargs)
     85             @wraps(method)
     86             def manager_method(self, *args, **kwargs):
---> 87                 return getattr(self.get_queryset(), name)(*args, **kwargs)
     88
     89             return manager_method

/usr/local/tracebase/lib/python3.9/site-packages/django/db/models/query.py in get(self, *args, **kwargs)
    635             return clone._result_cache[0]
    636         if not num:
--> 637             raise self.model.DoesNotExist(
    638                 "%s matching query does not exist." % self.model._meta.object_name
    639             )

DoesNotExist: Sample matching query does not exist.

I think that probably, the ArchiveFile record should be allowed to be created.  No expected sample was provided with the mzXML file.  It's the attempted automated lookup based on the filename that fails.  It should probably be a warning, not an error that causes a rollback.  We want to be able to load mzXML files at any time.
Another note is that this sample, according to the Samples sheet is a sample that is to be excluded.  Thus, the Peak Annotation Details sheet must be supplied and skip must be entered in the skip column to skip it.  This could suggest that there could exist a way to supply a --skip option to skip MSRunSamples to complement the automatic handling of mapping mzxml file names to samples in the database.  (And again, this is unrelated to this specific issue.)
The first exception is a NoLoadData warning.  This exception is suppressed from the summary, because it is handled retro-actively.

Lastly, here is an updated trace (during buffering) for all the exceptions that come out:
  File "/var/www/tracebase/manage.py", line 22, in <module>
    main()
  File "/var/www/tracebase/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 408, in handle_wrapper
    retval = fn(self, *args, **options)
  File "/var/www/tracebase/DataRepo/management/commands/load_msruns.py", line 150, in handle
    self.load_data()
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 448, in load_data
    return self.loader.load_data(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/base/table_loader.py", line 2110, in load_wrapper
    self.check_dataframe_headers()
  File "/var/www/tracebase/DataRepo/loaders/base/table_loader.py", line 1332, in check_dataframe_headers
    self.aggregated_errors_object.buffer_warning(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2366, in buffer_warning
    return self.buffer_exception(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2315, in buffer_exception
    buffered_tb_str = self.get_buffered_traceback_string()
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2295, in get_buffered_traceback_string
    for step in traceback.format_stack()
EXCEPTION1(WARNING): NoLoadData: No dataframe [df] provided.  Nothing to load.
  File "/var/www/tracebase/manage.py", line 22, in <module>
    main()
  File "/var/www/tracebase/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 408, in handle_wrapper
    retval = fn(self, *args, **options)
  File "/var/www/tracebase/DataRepo/management/commands/load_msruns.py", line 150, in handle
    self.load_data()
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 448, in load_data
    return self.loader.load_data(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/base/table_loader.py", line 2114, in load_wrapper
    retval = fn(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/models/maintained_model.py", line 1241, in wrapper
    fn(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 492, in load_data
    sample = self.get_sample_by_name(sample_name, from_mzxml=True)
  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 1049, in get_sample_by_name
    self.aggregated_errors_object.buffer_error(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2361, in buffer_error
    return self.buffer_exception(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2315, in buffer_exception
    buffered_tb_str = self.get_buffered_traceback_string()
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2295, in get_buffered_traceback_string
    for step in traceback.format_stack()

The above caught exception had a partial traceback:

  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 1041, in get_sample_by_name
    rec = Sample.objects.get(name=sample_name)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/db/models/query.py", line 637, in get
    raise self.model.DoesNotExist(
EXCEPTION2(ERROR): RecordDoesNotExist: Sample record matching the mzXML file's basename [exp027f4_free_M01_01_plasma] does not exist.  Please identify the associated sample and add a row with it, the matching mzXML file name(s), and the Sequence Name to the Peak Annotation Details sheet/file.
  File "/var/www/tracebase/manage.py", line 22, in <module>
    main()
  File "/var/www/tracebase/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 408, in handle_wrapper
    retval = fn(self, *args, **options)
  File "/var/www/tracebase/DataRepo/management/commands/load_msruns.py", line 150, in handle
    self.load_data()
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 448, in load_data
    return self.loader.load_data(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/base/table_loader.py", line 2114, in load_wrapper
    retval = fn(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/models/maintained_model.py", line 1241, in wrapper
    fn(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 507, in load_data
    self.clean_up_created_mzxmls_in_archive()
  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 1794, in clean_up_created_mzxmls_in_archive
    self.aggregated_errors_object.buffer_error(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2361, in buffer_error
    return self.buffer_exception(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2315, in buffer_exception
    buffered_tb_str = self.get_buffered_traceback_string()
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2295, in get_buffered_traceback_string
    for step in traceback.format_stack()
EXCEPTION3(ERROR): ProgrammingError: Cannot delete a file [/tracebasedev-archive/archive/archive_files/2024-08/ms_data/exp027f4_free_M01_01_plasma.mzXML] from the os that is associated with existing ArchiveFile database record: {'id': 654, 'filename': 'exp027f4_free_M01_01_plasma.mzXML', 'checksum': 'f9d2074dff00cf659a0888617bc92163725a45e4', 'file_location': <FieldFile: archive_files/2024-08/ms_data/exp027f4_free_M01_01_plasma.mzXML>, 'data_type': 2, 'data_format': 4}.
mzXML file rollback disk archive clean up stats: 0 deleted, 0 failed to be deleted, and 0 expected files did not exist.
  File "/var/www/tracebase/manage.py", line 22, in <module>
    main()
  File "/var/www/tracebase/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 408, in handle_wrapper
    retval = fn(self, *args, **options)
  File "/var/www/tracebase/DataRepo/management/commands/load_msruns.py", line 150, in handle
    self.load_data()
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 448, in load_data
    return self.loader.load_data(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/base/table_loader.py", line 2187, in load_wrapper
    self.aggregated_errors_object.buffer_exception(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2315, in buffer_exception
    buffered_tb_str = self.get_buffered_traceback_string()
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2295, in get_buffered_traceback_string
    for step in traceback.format_stack()
EXCEPTION2(ERROR): MissingRecords: The following records were missing either because they were not in the corresponding sheets or because errors were encountered during their load attempt that prevented their creation (in which case you will see those errors, likely above this one):
1 Sample records were not found (using search field(s): name) with values found in the Peak Annotation Details sheet/file:
	exp027f4_free_M01_01_plasma from row(s): ['no row number supplied']

And here is the final summary exception:
AggregatedErrors Summary (2 errors / 0 warnings):
	EXCEPTION1(ERROR): ProgrammingError: Cannot delete a file [/tracebasedev-archive/archive/archive_files/2024-08/ms_data/exp027f4_free_M01_01_plasma.mzXML] from the os that is associated with existing ArchiveFile database record: {'id': 654, 'filename': 'exp027f4_free_M01_01_plasma.mzXML', 'checksum': 'f9d2074dff00cf659a0888617bc92163725a45e4', 'file_location': <FieldFile: archive_files/2024-08/ms_data/exp027f4_free_M01_01_plasma.mzXML>, 'data_type': 2, 'data_format': 4}.
	EXCEPTION2(ERROR): MissingRecords: The following records were missing either because they were not in the corresponding sheets or because errors were encountered during their load attempt that prevented their creation (in which case you will see those errors, likely above this one):
1 Sample records were not found (using search field(s): name) with values found in the Peak Annotation Details sheet/file:
	exp027f4_free_M01_01_plasma from row(s): ['no row number supplied']

Traceback (most recent call last):
  File "/var/www/tracebase/manage.py", line 22, in <module>
    main()
  File "/var/www/tracebase/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 425, in handle_wrapper
    raise self.saved_aes
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 408, in handle_wrapper
    retval = fn(self, *args, **options)
  File "/var/www/tracebase/DataRepo/management/commands/load_msruns.py", line 150, in handle
    self.load_data()
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 448, in load_data
    return self.loader.load_data(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/base/table_loader.py", line 2271, in load_wrapper
    raise self.aggregated_errors_object
DataRepo.utils.exceptions.AggregatedErrors: 2 exceptions occurred, including type(s): [ProgrammingError, MissingRecords].
AggregatedErrors Summary (2 errors / 0 warnings):
	EXCEPTION1(ERROR): ProgrammingError: Cannot delete a file [/tracebasedev-archive/archive/archive_files/2024-08/ms_data/exp027f4_free_M01_01_plasma.mzXML] from the os that is associated with existing ArchiveFile database record: {'id': 654, 'filename': 'exp027f4_free_M01_01_plasma.mzXML', 'checksum': 'f9d2074dff00cf659a0888617bc92163725a45e4', 'file_location': <FieldFile: archive_files/2024-08/ms_data/exp027f4_free_M01_01_plasma.mzXML>, 'data_type': 2, 'data_format': 4}.
	EXCEPTION2(ERROR): MissingRecords: The following records were missing either because they were not in the corresponding sheets or because errors were encountered during their load attempt that prevented their creation (in which case you will see those errors, likely above this one):
1 Sample records were not found (using search field(s): name) with values found in the Peak Annotation Details sheet/file:
	exp027f4_free_M01_01_plasma from row(s): ['no row number supplied']

Scroll up to see tracebacks for these exceptions printed as they were encountered.

Also note that the following are the stats from the load attempt:
Done.
MSRunSample records created: [0], existed: [0], updated: [0], skipped [1], errored: [0], and warned: [0].
PeakGroup records created: [0], existed: [0], updated: [0], skipped [0], errored: [0], and warned: [0].
ArchiveFile records created: [2], existed: [0], updated: [0], skipped [0], errored: [0], and warned: [0].

So it looks like there was no MSRunSample record created.  That means that the attempt to create it never happened.  It was during the attempt to find an existing record when it was discovered that the Sample record didn't exist, which meant that the MSRunSample record couldn't exist (and couldn't be created).  That's what should be a warning.

...and the 2 ArchiveFile records are the mzXML and the associated RAW file.

So it appears that the problem is that for rec in self.created_mzxml_archive_file_recs: is looping on the archive file records it created, but is complaining that it can't delete them because the records exist, which is circular logic.  What was intended here was to prevent the deletion of pre-existing ArchiveFile records.  Note the comment above the conditional that is checking the record from the loop:
# To be extra safe, we will confirm that the record does not exist in the database before deleting
if exists_in_db(rec):


Fix one more YARPC discrepancy in readme

[noci]

@uber-go/service-framework

Coverage remained the same at 87.704% when pulling b7e77a751002aae49ea1d757fea6e0146d37eeb3 on more-readme into d65759c96ba059e9b285e308c7881b4764116b60 on master.

[no ci] with a space, i believe


How to simulator an ASTM server?

Hi, kepal, Sorry to trouble you. But I am wonder to know how to simulator an ASTM server with your code. I am working with a data reader(a lite vesion LIS ) for a custom's HITCHI 7600 Hitachi 7600 modular chemistry analyzer. Unfortunatly, I can't connect to the machine directly, we are not at the same city. And I wonder if I can silmulator an ASTM server  send data as the machine dose to verify my ASTM client(wriitern with c#).

Hi @HamGuy !
I'm clear remember all the details today, but it should be something like
import logging
from astm.asynclib import loop
from astm.server import Server, BaseRecordsDispatcher

# global astm logging channel configuration
log_root = logging.getLogger('astm')
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter(
    '[%(asctime)s] [%(name)s] [%(levelname)s] %(message)s'
))
log_root.addHandler(handler)
log_root.setLevel(logging.DEBUG)

# special logging configuration for server connection handler
log_conn = logging.getLogger('astm.server.conn')
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter(
    '[%(asctime)s] [%(name)s] [%(levelname)s] [%(host)s:%(port)s] %(message)s'
))
log_conn.addHandler(handler)
log_conn.setLevel(logging.DEBUG)
log_conn.propagate = False

class RecordsDispatcher(BaseRecordsDispatcher):

    log = logging.getLogger('astm.server.dispatcher')

    def on_header(self, record):
        self.log.info('Header: %r', record)

    def on_patient(self, record):
        self.log.info('Patient: %r', record)

    def on_order(self, record):
        self.log.info('Order: %r', record)

    def on_result(self, record):
        self.log.info('Result: %r', record)

    def on_comment(self, record):
        self.log.info('Comment: %r', record)

    def on_unknown(self, record):
        self.log.warn('Unknown record: %r', record)

    def on_terminator(self, record):
        self.log.info('Terminator: %r', record)

if __name__ == '__main__':
    server = Server('localhost', 15200, dispatcher=RecordsDispatcher, timeout=10)
    loop(timeout=1)

Just fill message handler methods with your logic about how to process incoming messages.

Wow... You must be the angel sent by the God.


Intel QuickSync not working

In /System/Library/PrivateFrameworks/AppleGVA.framework/Resources/Info.plist, under System Support, the entry for Mac-7BA5B2DFE22DDD8C does not contain forceOfflineRenderer. On a real Macmini8,1 there is no dGPU, so this is fine but we need offline rendering.
Changing the model to iMac18,2 or iMac18,3 works but causes another issue: the USB-C in DP mode no longer output video.
Another solution is to use WhateverGreen's Shiki with boot-arg shikigva=32 shiki-id=Mac-BE088AF8C5EB4FA2. This works but currently, WG is incompatible with Polaris22Fixup until https://github.com/osy86/Polaris22Fixup/issues/3 is resolved. Additionally, WG has a whitelist system so QuickSync will only work with the apps that are whitelisted, which is not a good solution imo.
We have to either resolve the USB-C DP issues with board-id Mac-BE088AF8C5EB4FA2 or somehow fake the forceOfflineRenderer check in AppleGVA without WG.

So this is the reason why my Ultrafine 5K does not work.

@dongyj1987 Which part? The quicksync or the iMac18,2 spoofing? You can try the WG workaround I talked about, but you have to remove Polaris22fixup.kext first. This will cause the graphics issue to show up but you can confirm if quicksync is the issue.

@dongyj1987 Which part? The quicksync or the iMac18,2 spoofing? You can try the WG workaround I talked about, but you have to remove Polaris22fixup.kext first. This will cause the graphics issue to show up but you can confirm if quicksync is the issue.

The quicksync, cause it's Mac mini spoofing now. I will try WG while removing Polaris22fixup like you said and see what will happen. Thanks a lot!

I can confirm changing smbios to iMac18,3 activating QuickSync (without WG), tested with Safari (youtube) QuickTime Recording, Kodi. basically at the moment for Macmini8,1 there is no any GPU acceleration for Enc/dec. Im just wonder why nothing work through AMD GPU ?
Good for testing use Intel Power Gadget tool, at least for Intel GPU - showing if something going through GPU, iStats will no show any load. also Kodi showing Procesing info if we press O during playback.

Im just wonder why nothing work through AMD GPU ?

Apple's drivers does not support video encoding/other features on many AMD chipsets including Baffin (which we're spoofing). I think they've added some support in 10.15 but I doubt it applies to older chipsets.

It has - apple using Video Toolbox which suppose to work even Buffin at least for H.264 and HEVC
https://developer.apple.com/documentation/videotoolbox
Anyway I will deep more, check what the story, people with RX460 (Polaris Buffin family)and others they got full acceleration. For me target is FXCP.
And solve mistery of VP9

If so then it's probably some config in AppleGVA. Play around with the WG shiki settings I guess.

dos change PEGP to GFX0 and dAGPM.kext necessary to active quicksync?

They are not necessary.

@slimx just enough change SMBIOS to Mac18,3 and you will get. Today I got info to get HW dec/enc HEVC with AMD best to change.
"Only SMBIOS iMacPro1,1 support AMD GVA H264/HEVC encoder, no other options on latest macOS."
for some reason the removed for other options after 10.14.4
@osy86 maybe config of iMacPro1,1 will work with USB-C/DP etc ?

with SMBIOS iMac19,2 im getting both HW acceleration, not everywhere. but safari, iTunes using QSV and FCP AMD ... so for me perfect.

Closing as the original Intel QuickSync issue was resolved. Please open a new issue to discuss AMD HW acceleration issues.

Release 2.0 didn't work at least for Chrome and safari, clean install with WG (default config nothing changed)

@osy86 @piotrasd
hi there,
Quick questions here Quicktime player to encode videos. seems like it's not using quick sync ?
i trying to use handbrake to encode with quick sync, it crash.
This also including video decoding on h264, on quick time player is that known limitation of this build ?
i am using latest clean new install
OS: 10.15.6
HacMini: v2.10


Feature/52

Closes #52

Adds full text search on antibodies with ranking

@aranega can you please check the files that should be in the git repo? Imo at least django static admin should not be in because the collectstatic management command of Django will take care of that
i put the pr to draft so you can continue to work on it


A common interface for Modifier and CSS styles instead of HTML attributes

After working on this project for a while it seems to me that a Compose Modifier is more similar to the CSS styles in the style attribute instead of all the attributes of an HTML element, while HTML attributes are more similar to a Composable's arguments. If this is investigated to be true in most cases, modify the existing ModifierOrAttrs interface into a common interface for Modifier and CSS styles instead of HTML attributes.

This requires furthur consideration as CSS styles can also be set through the class property.


Seanera

adding more information:

guild
boss icons
all attackers in pvp kills
exiva copy/paste
notification color
grammar has been polished


(fix) Static pages should not have NOINDEX unless it is the user-not-…

…authorised page
Trello card URL:
https://trello.com/c/JfT23ZhJ/579-fix-google-search-coverage-error-for-static-pages-that-are-included-in-the-sitemap
Changes in this PR:
Remove the NOINDEX header from static pages so that Google will index them.
However, I have left NOINDEX on the user-not-authorised page as I don't think we want Google to crawl that (@tahb can you confirm?)

@lozette agreed, we don't want Google directing any users to the not authorised page as it is effectively an error page.


Disable lowering of shape cast operation by default in Codegen Driver

Shape cast operations are usually no-ops that canonicalize away at some
point in the pipeline. However, if they are not, the codegen driver lowers
them to an inefficient sequence of insert/extract ops. This lowering is
enabled at stage 5 and later ones by default. Unfortunately, the AVX2 lowering
for transpose operations (stage 6) generates shape cast operations that
could be canonicalized away but they are immediately lowered to the insert/extract
sequence before canonicalization can happen (the shape cast lowering patterns are
also loaded at stage 6).
This patch introduces a flag to enable/disable the lowering of shape
cast ops. I implemented this mostly to provide context but I think this
is not a good solution. I think we should move the shape cast stage after
the transpose one. Do you see any problem with that?

Shape cast operations are usually no-ops that canonicalize away at some point in the pipeline. However, if they are not, the codegen driver lowers them to an inefficient sequence of insert/extract ops. This lowering is enabled at stage 5 and later ones by default. Unfortunately, the AVX2 lowering for transpose operations (stage 6) generates shape cast operations that could be canonicalized away but they are immediately lowered to the insert/extract sequence before canonicalization can happen (the shape cast lowering patterns are also loaded at stage 6).
This patch introduces a flag to enable/disable the lowering of shape cast ops. I implemented this mostly to provide context but I think this is not a good solution. I think we should move the shape cast stage after the transpose one. Do you see any problem with that?

This seems like it is side stepping the problem. Issue here is that you want one canonicalization to kick in before others. Maybe we can just order them explicitly (i.e. different calls to applyPatternsAndFoldGreedily, one that does the shape cast canonicalization before it gets lowered to something else.
(And of course all this would be easier with transformation dialect).

Marking as requesting change cause it isnt clear to me what decides when to lower the shape case and when not to (or what would happen if we didnt have the fallback lowering of shape cast)

If a shape cast is not folded away, the lowering will fail because the op will become illegal when lowering to LLVM. However, according to the shape cast doc, that should never happen: It is currently assumed that this operation does not require moving data, and that it will be folded away before lowering vector operations. I'm not sure if that is up to date because, why do we need fallback lowering then? :). As I said, the current implementation is mostly to provide context.

This seems like it is side stepping the problem. Issue here is that you want one canonicalization to kick in before others. Maybe we can just order them explicitly (i.e. different calls to applyPatternsAndFoldGreedily, one that does the shape cast canonicalization before it gets lowered to something else.

The canonicalization needed happens between stages. The problem is that each stage includes all the patterns of the previous stages. We would have to special case stage 7 to do what you described. Actually, to some extent, what you described is almost equivalent to interchanging stage 6 (shape cast) and stage 7 (transpose lowering), since canonicalization would run between the two and everything would work even if a shape cast is not folded away. However, I don't if that change would have other side effects. I have the impression that having shape casts around shouldn't interfere with the transpose lowering but maybe I'm missing something. Hopefully, @nicolasvasilache can clarify.
To provide a bit more context, the transpose lowering was not generating any shape cast ops until a few weeks ago, when I extended the AVX2 patterns so this issue is relatively new and was not taken into account when the order of the stages was decided.

The various stages ordering has been determined empirically but without a deep study, I think this is all fine to reshuffle if the data supports it. Still, all these issues will soon go away as we move away from hardcoded C++ ordering.

I swapped the stages and the problem is fixed locally. I'll add the test in a follow-up PR. I haven't seen any test crashing on the IREE side after the change.

all these issues will soon go away as we move away from hardcoded C++ ordering.

Why they will go away when we migrate to transform dialect? I thought it's a scheduling dialect, and we'll have the same order no matter using which approach?
I'm fine with this if it brings performances to our tracking models. My concern is that we're getting diverge from Sandbox. If we're going to update the Sandbox side, that would be great. This change may need some data support.

To provide a bit more context, you can take a look at the large sequence of insert/extract ops generated before and after the shuffle sequence in this transpose test: https://github.com/google/iree/pull/8750/files#diff-ba78b5a877da6ff11946d24a7a189104482ebc5a1da644817d68fb55e6c68fabR117. Those insert/extract ops are the result of lowering shape cast ops. They also happen for all the tests below that one but I didn't add the CHECKs hoping to get this change in.


Using tqdm in the cpu train setup

If --trainer torchCPUTrainer,
https://github.com/dice-group/dice-embeddings/blob/0849d2c46ea4bf06d07db6f3d0d2ad01fc230af5/dicee/scripts/run.py#L56
then TorchTrainer is initialized to train an embedding model.
https://github.com/dice-group/dice-embeddings/blob/0849d2c46ea4bf06d07db6f3d0d2ad01fc230af5/dicee/trainer/torch_trainer.py#L9
Yet, TorchTrainer does not use a progress bar. We should integrate https://github.com/tqdm/tqdm into the TorchTrainer that is being initialized. To this end, we should focus on this particular for loop (guess)
https://github.com/dice-group/dice-embeddings/blob/0849d2c46ea4bf06d07db6f3d0d2ad01fc230af5/dicee/trainer/torch_trainer.py#L75

We need a description tag for the progress bar : Is "Training progress" a suitable description for the progress bar?

Please train a model with PL trainer, to see the displayed info in the progress bar. Ideally, we should use the same description tag. However, it is not a must

As per the PL trainer, they do not seem to be using a description tag. The way progress bar will be shown for the two trainers is also different, PL trainer shows a single persistent progress bar whereas TorchCPUTrainer shows the logs for each epoch and batch separately.

Alrighty. Perhaps for the time being, we can settle for two different TQDM progress bars for different trainers

No worries. Could you please create a feature branch from dev branch and push the changes? So that I can take a look at it :)

Can you check at https://github.com/dice-group/dice-embeddings/tree/tqdm-support . For now I have just added the progress bar to the loop running all the batch and epochs.

Completed with https://github.com/dice-group/dice-embeddings/pull/258


Getting Unable to unserialize value error

Hello,
We have installed this module on Magento 2.2.6 and it's working fine. But the problem is it's conflicting other module functionality.
I'm getting Unable to unserialize value error. Please see the below screenshot:

Please have a look at the screenshot.

I think this was fixed


patches: mainline: Drop this_cpu_has_cap patch

It has been merged:
https://git.kernel.org/torvalds/c/c55a04176cba5a2de085461438b65e1c46541ef3

ping @nickdesaulniers (I assume your GitHub notifications are not doing you any favors :P)

yes, sorry, this is one that I didn't see a notification in the github UI about. Looking now.


feat: add banner in institution portal

Ticket
https://agile-jira.pearson.com/browse/PADV-1711
Description
Add Maintenance banner
Changes made
Update topaz-library
Configure maintenance banner
Screenshots

How to test
Clone the Institution Portal MFE
Run npm install.
Run npm run start
Go to http://localhost:1980/institution-portal

@AuraAlba could you mention how to configure MAINTENANCE_BANNER_TEXT in the PR description?


Customizable string formatting through format()

Specification syntax unknown at the moment. Planned fields are, roughly:

Maximum number of rows
Maximum number of columns
Cell width
Cell alignment (left, right, center) (dropped)

Possible addition for version 0.2.2.

Forwarded to post version 0.2.2.

Planned specification syntax:
[nrows_max[,ncols_max[,x_max]]

In Python, this would look something like:
a: Matrix
f"{a:8,8,8}"   # Planned default specification
f"{a:,,}"      # Any argument can be dropped - will default to 8
f"{a:12,,7}"   # Equivalent to :12,8,7
f"{a:0,-1,0}"  # Arguments can be <= 0, in which case, all items will be written to the string

When the x_max argument is <= 0, it will fallback to the default of 8, since items types will have an indeterminable maximum string length without a second pass through the relevant items (which would cause significant slow-down).
Cell alignment dropped as a part of the design draft - potentially a future addition.

This issue will remain open, for now, as future versions may expand upon this idea. I think cell alignment is one potential candidate, and perhaps a field for head vs. tail, similar to DataFrame.head() and DataFrame.tail() of the pandas library.
New formatting syntax could be something like:
format_spec ::= [alignment][from][nrows_max[,ncols_max[,chars_max]]]
alignment   ::= "<" | "^" | ">"
from        ::= "h" | "t"
nrows_max   ::= integer
ncols_max   ::= integer
chars_max   ::= integer

Default specification would become:
f"{a:>h8,8,8}"

And with these arguments being optional, this remains backwards compatible.


direct parsing fuzzing test

Two issues were found by the new fuzzing target: https://github.com/boostorg/json/issues/1047 and https://github.com/boostorg/json/issues/1048
I think detecting memory issues in case of new changes or refactoring might be useful.
I'm not sure about code formatting for the project, I will appreciate your feedback.

While the idea is interesting, I am sceptical fuzzing with random strings can be effective when the parser expects a very specific structure of the input. It will probably always fail on the second (on_object_begin), the third (on_key), or the fourth (on_bool) event.

According to coverage reports, it works well enough to find valid inputs, these two issues were found quite quickly. We can provide a dictionary for the fuzzer to help it a bit if we want to improve the performance.

It may be a coincidence. This pull request mentions "detecting memory issues". The drone agent on IBM s390 crashed with out-of-memory errors.  I have rebooted the server.

It's also possible that linking the new fuzzer requires too much memory (variant parsers do do that).

I am sceptical fuzzing with random strings can be effective when the parser expects a very specific structure of the input. It will probably always fail on the second (on_object_begin), the third (on_key), or the fourth (on_bool) event.

Just for an experiment: I reverted the fix for https://github.com/boostorg/json/issues/1047 and measured the time required to find that bug. Without a dictionary and with an empty corpus (the worst scenario) it takes about 10 seconds. The reason is the fuzzer doesn't have to find a valid input with all required fields. It's enough to generate a JSON with some fields to verify parser for a specific type only.
Adding corpus doesn't improve speed much. But I've added two valid JSONs to seed files, and it gives the fuzzer the ability to reach the "success" point. With this change, the fuzzer finds https://github.com/boostorg/json/issues/1047 in less than one second. Also, I changed C++ version to C++14 to avoid potential OOM.
Please let me know if it makes sense

Can you please squsah and rebase on the current develop (if it's not already). I can also do it manually myself, but then the PR won't be considered merged by GitHub.

Done

Thank you for your contribution.


Compile bug?

Bug report
I literally can't compile any build of psych engine for some reason?

Did you edit anything? If so, mention or summarize your changes.
nope, didn't change anything. Vanilla psych engine build

These aren't errors, these are Warnings, they don't do anything at all other than tell you that the functions i've used are outdated, and wouldn't stop you from compiling.


Content-Length

Hello,
i am wondering, parser extracts Content-Length correct (at POST Method) and stores it.
In Header Complete callback Content-Length is still correct.
In Body-Data-Callback (which should store body data?) Content-Length is 0, (cleared).
WHY?
Or is it required to indicate EOF by calling http_parser_execute  with len=0?

@SpenZerX this field is used for internal bookkeeping. We decrement it when parsing new data to determine where the next http request/response will start.
Thanks for asking!


Make cache editing more robust against corner cases

The cache editor UI exhibits erroneous behavior in several corner cases:

If the value of the a cache variable occurs somewhere in its name (like ON at the end of  SOME_OPTI*ON*), that part of the name is replaced with the new value (resulting in SOME_OPTIFALSE in the previous example).
In case a variable name ends with the name of another cache variable (e.g. ABC_SOME_OPTION and SOME_OPTION), the editor may modify the prefixed entry when the user changed the unprefixed variable (depending on order of variables within CMakeCache.txt).
The extension can read cache variables names stored in quoted form in CMakeCache.txt (necessary when the variable name contains a colon), but it cannot change them via the editor UI, because it always looks for the unquoted form in CMakeCache.txt.
The cache editor UI displays ...-STRINGS entries as editable cache variables, but these are just properties of other variables and therefore should be excluded.

Furthermore, there is no proper escaping of variable names and values when building the HTML markup for the editor webview, which can mess up the GUI.
This PR attempts to fix all of the above issues and also uses the -STRINGS properties to add dropdowns to the editor UI. If you prefer, I can factor out this part into another PR.
I'm not completely happy with it anyway, because with the currently used <datalist> approach, the dropdown list is filtered by the text in the input field and thus all choices except for the currently selected value are usually hidden. To see the full list of choices, one has to clear the text input first. Unfortunately, I was unable to find a way to realize a plain old combo box with HTML/CSS, but was also reluctant to pull in a 3rd party Javascript library for that seemingly simple task. If you have any suggestions, I'd be happy to try it.

@aemseemann, thank you for this contribution. We will review and test once we are done with the CMake Tools 1.7.2 patch release. Very good findings, thank you.

@aemseemann, do you think this PR could address the symptoms described in https://github.com/microsoft/vscode-cmake-tools/issues/1591?

@aemseemann, do you think this PR could address the symptoms described in #1591?

Cannot rule it out, but it is quite unlikely. I'll comment over there.

@aemseemann, I reviewed and tested these scenarios and I am approving this PR. I committed some linter fixes in your branch. I will also merge it as soon as the tests complete.
I want to let you know that for the third bullet with quotes, I did see how before the PR the cache editor was not able to update a quoted variable and after the PR it does correctly. But even with the PR, once that cache variable is saved, it loses its quotes. I can investigate and make an additional fix. Let me know if you don't think this is a concern or if you'd like to follow up with an additional small fix. I will merge this PR without that to avoid more integration work and since all works fine.

@aemseemann, you need to sign the Contributor License Agreement in order for us to merge this PR.

Thanks for reviewing the changes and sorry for abandoning this PR for several weeks. I didn't find the time to familiarize myself with the CLA and the general concept behind it.
Regarding the quoting issue: The behavior should be fine as it is now. I guess you tried it with a variable name that does not require quoting. In this case, cmake removes the quotes the next time it reconfigures the project (which is triggered by the cache editor). If you try it with a variable name containing a literal colon (:) or starting with //, cmake will keep the entry quoted (see also https://gitlab.kitware.com/cmake/cmake/-/blob/master/Source/cmCacheManager.cxx#L374)