Update to v1.0.0

Allows functionality on 1.1.0 as well.

it seems to multiply the hack, weaken and grow times by 1000, in my game i removed the * 1000 multiplier

it seems to multiply the hack, weaken and grow times by 1000, in my game i removed the * 1000 multiplier

According to https://bitburner.readthedocs.io/en/latest/v1.0.0_migration.html getHack(-Grow/-Weaken)Time "will return milliseconds instead of seconds."

Is this supposed to take hours?
mainHack.ns: [08:42:51] Starting mainHack.ns
mainHack.ns: [08:42:51] Selected phantasy for a target. Planning to grow the server. Will wake up around 11:33:53
mainHack.ns: [08:42:51] Stock values: baseSecurity: 20; minSecurity: 7; maxMoney: $600,000,000
mainHack.ns: [08:42:51] Current values: security: 7; money: $24,000,000
mainHack.ns: [08:42:51] Time to: hack: 18:42:45; grow: 11:52:48; weaken: 02:51:01
mainHack.ns: [08:42:51] Delays: 08:07:45 for hacks, 14:57:57 for grows
mainHack.ns: [08:42:51] Cycles ratio: 259 grow cycles; 23 weaken cycles`

Is this supposed to take hours?
mainHack.ns: [08:42:51] Starting mainHack.ns mainHack.ns: [08:42:51] Selected phantasy for a target. Planning to grow the server. Will wake up around 11:33:53 mainHack.ns: [08:42:51] Stock values: baseSecurity: 20; minSecurity: 7; maxMoney: $600,000,000 mainHack.ns: [08:42:51] Current values: security: 7; money: $24,000,000 mainHack.ns: [08:42:51] Time to: hack: 18:42:45; grow: 11:52:48; weaken: 02:51:01 mainHack.ns: [08:42:51] Delays: 08:07:45 for hacks, 14:57:57 for grows mainHack.ns: [08:42:51] Cycles ratio: 259 grow cycles; 23 weaken cycles

Note: I am still a novice, but I will share what I have found.
if you edit mainHack.ns around line 180 you will find:
const hackTime = ns.getHackTime(bestTarget) * 1000
const growTime = ns.getGrowTime(bestTarget) * 1000
const weakenTime = ns.getWeakenTime(bestTarget) * 1000
rreplace the "* 1000" with "* 1" or just remove that section in each of those lines. This is what lapryx was referencing to I believe. Even if it isn't I have had the script behave much better after that change.

Note: I am still a novice, but I will share what I have found. if you edit mainHack.ns around line 180 you will find: const hackTime = ns.getHackTime(bestTarget) * 1000 const growTime = ns.getGrowTime(bestTarget) * 1000 const weakenTime = ns.getWeakenTime(bestTarget) * 1000
rreplace the "* 1000" with "* 1" or just remove that section in each of those lines. This is what lapryx was referencing to I believe. Even if it isn't I have had the script behave much better after that change.

also in
const growDelay = Math.max(0, weakenTime - growTime - 15 * 1000)
const hackDelay = Math.max(0, growTime + growDelay - hackTime - 15 * 1000)
a few lines below.

Is this supposed to take hours?
mainHack.ns: [08:42:51] Starting mainHack.ns mainHack.ns: [08:42:51] Selected phantasy for a target. Planning to grow the server. Will wake up around 11:33:53 mainHack.ns: [08:42:51] Stock values: baseSecurity: 20; minSecurity: 7; maxMoney: $600,000,000 mainHack.ns: [08:42:51] Current values: security: 7; money: $24,000,000 mainHack.ns: [08:42:51] Time to: hack: 18:42:45; grow: 11:52:48; weaken: 02:51:01 mainHack.ns: [08:42:51] Delays: 08:07:45 for hacks, 14:57:57 for grows mainHack.ns: [08:42:51] Cycles ratio: 259 grow cycles; 23 weaken cycles

Note: I am still a novice, but I will share what I have found. if you edit mainHack.ns around line 180 you will find: const hackTime = ns.getHackTime(bestTarget) * 1000 const growTime = ns.getGrowTime(bestTarget) * 1000 const weakenTime = ns.getWeakenTime(bestTarget) * 1000
rreplace the "* 1000" with "* 1" or just remove that section in each of those lines. This is what lapryx was referencing to I believe. Even if it isn't I have had the script behave much better after that change.

Thanks. I was trying to remove all the 1000 and see if it stills works xD
After removing the * 1000 it seems it behaves better
mainHack.ns: [08:54:35] Selected phantasy for a target. Planning to grow the server. Will wake up around 08:59:04
mainHack.ns: [08:54:35] Stock values: baseSecurity: 20; minSecurity: 7; maxMoney: $600,000,000
mainHack.ns: [08:54:35] Current values: security: 7; money: $24,000,000
mainHack.ns: [08:54:35] Time to: hack: 00:01:07; grow: 00:03:35; weaken: 00:04:29
mainHack.ns: [08:54:35] Delays: 00:03:22 for hacks, 00:00:53 for grows
mainHack.ns: [08:54:35] Cycles ratio: 259 grow cycles; 23 weaken cycles

Just wanted to add my view:
After noticing the same errors as the few messages above me mentioned, I tried to fix them.
After applying patch-1 to mainhack.ns the sleep timer after each cycle was still wrong. So after additionally removing the " * 1000" multipliers from hackTime, growTime, and so on, the timers for each cycle seem to work just fine.
Been running the script now for a few hours. So far no error.

I fixed stockMarketer4S.ns. It requires bitnode 8 to run. does anyone have a stock script for  starters?
`let stockSymbols
let corpus
let underperformingShares
const commission = 100000
function localeHHMMSS(ms = 0) {
if (!ms) {
ms = new Date().getTime()
}
return new Date(ms).toLocaleTimeString()
}
function getMoney(ns) {
return ns.getServerMoneyAvailable('home') - 5 * commission
}
function sellShorts(ns, stockSymbol) {
const stockInfo = getStockInfo(ns, stockSymbol)
const shortSellValue = ns.stock.sellShort(stockSymbol, stockInfo.sharesShort)
if (shortSellValue) {
corpus += stockInfo.sharesShort * (stockInfo.avgPriceShort - shortSellValue) - 2 * commission
ns.print(
[${localeHHMMSS()}][${stockSymbol}] Sold ${stockInfo.sharesShort} shorts for ${ns.nFormat(shortSellValue, '$0.000a')}. Profit: ${ns.nFormat(         stockInfo.sharesLong * (stockInfo.avgPriceShort - shortSellValue) - 2 * commission,         '$0.000a'       )}
)
}
}
function sellLongs(ns, stockSymbol) {
const stockInfo = getStockInfo(ns, stockSymbol)
const longSellValue = ns.stock.sell(stockSymbol, stockInfo.sharesLong)
if (longSellValue) {
corpus += stockInfo.sharesLong * (longSellValue - stockInfo.avgPriceLong) - 2 * commission
ns.print(
[${localeHHMMSS()}][${stockSymbol}] Sold ${stockInfo.sharesLong} longs for ${ns.nFormat(longSellValue, '$0.000a')}. Profit: ${ns.nFormat(         stockInfo.sharesLong * (longSellValue - stockInfo.avgPriceLong) - 2 * commission,         '$0.000a'       )}
)
}
}
// Only if not going to lose money
function sellUnderperforming(ns, stockSymbol) {
const stockInfo = getStockInfo(ns, stockSymbol)
if (stockInfo.sharesShort && stockInfo.sharesShort * (stockInfo.avgPriceShort - stockInfo.stockAskPrice) > 2 * commission) {
sellShorts(ns, stockSymbol)
}
if (stockInfo.sharesLong && stockInfo.sharesLong * (stockInfo.stockBidPrice - stockInfo.avgPriceLong) > 2 * commission) {
sellLongs(ns, stockSymbol)
}
}
function sellWrongPosition(ns, stockSymbol) {
const stockInfo = getStockInfo(ns, stockSymbol)
// Sell shorts if going up
if (stockInfo.position === 'Long' && stockInfo.sharesShort) {
sellShorts(ns, stockSymbol)
}
// Sell longs if going down
if (stockInfo.position === 'Short' && stockInfo.sharesLong) {
sellLongs(ns, stockSymbol)
}
}
function buyNewShares(ns, stockSymbol) {
const stockInfo = getStockInfo(ns, stockSymbol)
const minimumMoneyToInvest = 10 * commission
if (!stockInfo.haveMaxShares && getMoney(ns) > minimumMoneyToInvest) {
let maxSharesToBuy
let sharesToBuy
let buyValue
let shareType
if (stockInfo.position === 'Long') {
  maxSharesToBuy = stockInfo.maxShares - stockInfo.sharesLong
  sharesToBuy = Math.max(0, Math.min(maxSharesToBuy, Math.floor(getMoney(ns) / stockInfo.stockAskPrice)))
  if (sharesToBuy) {
    buyValue = ns.buyStock(stockSymbol, sharesToBuy)
  }
  shareType = 'longs'
} else {
  maxSharesToBuy = stockInfo.maxShares - stockInfo.sharesShort
  sharesToBuy = Math.max(0, Math.min(maxSharesToBuy, Math.floor(getMoney(ns) / stockInfo.stockBidPrice)))
  if (sharesToBuy) {
    buyValue =ns.stock.short(stockSymbol, sharesToBuy)
  }
  shareType = 'shorts'
}

if (sharesToBuy) {
  const invested = ns.nFormat(buyValue * sharesToBuy, '$0.000a')
  ns.print(`[${localeHHMMSS()}][${stockSymbol}] Bought ${sharesToBuy} ${shareType} for ${ns.nFormat(buyValue, '$0.000a')} each. Invested: ${invested}`)
}

}
}
function getStockInfo(ns, stockSymbol) {
const [sharesLong, avgPriceLong, sharesShort, avgPriceShort] = ns.stock.getPosition(stockSymbol);
const volatility = ns.stock.getVolatility(stockSymbol)
const probability = ns.stock.getForecast(stockSymbol) - 0.5
const expectedReturn = Math.abs(volatility * probability)
const maxShares = ns.stock.getMaxShares(stockSymbol)
const haveAnyShares = sharesLong + sharesShort > 0
const haveMaxShares = sharesLong + sharesShort === maxShares
const stockAskPrice = ns.stock.getAskPrice(stockSymbol)
const stockBidPrice = ns.stock.getBidPrice(stockSymbol)
const position = probability >= 0 ? 'Long' : 'Short'
return {
stockSymbol,
maxShares,
haveAnyShares,
haveMaxShares,
sharesLong,
avgPriceLong,
stockAskPrice,
sharesShort,
avgPriceShort,
stockBidPrice,
volatility,
probability,
expectedReturn,
position,
}
}
export async function main(ns) {
ns.disableLog('ALL')
let tickCounter = 1
stockSymbols = ns.stock.getSymbols();
corpus = ns.getServerMoneyAvailable('home') - 1000000
stockSymbols.forEach((stockSymbol) => {
const stockInfo = getStockInfo(ns, stockSymbol)
corpus += stockInfo.sharesLong * stockInfo.avgPriceLong + stockInfo.sharesShort * stockInfo.avgPriceShort

})
const startingCorpus = corpus
while (true) {
ns.clearLog()
ns.print([${localeHHMMSS()}] Tick counter: ${tickCounter}, corpus: ${ns.nFormat(corpus, '$0.000a')})
ns.print([${localeHHMMSS()}] Starting corpus: ${ns.nFormat(startingCorpus, '$0.000a')})
stockSymbols.sort((a, b) => {
  const stockA = getStockInfo(ns, a)
  const stockB = getStockInfo(ns, b)

  if (stockB.expectedReturn === stockA.expectedReturn) {
    return Math.abs(stockB.probability) - Math.abs(stockA.probability)
  }

  return stockB.expectedReturn - stockA.expectedReturn
})

stockSymbols
  .filter((stockSymbol) => getStockInfo(ns, stockSymbol).haveAnyShares)
  .filter((stockSymbol, index) => stockSymbol !== stockSymbols[index])
  .forEach((stockSymbol) => sellUnderperforming(ns, stockSymbol))
await ns.sleep(5)

stockSymbols.forEach((stockSymbol) => sellWrongPosition(ns, stockSymbol))
await ns.sleep(5)

stockSymbols.forEach((stockSymbol) => buyNewShares(ns, stockSymbol))
await ns.sleep(5)

ns.print(`[${localeHHMMSS()}] After transactions: corpus: ${ns.nFormat(corpus, '$0.000a')}`)
await ns.sleep(3500)
tickCounter++

}
}`

stockmaster4.ns is a Tix only based script as far as i know. It is not working in my game on steam tho as it doesn't recognize the TIX api functions.

stockmaster4.ns is a Tix only based script as far as i know. It is not working in my game on steam tho as it doesn't recognize the TIX api functions.

Yea. TIX is not recognized. I used ns.stock

I fixed stockMarketer4S.ns. It requires bitnode 8 to run. does anyone have a stock script for starters?

Someone wrote an adequate one here: https://www.reddit.com/r/Bitburner/comments/qvqwkw/help_with_stock_market_script/

stockmaster4.ns is a Tix only based script as far as i know. It is not working in my game on steam tho as it doesn't recognize the TIX api functions.

Yea. TIX is not recognized. I used ns.stock. But ns.stock.short need bitnode 8

it seems that the stock keyword has been moved from say ns.buyStock() to ns.stock.buy(), this is pretty easy to replace across a file. i modified a file of off reddit to work with the new naming scheme:
`//Requires access to the TIX API and the 4S Mkt Data API
let fracL = 0.1;     //Fraction of assets to keep as cash in hand
let fracH = 0.2;
let commission = 100000; //Buy or sell commission
let numCycles = 2;   //Each cycle is 5 seconds
function refresh(ns, stocks, myStocks){
let corpus = ns.getServerMoneyAvailable("home");
myStocks.length = 0;
for(let i = 0; i < stocks.length; i++){
let sym = stocks[i].sym;
stocks[i].price = ns.stock.getPrice(sym);
stocks[i].shares  = ns.stock.getPosition(sym)[0];
stocks[i].buyPrice = ns.stock.getPosition(sym)[1];
stocks[i].vol = ns.stock.getVolatility(sym);
stocks[i].prob = 2* (ns.stock.getForecast(sym) - 0.5);
stocks[i].expRet = stocks[i].vol * stocks[i].prob / 2;
corpus += stocks[i].price * stocks[i].shares;
if(stocks[i].shares > 0) myStocks.push(stocks[i]);
}
stocks.sort(function(a, b){return b.expRet - a.expRet});
return corpus;
}
function buy(ns, stock, numShares){
ns.stock.buy(stock.sym, numShares);
ns.print(Bought ${stock.sym} for ${format(numShares * stock.price)});
}
function sell(ns, stock, numShares){
let profit = numShares * (stock.price - stock.buyPrice) - 2 * commission;
ns.print(Sold ${stock.sym} for profit of ${format(profit)});
ns.stock.sell(stock.sym, numShares);
}
function format(num){
let symbols = ["","K","M","B","T","Qa","Qi","Sx","Sp","Oc"];
let i = 0;
for(; (num >= 1000) && (i < symbols.length); i++) num /= 1000;
return ( (Math.sign(num) < 0)?"-$":"$") + num.toFixed(3) + symbols[i];

}
export async function main(ns) {
//Initialise
ns.disableLog("ALL");
let stocks = [];
let myStocks = [];
let corpus = 0;
for(let i = 0; i < ns.stock.getSymbols().length; i++)
stocks.push({sym:ns.stock.getSymbols()[i]});
while(true){
    corpus = refresh(ns, stocks, myStocks);

    //Sell underperforming shares
    for (let i = 0; i < myStocks.length; i++){
        if(stocks[0].expRet > myStocks[i].expRet){
            sell(ns, myStocks[i], myStocks[i].shares);
            corpus -= commission;
        }
    }
    //Sell shares if not enough cash in hand
    for (let i = 0; i < myStocks.length; i++){
        if( ns.getServerMoneyAvailable("home") < (fracL * corpus)){
            let cashNeeded = (corpus * fracH - ns.getServerMoneyAvailable("home") + commission);
            let numShares = Math.floor(cashNeeded/myStocks[i].price);
            sell(ns, myStocks[i], numShares);
            corpus -= commission;
        }
    }

    //Buy shares with cash remaining in hand
    let cashToSpend = ns.getServerMoneyAvailable("home") - (fracH * corpus);
    let numShares = Math.floor((cashToSpend - commission)/stocks[0].price);
    if ((numShares * stocks[0].expRet * stocks[0].price * numCycles) > commission)
        buy(ns, stocks[0], numShares);

    await ns.sleep(5 * 1000 * numCycles + 200);
}

}`

Should there be *1000 at the top in line 9 and 10? I removed all 5 around line 180

yes, the sleeptime is based in milliseconds aswell as the getXtime functions return the time in milliseconds. so this makes sure that the max weaken time is 30 min and the map is refreshed every 24 hours.

also in const growDelay = Math.max(0, weakenTime - growTime - 15 * 1000) const hackDelay = Math.max(0, growTime + growDelay - hackTime - 15 * 1000) a few lines below.

This isn't quite correct. These two lines should keep the * 1000, as it simply multiplies the 15 seconds to 15000 milliseconds.
Unlike the "getXTime" functions, this is just a static equation and didn't get changed by the game devs.
It's to note, that both removing it and not removing it works, however, if you remove it you lose a tiny bit of efficiency. (15 seconds extra on each delay)
In summary, if you have an issue with your wake up times being astronomically high:

remove the " * 1000" on line 179.
remove the " * 1000" on line 180.
remove the " * 1000" on line 181.

and you should be good to go.


[docs] Add emotion CSS theming example without styled components

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Summary 💡
In the docs it only gives an example how to use the Theme with the styled components flavor of emotion.
Examples 🌈
This codebox features both ways: the current MUI documented and my proposal using the useTheme hook: https://codesandbox.io/s/emotioncss-material-demo-forked-tq2pmk?file=/demo.js
import Slider from "@mui/material/Slider";
import Box from "@mui/material/Box";
import { css } from "@emotion/react";
import { useTheme } from "@mui/material/styles";

export default function EmotionCSS() {
  const theme = useTheme();
  const emotion = css`
    color: ${theme.palette.secondary.main};

    :hover {
      color: ${theme.palette.secondary.main};
    }
  `;

  return (
    <Box sx={{ width: 300 }}>
      <Slider defaultValue={30} />
      <CustomizedSlider defaultValue={30} />
      <Slider defaultValue={30} css={emotion} />
    </Box>
  );
}

Motivation 🔦
I personally like the pure CSS better than the styled components way, so do you think it would make sense to feature a way to use the css={css` my css here `} way together with the theme?
If you think it's a good idea I'm happy to provide a PR short-term.

We have an example for it under the Style library interoperability guide. We don't want to use it in all other demos, as we prefer the styled() and sx prop which we believe bring better developer experience.

Yeah that was the page that I meant it doesn't look obvious for me that this is also an option. But if you are anyways more preferring the other ways then this might be obsolete.

@Primajin anything that you think we can improve on this? otherwise, we can close this issue.

As far as I understood this is not the preferred way and doesn't necessarily need documentation thus you could close this. Otherwise I am still happy to provide a quick MR to feature this in the docs as an alternative way of writing "CSSier CSS Strings" as opposed to "JSSier CSS Objects" with sx 😄🤷🏻


How to use the component tabs

Issue Type
question
Description
How do I use the Tabs with React component?
And how do I display different data on each tab? Ex:
Tabs
    Tab1: Content1
    Tab2: Content2
    Tab3: Content3
Thanks,
Homero

And the content inside the tab?
Thanks


Memory Overhead is Too High (10x or more)

Context:
I was parsing a 700MB json file, in Visual Studio 2019. The process took almost 9GB of memory! This at first looks like a 10x overhead.
We all know that std::strings have on the order of 2x overhead, so this doesn't seem to explain it.
All std::TreeNode<basic_string, basic_json> are showing up at taking exactly 140 bytes, which seems excessive. Since they are all the same size, it seems likely the keys are all being allocated much larger than necessary, perhaps. Keys in maps tend to be much shorter than the data that is being mapped to.
Possible solution is to move away from std::map ?
For reference, allocating strings looks like it's about 50% of memory, and treenodes are about 25% of memory in my instance.

We all know that std::strings have on the order of 2x overhead, so this doesn't seem to explain it.

This is highly dependent upon string length.  If all your strings are 1 character, then overhead is typically 20-30x.  If your strings are 1000 characters, then overhead is more like 2%.  Most std::string implementations have an allocation-avoidance path below a certain length (depending upon the size of the std::string itself, which is usually about 24-32 bytes on 64-bit systems), whereby they store the string in the std::string structure.  So the optimal string length is around 20-24 bytes.

All std::TreeNode<basic_string, basic_json> are showing up at taking exactly 140 bytes, which seems excessive

Yes, that does seem like too much.  Worth investigating.  My experience with std::map is that its overhead per entry is roughly 3 pointers worth, plus the memory allocator overhead and padding.  On 64-bit systems, that is again 24-32 bytes.  That plus the string for the key, means the nodes should be roughly 64 bytes plus the size of the basic_json.  Is the basic_json around 70-80 bytes?  That seems like more than it should be as, IIRC, it is a union of the possible types -- string, map, vector being the large ones.  Plus a few bits for the type information.  Best case scenario (on 64 bit systems) is going to be roughly 96 bytes though, so only about 40-45% smaller.
Doing better than the STL containers is actually quite a challenge unless you are willing to sacrifice generality and functionality fairly substantially.  On 64-bit systems some cunning memory management could let you use smaller offsets in place of pointers, but this is a lot of effort (and bugs) for what is likely going to be no more than a 2-3x improvement.  The win on 32-bit systems will be less.  That's a pretty steep maintenance and generality cost, for relatively modest gains.  Doesn't mean its not worth doing, but not to be undertaken lightly.
For large piles of json data you really should consider whether a DOM representation is even the right choice.  It may simply be better to parse it SAX style and keep it around in its text form to reparse as-needed.  Or keep pointers/offsets into the image and partially re-parse when needed.

For what it's worth, I think it's a universal truth that keys into the JSON will be smaller than 100 characters. The data could be very long, but optimizing the storage of the keys in the key-value pairs could be a fairly substantial gain across all use-cases.
In terms of the size of basic_json, the documentation claims:

Each JSON object has an overhead of one pointer (the maximal size of a union) and one enumeration element (1 byte).

I can confirm that it looks like the storage for the single json value is indeed just a pointer, defined here: https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp#L13671
That should be only 9 total bytes for basic_json (plus all the memory that it points to, of course). It's possible that Visual Studio is including the de-referenced string in the allocation calculation for the TreeNode, but that wouldn't explain why the value was a constant 140, instead of variable (I have some string data that are 1kb, though most are small).

BTW, how are you measuring the size of each TreeNode?

I'm using the Memory Profiler in Visual Studio, and running through the allocation output. That gives me the function call stack, and the size of the allocated object.

Relevant: https://github.com/nlohmann/json/issues/1516
Have you tried building in Release mode? That sometimes can make a huge difference.

I would expect Release mode to improve speed, but not memory. At least not by 10x memory. That said, I switched over to RapidJson, so I don't have the code to run this again at this moment. The file I was using was this large file, if you'd like to reproduce the issue for analysis.
https://archive.scryfall.com/json/scryfall-all-cards.json

Just as a reference, the memory consumption was ~2.79GB for the following code built by GCC 7.4 x86/64 with -O3 flag. I currently don't have access to msvc compiler to test it but I imagine it can be around the same number with a suitable configuration and compiler/build flags.
#include <string>
#include <iostream>
#include <fstream>
#include "json.hpp"

using namespace std;

int main() {

    ifstream fin("scryfall-all-cards.json");
    if(!fin)
    {
        cout << "Failed to open file." << endl;
        return 1;
    }

    auto json = nlohmann::json::parse(fin);

    cout << "json parsed. Press any key to exit" << endl;
    string dummy;
    cin >> dummy;

    return 0;
}

Release mode will definitely improve memory on Visual C++, as the debug versions of the containers include extra information to aid with iterator debugging issues.  FYI, TreeNode is an internal implementation detail of std::map on VC++.  You'd want to look at the size of basic_json, not the size of TreeNode<basic_string, basic_json> to see what memory size comes from the library itself.  As a benchmark, you could compare it to the size of TreeNode<char, char> which should be the smallest possible TreeNode.

Already posted in other issue, but reposting here...
I just got into this same issue, some 1.6GB json wouldn't load (even with 10GB memory).
I tried other libraries, but couldn't find any solution, so I coded my own solution here: https://github.com/igormcoelho/vastjson/
If this is useful to someone in the future, it just indexes top-level entries as strings (that consumes less space), and put into nlohmann::json when necessary. It worked and I could finally process my big json (with around 2.3GB RAM)!

Sounds great! Is it something that is compatible to this library? Would be great to have that feature.

Hi @nlohmann, thanks for the kind feedback. I think it's very possible and fundamental to have such feature in this project, as we live in "Big Data World" now, and as currently nlohmann::json is de facto standard C++ json library, the way I know it. I've never looked inside the code, so I'll just put some thoughts here, that may fit or not.
What I did was a very fast workaround, that has some weaknesses (but worked for the purposed I needed). I needed to work with a 1.6GB JSON file, that have around 5000 top level entries (around 150KB each, all entries in the format: "key" : { /* other substructures here */} ). To solve this quickly, I counted nesting levels  '{' and '}' and stored all top-level structures as pure std::string, in a std::map<string, string>, that works side-by-side with another std::map<string, nlohmann::json>.
Every time user asks for some specific key in operator[], I check if json structure already exists, then I return a reference to it. If it doesn't exist yet, I parse the corresponding string with nlohmann::json::parse( str ) before returning, and clear the corresponding string in cache, to avoid wasting precious memory. This way, the global storage is roughly the same size as in disk, besides the previously loaded json structures.
One nice feature there is that it allows creating json from std::ifstream* (or std::unique_ptr<std::ifstream>), meaning that the JSON is consumed on disk only when user requires some key not yet known in cache (and this makes the object creation instant, regardless of size in disk, which is good when experimenting several times with large files).
I added some auxiliar methods, like cacheSize(), unload(key), etc, that are large-file-application specific (for example, when some user wants to return some json back to string cache, for future usage, and less memory consumption).
Now the drawbacks: currently no symbols ", { and } are allowed inside key strings; all top-level entries must be objects (not direct numbers, strings, etc). This means that, as it is now, it is still quite limited (regarding all possible json formats), but I think it's possible to fix. Another drawback: only top-level entries are cached, so if JSON has a single top-level entry, the library is useless.
Regarding nlohmann::json, as I understood from some comments in this thread, I think that there could be some alternative ready-to-use parsers, that user just pass as some option (maybe some template option on nlohmann::json? I haven't looked at any of it now...), and I would suggest at least two strategies:

top-level caching: means that all entries in top level are stored as strings, and parsed only when necessary (similar to what I did there, but fixing the character parsing drawbacks)
global caching: means that all json structures are kept as strings, unless some user "visits" them.

There is still another problem, as users keep visiting "nodes", these nodes expand, so it would be nice to have some strategy to move them back to string cache strategy (as my unload(key) strategy), to allow "unvisiting" nodes (something like "I'm done working with you, for now...").
Sorry for the long post, if you think some of these ideas are reasonable within the scope of nlohmann::json, I could take some more time to understand how it works and maybe help "solving this issue". Best regards!

Some updates on this @nlohmann.. afterall, I had to study a little bit more on nlohamann::json library, and learned a few interesting things.
One thing I learned is that my first approach was easily broken, as long as someone had a list that included objects inside... something like:
{ "A" : [ { "B" : 1 } ]  }

The simplest parenthesis counting that I did, could not handle any sort of arrays (ok, my file did not have any lists...). So, in order to provide greater generality, I used nlohmann::json to parse fields, and I had to do it twice, as I couldn't find a way to escape exception handling over partial parsings (maybe it's something that exists already, but I just did it twice and it worked).
Another thing is that I realized two "greater" strategies may exist:

huge objects (and someone wants to lazily explore fields)  - this was my original inspiration
huge lists (and someone wants lazily explore list items) - I saw an interesting question regarding this

But for even greater generality, maybe what users want, is somehow indicating which fields should be explored lazily, as they are supposed to be huge (huge objects or huge lists).
So, maybe a more informed outline of an approach could be:

create a basic_json type, named basic_json_lazy, that holds a string internally, but can actually represent a json string, json object, json list, anything... it will be a string, to keep memory low, and as soon as user tries to "explore" its content (by conversion operators), it gets "parsed" into some real object/list/etc
user can manually perform the reverse operation, for example, transforming some json object field into json_lazy field (with string serialized content of json object) - this prevents the need of the "unvisit" operation mentioned earlier
during nlohmann::json::parse operation, user can add some sort of lazy_map, that indicates which fields are supposed to be parsed "lazily". Example: "B" -> "B1" is lazy, but rest is parsed instantly, meaning that object "B1" will be "basic_json_lazy" type, only containing string, not the real constructed type.

Just to conclude this, I'm quite bothered that a huge JSON that only contain dictionaries (no lists) can allow such simple strategy that parses it in 20 seconds, but as soon as I try to achieve greater generality, now it costs 244 seconds (10x times more), but with memory controlled, at least... so, it really gets me to think that perhaps some constexpr/template flags that informed no list allowed or no dictionary/object allowed can actually help a lot when files get big. Or maybe it's just that my implementation is so bad. :joy: anyway, it's working, thanks for all the efforts on this amazing library.

Shouldn't you just use the nlohmann library's SAX parser for this?  Instead of building the object-representation in memory like the normal parsing does, it could instead build a map or hashtable that contains keys -> offsets into the original data.  This could do things like record only the base offset of arrays, use a hashing strategy for the keys at each level (rather than a hashtable with string keys), etc.  This "index" data structure could then be used to extract json objects on demand by looking up their location rapidly.  Properly abstracted it could also index the original file on disk so that none of the data is in memory.

@abrownsword I'm speaking as a regular user that just tried to open a 1.6GB json file and realized that it wouldnt work on a 16GB RAM computer... and maybe thats acceptable somewhere worldwide, but here in Brazil we usually dont have access to such machinery. On the other hand, I have quickly coded a solution that worked for me, by counting brackets,but I feel that more and more people will arrive here snd want some straightforward solution.
That said, saybe the solution is to write some sax_parser, but personally I dont have a clue on how to do that (ok, if I read documentation, etc, I'll be able to), but maybe there is some fundamental abstraction missing on the library, something that represents this caching behavior. Or maybe not.

There is an abstraction to respond to parsing events in a JSON SAX stream.  This is fairly easy to use, and it wouldn't be to hard to use it to build an index... except for one problem:  I don't believe it provides a current index.  Perhaps the API could be enhanced to do so?  I could certainly use that in my use of the SAX parser for error reporting.

I don't know about the JSON SAX API, but the few lines I've read of the code, I get the feeling that many things can be done, flexibility seems to be quite high. Maybe some internal things needs some adjustments, I'm not capable of giving any suggestions in that direction, at the moment.
But one thing I could give advices is: if it's indeed possible to implement such strategies as external/auxiliary SAX parsers, then I think some of these should be offered to users, as "standard" implementations for "standard big-data problems". That's the spirit of the contribution I'm trying to pass on to this amazing library, is that maybe there should be some responses to users that come here and couldn't parse large files. Maybe an answer is: "do you have so many fields in top-level object/dictionary? try this sax_parser_big_objects. Or maybe, do you have long lists? try this sax_parse_big_lists".
It's a complicated compromise for a general library like this, being able to be efficient for tiny json, but I feel that the needs for large-file parsing will be highly necessary, every day more and more, and simply there are no ready-to-use alternatives out there at the moment.
Thanks for the directions @abrownsword , from the little I understood, maybe I could at least avoid the double-parsing I'm making, that saves me 50% of computational time, which is good. But as long as memory is contained, I'm currently ok with 10x more computational costs, even I can decrease it in next weeks/months.

I think that's a good suggestion @igormcoelho.  My (limited) experience with the existing SAX interface is that it doesn't provide the byte offset at which each parsing event happens.  If it did provide that, it would be a lot more powerful.  @nlohmann -- perhaps there is a non-breaking way to make the current byte offset available?

For what it's worth, RapidJson by Tencent resolved my issues. You can
examine what they did to keep memory low.
On Mon, Apr 12, 2021, 10:44 AM Andrew Brownsword @.***>
wrote:

I think that's a good suggestion @igormcoelho
https://github.com/igormcoelho. My (limited) experience with the
existing SAX interface is that it doesn't provide the byte offset at which
each parsing event happens. If it did provide that, it would be a lot more
powerful. @nlohmann https://github.com/nlohmann -- perhaps there is a
non-breaking way to make the current byte offset available?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/nlohmann/json/issues/1613#issuecomment-818002195, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AB6DF2BM4LTQFHVYTNRACETTIMWQVANCNFSM4HQOUQFQ
.

For what it's worth, RapidJson by Tencent resolved my issues. You can examine what they did to keep memory low.

This is interesting to know @Meerkov, because as soon as I got this issue here, I tried with RapidJson and had exactly the same issue. Document loading consumed all my memory. And it was because of your message up there that I decided to see their project hahaha :joy: but as soon as I had issues again, I decided to code my own solution. I'll take another look at their project.

@Meerkov just re-did my test again, I realized I was using rapidjson 1.1.0, quite old (2016), so now I tried main branch version.
   FILE* fp = fopen("giant.json", "rb");
   char readBuffer[65536];
   FileReadStream is(fp, readBuffer, sizeof(readBuffer));
   Document d;
   d.ParseStream(is);
   fclose(fp);

It consumed 4.37GB for my 1.6GB file. It's better than 10GB, but still unusable to me, as our machines are mostly limited by 8GB (some have even less memory...). But anyway, thanks for the advice, it may indeed work for many people and it's good to know that.

@igormcoelho Can you share the file?

@igormcoelho Can you share the file?

Sorry for the delay in the response @nlohmann... I think that, for the moment, I cannot share the file due some privacy issues. But I can later mimic the file structure and then come back here.. maybe with a random generator of few lines (it's much better approch for testing I guess).
In any case, as a followup, I continue to work with the workarounds at VastJSON (which are now multiple, due to many distinct big situations), but I've already noticed that some of the workarounds could be ported here. My ugly solution via exception catching and injecting a manual stream over nlohmann::json could perhaps provide the offset indexer we need @abrownsword, without needing to change the library. However, it's ugly. It's still on stack to study how to do that better, as for the moment I'm just successfully surviving those big jsons. I'll come back here.

In fact @nlohmann , I notice that the worst behavior seems to happen in a file with quite simple structure.
It's a set of square matrices, but organized as dictionaries.
Example:
{
   "Entry1": {
          "A": {"A": 0.0, "B": 0.0, "C": 0.0},
          "B": {"A": 0.0, "B": 0.0, "C": 0.0},
          "C": {"A": 0.0, "B": 0.0, "C": 0.0}
   },
   "Entry2": {
          "A": {"A": 0.0, "B": 0.0, "C": 0.0},
          "B": {"A": 0.0, "B": 0.0, "C": 0.0},
          "C": {"A": 0.0, "B": 0.0, "C": 0.0}
   },
   ...
}

If you repeat this set into around 5000 top-level Entries (Entry1, Entry2, ..., Entry5000), where each "matrix" has around 100 to 200 elements, you will likely have a file with ~1.5G (I didn't compute things precisely here).
And this file is the one which seems to expand faster in memory, in a prohibitive manner.
The good side of this is that this is quite tractable, in this case, by providing a top-level parser that takes advantage of no list existing inside the json. Like I said before, maybe the best solution here into nlohmann::json project is to provide some "big parser examples", where this is one of the possible scenarios... on general, if file is big, I think that it will be very likely that one has too many top-level entries, being those dictionaries or some sort of huge list (I just took care of first case at the moment, as it's the one that mattered to me until now, but other cases are also tractable).
Another interesting point that when you have some of those numbers as NaN, as it happened to me, things get even worse, as I needed to find other good solution for that, in a context where the file itself wasn't tractable even with numbers. So, it's maybe another example to provide.


on freebsd, in k_req_incoming, if source is local, the return value must be TRUE

Problem : the election of the BSR is unstable, if the BSR is on a direct route.
Solution : fix the regression brought by the commits :
The commit c8d06b0a3b277cdc8b488845973f4a33c58f112d modified find_vif_direct_local() to take into account addresses that are not part of the interfaces/masks but that are known by the kernel as direct. see https://github.com/troglobit/pimd/pull/153
The commit 4a6b1a94735b7c25de42d72d08ba7f33f7516662 fixed a blocking of pimd by a recusive call of k_req_incoming by removing the call to find_vif_direct_local(), which caused the regression.
The solution chosen is to add a parameter to find_vif_direct_local() to indicate whether a call to k_req_incoming is made.

:clap: Nicely done! :+1:

For future reference, please put whitespace fixes/changes in a separate commit.
Only thing I'll fix after this is minor parameter rename, otherwise very clean :)


Remove old window borders on linux

Chromium still has 4px window borders on linux that make it look outdated. There's also an official bug report about this and a few suggested patches.
https://bugs.chromium.org/p/chromium/issues/detail?id=920768
The only alternative is

Showing a titlebar instead
Then removing the titlebar via the window manager
but that looks terrible

I had looked into the issue not too long after you made it.  I'm sorry for not responding then, I kind of lost track of it until it hit the closed list.
Google is working on fixing this upstream:  2901661.  The changes were initially merged for v93, but it looks like there were issues with the shadows and it has been reverted.  Most likely it will land in v94 now.  There's a different bug report that they seem to be using:  811515
If you're using a tiling window manager and want the border removed entirely while using chromium's CSD, all you need is this change:
--- a/chrome/browser/ui/views/frame/browser_non_client_frame_view.cc
+++ b/chrome/browser/ui/views/frame/browser_non_client_frame_view.cc
@@ -99,7 +99,7 @@
 }

 bool BrowserNonClientFrameView::IsFrameCondensed() const {
-  return frame_->IsMaximized() || frame_->IsFullscreen();
+  return true;
 }

 bool BrowserNonClientFrameView::HasVisibleBackgroundTabShapes(

Thank you very very much for your help :D


refactor(steer_offset_estimator): rework parameters

Description
Implement the ROS Node configuration layout described in https://github.com/orgs/autowarefoundation/discussions/3371 for the steer_offset_estimator package.

Remove the default value from the source code in order to ensure all parameter values are passed from the parameter files.
Create the schema

Tests performed
Package build locally.
colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=Release --packages-up-to steer_offset_estimator
Effects on system behavior
More reliable and faster parameter configuration file creation.
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.
[ ] The PR has been properly tested.
[ ] The PR has been reviewed by the code owners.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[x] There are no open discussions or they are tracked via tickets.
[ ] The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

Looks Good


Use of uninitialized values for empty loops

Surprisingly, this returns zero in numba
@numba.njit
def foo(vals):
    for x in vals:
        pass
    return x

foo(np.zeros(0))

I think this is because it declares the loop variable before the loop without initializing it, and then returns the value.
We can turn this into segfaults eg by iterating over objects instead of plain values:
from numba.typed import List

list_inner = List([0])
object_list = List([list_inner])
object_list.pop()  # it is an empty typed list of objects now

foo(object_list)  # -> segfault

This is on numba HEAD, commit 34ec4ef21f

Thanks for the report. This is known behaviour, in translating from a dynamic to a static representation of the function Numba creates "slots" to store all variables and zero intialises them. There's a PR #7750 which will eliminate dead loops, but in this case that condition cannot be determined using the current type based static analysis (the size of the dynamic containers, like lists, is not part of the type). I think the fix might be quite involved and will probably require the slots carrying information about whether they have been used, but testing this state at runtime could well be costly as it would add branches.


[misc] rename IsCategoryEnabled

Rename to avoid conflict with other code bases.

@gerickson ? is the static inline change sufficient or are we asking for more reorganization?

@gjc13  is this really a rename as the title claims? it looks to me like a move from macro to inline function. Maybe it should be constexpr as well.

@gjc13 is this really a rename as the title claims? it looks to me like a move from macro to inline function. Maybe it should be constexpr as well.

Already renamed. It cannot be constexpr since gLogFilter is involved and this can be set at runtime.


Refactored; added handlers for when partner disconnects, manual disconnect button; other fixes

@Divi09112 Can we change the environment variables to have the full url of STUN server and trackers? It gets messy when testing locally, since on localhost I can't make use of a wss:// connection


[Teams] Update onTeamsMessagingExtensionFetchTask() & add tests for TeamsActivityHandler

Description

Update signature onTeamsMessagingExtensionFetchTask()
Add more Tests for TeamsActivityHandler

Specific Changes

Change onTeamsMessagingExtensionFetchTask() signature (Hackathon Feedback)

From:
protected async onTeamsMessagingExtensionFetchTask(context: TurnContext, query: MessagingExtensionQuery): Promise<MessagingExtensionActionResponse> {
to:
protected async onTeamsMessagingExtensionFetchTask(context: TurnContext, action: MessagingExtensionAction): Promise<MessagingExtensionActionResponse> {

Update scenarios with new signature for onTeamsMessagingExtensionFetchTask
Add Teams-specific ConversationUpdate tests for TeamsActivityHandler
Add final BadRequest tests for TeamsActivityHandler
Add more NotImplemented tests for TeamsActivityHandler

Testing

Pull Request Test Coverage Report for Build 83397

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.8%) to 87.683%

Totals

Change from base Build 83343:
0.8%

Covered Lines:
4840

Relevant Lines:
5306

💛  - Coveralls


parseMemInfo() ignores unit, leading to incorrect results from /proc/meminfo

Hi all :wave:
Is there a reason that the parseMemInfo() function ignores the optional unit field? This can lead to incorrect results on systems that do supply the field, as the units are obviously not adjusted accordingly.
A small example program/output:
meminfo.go
package main

import (
	"bufio"
	"fmt"
	"os"
	"strings"

	"github.com/dustin/go-humanize"
	"github.com/prometheus/procfs"
)

func main() {
	fmt.Println("Meminfo sample entries:")

	meminfoFile, err := os.Open("/proc/meminfo")
	if err != nil {
		panic(err)
	}
	defer meminfoFile.Close()

	scanner := bufio.NewScanner(meminfoFile)
	for scanner.Scan() {
		line := scanner.Text()
		fields := strings.Fields(line)
		key := strings.TrimSuffix(fields[0], ":")
		if key == "MemTotal" || key == "MemFree" {
			fmt.Println(line)

			val, err := humanize.ParseBytes(fmt.Sprintf("%s %s", fields[1], fields[2]))
			if err != nil {
				panic(err)
			}
			fmt.Printf("%s humanized: %s\n", key, humanize.Bytes(val))
		}
	}

	if scanner.Err() != nil {
		if err != nil {
			panic(err)
		}
	}

	fmt.Println()
	fmt.Println("procfs lib values:")
	fs, err := procfs.NewFS("/proc")
	if err != nil {
		panic(err)
	}

	meminfo, err := fs.Meminfo()
	if err != nil {
		panic(err)
	}

	fmt.Printf("MemTotal: %d bytes\n", *meminfo.MemTotal)
	fmt.Printf("MemTotal humanized: %s\n", humanize.Bytes(*meminfo.MemTotal))
	fmt.Printf("MemFree: %d bytes\n", *meminfo.MemFree)
	fmt.Printf("MemFree humanized: %s\n", humanize.Bytes(*meminfo.MemFree))
}

output:
/tmp/procfs -> go run meminfo.go
Meminfo sample entries:
MemTotal:       32588196 kB
MemTotal humanized: 33 GB
MemFree:         9787012 kB
MemFree humanized: 9.8 GB

procfs lib values:
MemTotal: 32588196 bytes
MemTotal humanized: 33 MB
MemFree: 9787012 bytes
MemFree humanized: 9.8 MB

This also looks like a good reason why the node exporter chooses to do (similar) but custom parsing of /proc/meminfo to account for the optional unit field, rather than use this library for meminfo as it's used in other collectors.
Using the humanize lib that's used elsewhere in other prometheus projects, I have what I believe to be a functional commit here that I'd be happy to contribute if this is determined to be a bug/should be fixed.
Example output with the patched lib:
/tmp/procfs -> go run meminfo.go
Meminfo sample entries:
MemTotal:       32588196 kB
MemTotal humanized: 33 GB
MemFree:         9685964 kB
MemFree humanized: 9.7 GB

procfs lib values:
MemTotal:<PHONE_NUMBER>0 bytes
MemTotal humanized: 33 GB
MemFree:<PHONE_NUMBER> bytes
MemFree humanized: 9.7 GB

Guess we were under the assumption that it's always the same unit. If not, that's an issue..

No worries, I've opened a PR to fix it :+1:

The humanize library produces incorrect results, as the kernel is exposing kB to mean 1024. But the humanize library kB means 1000.

Closing since #569 was merged


Unused variables (such as qasm(..., prec)) are being removed

Fixes #2452
Summary

Removed all unused variables from qiskit/qasm/node.
Removed all unused variables from qiskit/quantum_info/operators/channel/transformations.py.
Removed all unused variables from qiskit/tools/jupyter.
Removed all unused variables from qiskit/visualization.
Marked all unused variables from qiskit/qams as ignored variables ( _ ), they are required by the 'lex' package.
Unused variables from test/python/test_util.py cannot be removed, the 'mock' package requires them to function properly, even if they are not used.
The use of 'qobj' in qiskit/test/mock/fake_backend.py could only be reduced to the 'run' method, that extends the method from the superclass and requires the method to mantain the original method's signature.

Updated checklist

[x]  qiskit/qasm/node/binaryop.py
[x]  qiskit/qasm/node/binaryoperator.py
[x]  qiskit/qasm/node/external.py
[x]  qiskit/qasm/node/format.py
[x]  qiskit/qasm/node/id.py
[x]  qiskit/qasm/node/indexedid.py
[x]  qiskit/qasm/node/intnode.py
[x]  qiskit/qasm/node/prefix.py
[x]  qiskit/qasm/node/real.py
[x]  qiskit/qasm/node/unaryoperator.py
[x]  qiskit/qasm/qasmlexer.py
[x]  qiskit/qasm/qasmparser.py
[x]  qiskit/quantum_info/operators/channel/transformations.py
[x]  qiskit/test/mock.py
[x]  qiskit/tools/jupyter/backend_monitor.py
[x]  qiskit/tools/jupyter/backend_overview.py
[x]  qiskit/visualization/latex.py
[x]  qiskit/visualization/matplotlib.py
[x]  test/python/test_util.py

Thanks a lot for getting into this!
If you are planing to fully fix #2452 in this PR, you can rename this issue with the prefix [WIP] to signal that you are working on it and it should not be merged as-is yet.

Thanks again @1ucian0 , totally agreed with your comments! I've just found a few variables more, so I'll have a full look at it again. Should I remove the uses of 'prec' when converting to QASM strings then, according to your comment?
To fully fix #2452, I would need some advise on the three files that I left unchecked in the list, I'm not quite sure about those :)

Ready for review!

Ah, it looks like the conformance tests do specify a prec kwarg: https://github.com/Qiskit/openqasm/blob/master/test/harness.py#L43 although it's not clear why the warning is only raised once, instead of once for each file. Also, the warning isn't super helpful in this case because it's pointing to it's usage in qiskit/qasm/node/program and not the qasm() call in the conformance tests.

@rafamartinc Can you take a look at @mtreinish 's comments?

@rafamartinc any update on this?
@mtreinish are you suggesting that there should be changes to this PR?

@1ucian0 can you take this PR over?

Yeap. No problem.

Closing in favor of https://github.com/Qiskit/qiskit-terra/pull/4192, because the branch was deleted. Please, follow up the discussion there!


Crash with Blabber

Blabber adds a HandledScreen with no slots, which seems to conflict with the assumption made here that every HandledScreen has a backing inventory.
message.txt

Right, let me see if it's as simple of a fix as I think

By the way, I believe this @Redirect could be replaced with a @WrapOperation to limit mixin incompatibilities with other mods

In 1.20.5 I made it an inject and create my own packet rather than modifying the vanilla packet, but I don't really want to backport that for older versions (I probably should, I just want to move forward with the mc versions)

Published the fix for 1.20 - 1.21, could you please confirm that it's resolved?

Hey Gravity ! I'm the guy who reported the bug to Pyro's discord mod. It seem to be fixed :D
Thanks a lot for you quick fix dude. Really appreciate it ! <3

No worries, thanks for letting me know about the bug! :D


material_duration_picker : Null check operator used on a null value

I'm trying to use it like this:
onTap: () {
        showDurationPicker(
              durationPickerMode: DurationPickerMode.ms,
              context: context,
        );
},

Here's the stack :
══╡ EXCEPTION CAUGHT BY WIDGETS LIBRARY ╞═══════════════════════════════════════════════════════════
The following _TypeError was thrown building _FormScope:
Null check operator used on a null value

The relevant error-causing widget was:
  Form-[LabeledGlobalKey<FormState>#fa340]
  Form:file:///C:/Users/Victor/AppData/Local/Pub/Cache/hosted/pub.dev/material_duration_picker-0.0.2+1/lib/material_duration_picker.dart:22  34:30

When the exception was thrown, this was the stack:
#0      DurationPickerMaterialLocalizations.of (package:material_duration_picker/src/localizations.dart:62:54)
#1      _DurationPickerState.didChangeDependencies (package:material_duration_picker/material_duration_picker.dart:2401:57)
#2      StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:5766:11)
#3      ComponentElement.mount (package:flutter/src/widgets/framework.dart:5593:5)
...     Normal element mounting (29 frames)
#32     Element.inflateWidget (package:flutter/src/widgets/framework.dart:4468:16)
#33     MultiChildRenderObjectElement.inflateWidget (package:flutter/src/widgets/framework.dart:7035:36)
#34     MultiChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:7047:32)
...     Normal element mounting (190 frames)
#224    Element.inflateWidget (package:flutter/src/widgets/framework.dart:4468:16)
#225    Element.updateChild (package:flutter/src/widgets/framework.dart:3963:18)
#226    _LayoutBuilderElement._rebuildWithConstraints.updateChildCallback (package:flutter/src/widgets/layout_builder.dart:207:18)
#227    BuildOwner.buildScope (package:flutter/src/widgets/framework.dart:3038:19)
#228    _LayoutBuilderElement._rebuildWithConstraints (package:flutter/src/widgets/layout_builder.dart:231:12)
#229    RenderObject.invokeLayoutCallback.<anonymous closure> (package:flutter/src/rendering/object.dart:2719:59)
#230    PipelineOwner._enableMutationsToDirtySubtrees (package:flutter/src/rendering/object.dart:1098:15)
#231    RenderObject.invokeLayoutCallback (package:flutter/src/rendering/object.dart:2719:14)
#232    RenderConstrainedLayoutBuilder.rebuildIfNecessary (package:flutter/src/widgets/layout_builder.dart:278:5)
#233    _RenderLayoutBuilder.performLayout (package:flutter/src/widgets/layout_builder.dart:369:5)
#234    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#235    RenderPadding.performLayout (package:flutter/src/rendering/shifted_box.dart:234:12)
#236    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#237    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
#238    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#239    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
#240    RenderCustomPaint.performLayout (package:flutter/src/rendering/custom_paint.dart:569:11)
#241    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#242    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
#243    _RenderCustomClip.performLayout (package:flutter/src/rendering/proxy_box.dart:1448:11)
#244    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#245    RenderConstrainedBox.performLayout (package:flutter/src/rendering/proxy_box.dart:291:14)
#246    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#247    RenderPositionedBox.performLayout (package:flutter/src/rendering/shifted_box.dart:451:14)
#248    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#249    RenderPadding.performLayout (package:flutter/src/rendering/shifted_box.dart:234:12)
#250    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#251    RenderPadding.performLayout (package:flutter/src/rendering/shifted_box.dart:234:12)
#252    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#253    RenderPadding.performLayout (package:flutter/src/rendering/shifted_box.dart:234:12)
#254    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#255    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
#256    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#257    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
#258    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#259    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
#260    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#261    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
#262    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#263    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
#264    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#265    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
#266    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#267    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
#268    RenderOffstage.performLayout (package:flutter/src/rendering/proxy_box.dart:3728:13)
#269    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#270    RenderProxyBoxMixin.performLayout (package:flutter/src/rendering/proxy_box.dart:111:21)
#271    RenderObject.layout (package:flutter/src/rendering/object.dart:2608:7)
#272    _RenderTheaterMixin.layoutChild (package:flutter/src/widgets/overlay.dart:1002:13)
#273    _RenderTheater.performLayout (package:flutter/src/widgets/overlay.dart:1311:9)
#274    RenderObject._layoutWithoutResize (package:flutter/src/rendering/object.dart:2446:7)
#275    PipelineOwner.flushLayout (package:flutter/src/rendering/object.dart:1052:18)
#276    PipelineOwner.flushLayout (package:flutter/src/rendering/object.dart:1065:15)
#277    RendererBinding.drawFrame (package:flutter/src/rendering/binding.dart:602:23)
#278    WidgetsBinding.drawFrame (package:flutter/src/widgets/binding.dart:1164:13)
#279    RendererBinding._handlePersistentFrameCallback (package:flutter/src/rendering/binding.dart:468:5)
#280    SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:1397:15)
#281    SchedulerBinding.handleDrawFrame (package:flutter/src/scheduler/binding.dart:1318:9)
#282    SchedulerBinding._handleDrawFrame (package:flutter/src/scheduler/binding.dart:1176:5)
#283    _invoke (dart:ui/hooks.dart:312:13)
#284    PlatformDispatcher._drawFrame (dart:ui/platform_dispatcher.dart:419:5)
#285    _drawFrame (dart:ui/hooks.dart:283:31)

Any ideas ?

Thank you. Can you have a look here?
https://github.com/Oberhauser-Dev/dart_packages/issues/28#issuecomment-2523729724

Will do


Where is the dummy app?

For more information, please, take look at dummy app.

Where is the dummy app?!

https://github.com/api-hogs/ember-data-offline/tree/master/tests/dummy


Map blob type from response content type not working with React Native iOS

This fix #3842  is not working well with React Native iOS & Axios. For some reason, iOS / iPadOS does not accept / like the new Blob() of a Blob.
The error it throws when trying to convert the blob to base64 is: Unable to resolve data for blob

Anyone else facing this issue? We are not able to update to the latest versions due to the issue.


fix typo

remove a lost (

Hey! Thanks for this PR!
Can you change the commit message to "fix: typo" so it matches our usual messages?

done! thanks for the heads up!


Introduce new APLs: fileAPL and vercelAPL

Closes #33
The old implementation from the app-template had a single implementation for both use cases: local env and vercel. Decided to split it into separate APLs, which resulted in a simpler, one-purpose code.
Now the choice of storage will be on the App developer, for example:
import { fileAPL, vercelAPL } from "@saleor/app-sdk/APL";

// For local development store auth data in the `.envfile`.
// For app deployment on Vercel with Saleor CLI, use vercelAPL.
export const apl = process.env.VERCEL === "1" ? vercelAPL : fileAPL;

( https://github.com/saleor/saleor-app-template/pull/79 )

Usage example: https://github.com/saleor/saleor-app-template/pull/79

@krzysztofwolski can you please extend docs and add some section about usage - especially that app will locally use local env, but on vercel use its implementation?


Library does not support 'winsch'

The library does not currently support the winsch function. This allows you to insert a character into the window specified at the current position, pushing other characters to the right (and possibly pushing the final char off the screen entirely).
Ideally this would be implemented as an insch method on the Window type, along with the mvinsch variant as well.

Added insch and mvinsch to Window in 0.10 that I just released.


new aria in html rule: placeholer / aria-placeholder attributes

2 tests to implement checks for
https://w3c.github.io/html-aria/tests/placeholder-test.html

Added aria-palceholder as a supported value for input type="number".
Added check for aria-placeholder being used with the placeholder attribute.

retested and flags 8 issues as expected


Is there any way of loading the encoder from local path instead of downloading it?

I'm implementing the API on a website. I made a code that loads the model I trained and predicts on some images, but every time I load the model, the model start downloading the encoder weights. This slows down my tests. I downloaded the encoder so I was wondering if there was any way to load these weights by passing the path to the encoder?
modle_path = os.path.join(os.path.dirname(__file__), 'modelos_seg/model_dpl_efficientnet-b7')
model = smp.from_pretrained(modle_path) # Load the pre-trained model

Downloading: "https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b7-dcc49843.pth" to /root/.cache/torch/hub/checkpoints/efficientnet-b7-dcc49843.pth
100%|██████████| 254M/254M [00:57<00:00, 4.67MB/s]

Hi, thanks for the question! The reason is that in config "encoder_weights" is set to "imagenet". That is why the model first loads "encoder" weights, and then loads weights for the whole trained model. You can modify your saved config.json to store "encoder_weights: null", it should help.
This behavior should be avoided, I would appreciate the fix if you have the bandwidth! Thanks for reporting anyway!


Staging

Pull Request type

Please check the type of change your PR introduces:

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build-related changes
[ ] Documentation content changes
[x} Testing
[ ] Other (please describe):

What is the current behavior?

Issue Number: N/A
What is the new behavior?

Does this introduce a breaking change?

[ ] Yes
[ ] No

Other information

Codecov Report

Merging #5 (b19ca0f) into main (0592273) will increase coverage by 0.26%.
The diff coverage is n/a.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##             main       #5      +/-   ##
==========================================
+ Coverage   53.44%   53.71%   +0.26%
==========================================
  Files          14       14
  Lines        2277     2277
==========================================
+ Hits         1217     1223       +6
+ Misses       1060     1054       -6

see 1 file with indirect coverage changes


Terminal: Job table full or recursion limit exceeded, in long running instances

Version Details

Version: 1.80.0-insider
Commit: 73eb619afb87cab4aa83a569259c891b6983c11e
Date: 2023-06-19T05:56:57.953Z
Electron: 22.3.11
ElectronBuildId: 21658788
Chromium: 108.0.5359.215
Node.js: 16.17.1
V8: <IP_ADDRESS>-electron.0
OS: Darwin arm64 22.5.0

I use zsh:
zsh 5.9 (x86_64-apple-darwin22.0)

My terminals have been running for a while, since at least last week. They went through a few VS Code updates but were always resumed. This afternoon I logged in and saw this on some of them:

Related: #166303 #165648 #165648

No, I haven't disabled it.

This is almost certainly related to shell integration but it doesn't look like there's any recursion so I'm not sure how to action this one.

@joaomoreno I have not seen this, thus closing. Pls reopen if you've been able to reproduce it recently.


Apache Arrow can not read a Parquet File written with Parqet-Avro 1.11.0 with a Map field

From my StackOverflow in relation to an issue I'm having with getting Snowflake (Cloud DB) to load Parquet files written with version 1.11.0
The problem only appears when using a map schema field in the Avro schema. For example:

    {
      "name": "FeatureAmounts",
      "type": {
        "type": "map",
        "values": "records.MoneyDecimal"
      }
    }

When using Parquet-Avro to write the file, a bad Parquet schema ends up with, for example

message record.ResponseRecord {
  required binary GroupId (STRING);
  required int64 EntryTime (TIMESTAMP(MILLIS,true));
  required int64 HandlingDuration;
  required binary Id (STRING);
  optional binary ResponseId (STRING);
  required binary RequestId (STRING);
  optional fixed_len_byte_array(12) CostInUSD (DECIMAL(28,15));
  required group FeatureAmounts (MAP) {
    repeated group map (MAP_KEY_VALUE) {
      required binary key (STRING);
      required fixed_len_byte_array(12) value (DECIMAL(28,15));
    }
  }
}

From the great answer to my StackOverflow, it seems the issue is that the 1.11.0 Parquet-Avro is still using the legacy MAP_KEY_VALUE converted type, that has no logical type equivalent. From the comment on LogicalTypeAnnotation

// This logical type annotation is implemented to support backward compatibility with ConvertedType.
  // The new logical type representation in parquet-format doesn't have any key-value type,
  // thus this annotation is mapped to UNKNOWN. This type shouldn't be used.

However, it seems this is being written with the latest 1.11.0, which then causes Apache Arrow to fail with

Logical type Null can not be applied to group node

As it appears that Arrow only looks for the new logical type of Map or List, therefore this causes an error.
I have seen in Parquet Formats that LogicalTypes should be something like

// Map<String, Integer>
required group my_map (MAP) {
  repeated group key_value {
    required binary key (UTF8);
    optional int32 value;
  }
}

Is this on the correct path?
Reporter: Matthew McMahon / @maccamlc
Assignee: Matthew McMahon / @maccamlc
PRs and other links:

Pull Request

Note: This issue was originally created as PARQUET-1879. Please see the migration documentation for further details.

Daniel Dai / @daijyc:
This seems to be a backward-incompatible change. We cannot read parquet file created pre-1.11.1 using the new version. Here is a sample error message:

org.apache.parquet.io.InvalidRecordException: key_value not found in optional group canonicals (MAP) {
  repeated group map (MAP_KEY_VALUE) {
    required binary key (ENUM);
    optional group value {
      optional int32 index;
      optional int64 pinId;
      optional group indexableTextIndexes (LIST) {
        repeated int32 indexableTextIndexes_tuple;
      }
      optional int32 indexExpLq;
      optional int32 indexExp;
      optional boolean imageOnly;
      optional boolean link404;
      optional boolean unsafe;
      optional boolean imageNotOnPage;
      optional boolean linkStatusError;
    }
  }
}
	at org.apache.parquet.schema.GroupType.getFieldIndex(GroupType.java:176)
	at org.apache.parquet.schema.GroupType.getType(GroupType.java:208)
	at org.apache.parquet.schema.GroupType.checkGroupContains(GroupType.java:348)
	at org.apache.parquet.schema.GroupType.checkContains(GroupType.java:339)
	at org.apache.parquet.schema.GroupType.checkGroupContains(GroupType.java:349)
	at org.apache.parquet.schema.MessageType.checkContains(MessageType.java:124)
	at org.apache.parquet.hadoop.api.ReadSupport.getSchemaForRead(ReadSupport.java:56)
	at org.apache.parquet.hadoop.thrift.ThriftReadSupport.init(ThriftReadSupport.java:187)
	at org.apache.parquet.hadoop.InternalParquetRecordReader.initialize(InternalParquetRecordReader.java:200)
	at org.apache.parquet.hadoop.ParquetRecordReader.initializeInternalReader(ParquetRecordReader.java:182)
	at org.apache.parquet.hadoop.ParquetRecordReader.initialize(ParquetRecordReader.java:140)
	at org.apache.spark.rdd.NewHadoopRDD$$anon$1.liftedTree1$1(NewHadoopRDD.scala:216)
	at org.apache.spark.rdd.NewHadoopRDD$$anon$1.<init>(NewHadoopRDD.scala:213)
	at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:168)
	at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:71)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:131)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

I am not sure what's the best way to fix it. I am thinking about adding a walker in the construct of FileMetaData to fix the schema, is it a good idea?

Matthew McMahon / @maccamlc:
Opened a PR to address this at https://github.com/apache/parquet-mr/pull/798


ENH: Support for complex functions in binned_statistic_dd

Reference issue
Closes #9712
What does this implement/fix?

Implements the suggested fix in #9712. Adds support for complex numbers.
Additional information
This is my first pull request on open source software. I would greatly appreciate any and all criticisms or suggestions :)

Thanks for the suggestion. Wasn't sure how deep the test should be, and kind of guided myself with the discussion on #9712. I can definitely add more things to test if necessary.

Thanks for the comments, should I close this PR and open a new one with a proper branch?

No need. That's just a suggestion for the future.

I appreciate the comments and especially your patience. I've merged your pull request, I had no idea about that etiquette and I appreciate the heads up.  Only thing I don't understand is how to define the median, min, or max for complex numbers. Should those functions throw an error, should the components be calculated separately, or is there some way of assigning ordering?
I think I can handle the test cases. I'll do my best.

Oops, well here's the test I had in mind. Something like:
    @pytest.mark.parametrize("dtype", [np.float64, np.complex128])
    @pytest.mark.parametrize("statistic", [np.mean, np.median, np.sum,
                                           np.std, np.min, np.max,
                                           'count', lambda x: (x**2).sum()])
    def test_1d_all(self, dtype, statistic):
       def ref_statistic(x):
           return len(x) if statistic == 'count' else statistic(x)

       rng = np.random.default_rng(3704743126639371)
       n = 10
       x = rng.random(size=n)
       i = x >= 0.5
       v = rng.random(size=n)
       if dtype is np.complex128:
           v = v + rng.random(size=n)*1j

       stat, _, _ = binned_statistic_dd(x, v, statistic, bins=2)
       ref = np.array([ref_statistic(v[~i]), ref_statistic(v[i])])
       assert_allclose(stat, ref)
       assert stat.dtype == np.result_type(ref.dtype, np.float64)

Thanks, I left it in pretty much as-is since it kinda blew my mind a little bit w.r.t. testing, I had no idea you could stack parametrize like that, I've gone and implemented that with all my project tests. Thank you so much for the help.

This is looking pretty good (from my cell phone). I'll take a closer look, but are you happy with it?
@tirthasheshpatel could we get your eyes on this, too? I ended up influencing the code enough that an independent review would be good.

I'm very happy with it, and this experience has been extremely great for my growth as a developer so thank you for your patience and helpful understanding.

I haven't had the ability to go in and make some changes, but I do also want to add a test for a user-fed statistic that will always return complex values, just to be thorough.

I went ahead and added such a test. I checked the code pathway with the debugger, and I think this hits the path you were shooting for, so if CI comes back green I'll merge. BTW just a reminder: in the future, please open PRs from a feature branch (not named main). It came up because it's a little more work to work with someone else's branch when they give it a name that is already taken (e.g. by the main branch of SciPy.

lint failure is unrelated

Seems like #17276 would break the CI for all open PRs. I didn't think about that before merging! Can you please open a PR that removes the accents? I can review and merge it right away.

Thanks @cesaregarza @tirthasheshpatel!


Add benefit name to nobenefitreason string

Right now the judge won't be able to match reason for not having proof  to a specific benefit. Could we improve this by passing  [question title]":"[reasonfornobenefit] ?

This is complete on UAT.


Various traces API and CLI improvements

This pull request updates traces list API endpoint and st2 trace list CLI commands so it works in the manner user expects it to and so it's consistent with the st2 execution list command and related API endpoint:

Update default sort order for the API endpoint - now traces are sorted by start_timestamp (descending) first and trace_tag second
Fix -n parameter and make sure traces are sorted correctly (descending order) when this parameter is supplied
Allow user to specify sort order through the API and CLI (e.g. st2 trace list -n 5 --sort asc)

Closes #2855.

LGTM.


WIP: Needed changes to Ford.

Add needed changes to ford-devel. Add wheel radius for cars, add driver torque, SOD BLIS, and other things. Branch will be kept open until changes are merged to be working.

changes finished in ford-devel-master. new info in WIP: Ford Lockout


GE Dishwasher

Not pulling in sensor data for energy usage. No energy usage metrics available

I don't have the ERD codes that correspond to the energy usage metrics.  If you want to help, you'll need to go to the sdk repo and provide some debug info and interpretation from your device.  Will reopen should we make any progress on this.


Add saveEncoding to web options

Codecov Report

Merging #21 (f327dcd) into main (56c6a16) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##              main       #21   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           15        15
  Lines         2479      2479
  Branches       445       445
=========================================
  Hits          2479      2479

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


FancyZone not honouring Taskbar

Microsoft PowerToys version
0.60.1
Running as admin

[ ] Yes

Area(s) with issue?
FancyZones
Steps to reproduce
Following on from last post on #4694
Setup custom zone or use preset, windows go behind taskbar when fit to fancy zone.
✔ Expected Behavior
Fancy zone should calculate area excluding taskbar, and window should fit on screen space excluding task bar.
❌ Actual Behavior
The windows fit to the configured zones go behind the taskbar.
When setting taskbar to not display on all monitors and set it back again, it fixed it.
Other Software
No response

Following. I've had the same issue for months.

Also happening to me - no amount of restarting or reapplying seems to fix it.

Same here with 0.61.1

I'm on v0.61.1 and I solved this issue by enabling off and on again the FancyZones

I'm on v0.61.1 and I solved this issue by enabling off and on again the FancyZones

Same happend to me, but this helped.

I'm on v0.61.1 and I solved this issue by enabling off and on again the FancyZones

Same happend to me, but this helped.

That helps until you have some monitor switches, or some monitor(s) turned off/on because of power saving

I have the same issue. I have tried @dutzman suggestion but FZ still not honouring taskbar on second monitor.

I'm having the same issue.  I'm using multiple virtual desktops; it only happens on the first virtual desktop.

Windows 11 build 22621.900
Powertoys v0.64.1
I noticed this issue a couple of weeks ago, which looks to coincide with the installation of v.064.1, and I know this issue is recent. Turning FancyZones off and on resolves the problem and windows snap into the correct zones.

/dup of https://github.com/microsoft/PowerToys/issues/10903


[Bug] "ITMS-90809: Deprecated API Usage" in Xamarin.Youtube.Player.iOS

After adding the NuGet package for Xamarin.YouTube.Player.iOS to my project and submitting a version to TestFlight I receive the following mail from Apple:

Dear Developer,
We identified one or more issues with a recent delivery for your app, "xxxxxxx" x.x.x (x.x.x.x). Your delivery was successful, but you may wish to correct the following issues in your next delivery:
ITMS-90809: Deprecated API Usage - Apple will stop accepting submissions of apps that use UIWebView APIs starting from December 2020 . See https://developer.apple.com/documentation/uikit/uiwebview for more information.
After you’ve corrected the issues, you can upload a new binary to App Store Connect.
Best regards,

Taking a quick look over the sources it seems that the YouTube component for iOS is still using / referring to the now long deprecated UIWebView instead of using WKWebView.
NuGet version: Xamarin.YouTube.Player.iOS v0.1.6

This issue also seems present in Xamarin.iOS.MaterialComponents.
grep -r "UIWebView" .

Binary file ./iOS/obj/iPhoneSimulator/Debug/mtouch-cache/MaterialComponents.framework/MaterialComponents matches
Binary file ./iOS/bin/iPhone/Release/.../Frameworks/MaterialComponents.framework/MaterialComponents matches
Binary file ./iOS/bin/iPhoneSimulator/Debug/.../Frameworks/MaterialComponents.framework/MaterialComponents matches

@gadeweever which version does this relate to? I'm also using MaterialComponents (<IP_ADDRESS>) and don't get a reference:
./packages/Xamarin.iOS.MaterialComponents.<IP_ADDRESS>/lib/Xamarin.iOS$ grep -i UIWebView MaterialComponents.dll
./packages/Xamarin.iOS.MaterialComponents.<IP_ADDRESS>/lib/Xamarin.iOS$ strings MaterialComponents.dll | grep -c -i UIWebView
0

Compared to YouTube.Player.iOS (0.1.6):
./packages/Xamarin.YouTube.Player.iOS.0.1.6/lib/XamariniOS10$ grep UIWebView YouTube.Player.iOS.dll
Binary file YouTube.Player.iOS.dll matches
./packages/Xamarin.YouTube.Player.iOS.0.1.6/lib/XamariniOS10$ strings YouTube.Player.iOS.dll | grep -c UIWebView
117

@Shirk This is on <IP_ADDRESS>. Does the grep change if you also run it on your build output folder?


Export private key should require auth

What
The profile page allows the user to export the private key. This should trigger an auth verification.
Why
The private key is the most important secret for the account. It should never be readily available to anyone just now using the phone.

Not relevant / not sure we want this. Closing.


failed to solve: failed to read dockerfile

Hi,
I have built img from source and am trying to run the examples. I'm doing this on an Ubuntu 18.04 machine.
`sjames@sjames-ideapad:~/playground/img$ img build -t r.j3ss.co/img .
Building r.j3ss.co/img:latest
Setting up the rootfs... this may take a bit.
[+] Building 1.2s (2/2) FINISHED
=> [internal] load build definition from Dockerfile                       0.8s
=> => transferring dockerfile: 2B                                         0.0s
=> [internal] load .dockerignore                                          1.1s
=> => transferring context: 47B                                           0.1s
Error: failed to solve: failed to read dockerfile: open /tmp/buildkit-mount368416638/Dockerfile: no such file or directory
sjames@sjames-ideapad:~/playground/img$ uname -a
Linux sjames-ideapad 5.0.0-23-generic #24~18.04.1-Ubuntu SMP Mon Jul 29 16:12:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
`

I'm also getting this when I use img as a container in a Kubernetes Pod spawned by the Jenkins Kubernetes plugin. The Kubernetes Jenkins plugin mounts the working directory into each container in the pod and I'm wondering if this is something to do with the Dockerfile living on a mounted volume?
My Pod YAML:
apiVersion: v1
kind: Pod
metadata:
  generate-name: agent-k8s-
  labels:
    name: jnlp
    label: jnlp
spec:
  securityContext:
    runAsUser: 0
    privileged: true
  containers:
  - name: img
    image: r.j3ss.co/img:v0.5.7
    imagePullPolicy: Always
    command:
    - cat
    tty: true
  - name: git
    image: alpine/git:1.0.4
    command:
    - cat
    tty: true
  - name: charttester
    image: charttester:1.0.0
    command:
    - cat
    tty: true

Is there any updates on this issue? I also faced with it!
++++


resize.cpp:3785: error: (-215:Assertion failed)

It's seemed running smoothly, until this. (just follow "Get Start")
OpenCV(4.0.0) /io/opencv/modules/imgproc/src/resize.cpp:3785: error: (-215:Assertion failed) !dsize.empty() || (inv_scale_x > 0 && inv_scale_y > 0) in function 'resize'
Ubuntu 18.04, how can I solve it?
Thanks.

I knew it, juest switch to python3.

how do you solve it?i have the same problem

It's seemed running smoothly, until this. (just follow "Get Start")
OpenCV(4.0.0) /io/opencv/modules/imgproc/src/resize.cpp:3785: error: (-215:Assertion failed) !dsize.empty() || (inv_scale_x > 0 && inv_scale_y > 0) in function 'resize'
Ubuntu 18.04, how can I solve it?
Thanks.

Could you tell me if it's smooth to use style2paints on Ubuntu18.04?
I mean to run on a Ubuntu 18.04 Server.


⚠ AsianCinema has degraded performance

In 9c52d1f, AsianCinema ($ACM) experienced degraded performance:

HTTP code: 200
Response time: 1288 ms

Resolved: AsianCinema performance has improved in b211fee.


Remove javaharness references in bazelproject file

And a couple of other files too

Thanks, I have https://github.com/PolymerLabs/arcs/pull/4411 but it included removing a BUILD file and it hit some Copybara sadness IIRC and I haven't had a chance to look further yet. Fine to go ahead with yours. I may have fixed the bazelproject part already, but you'll see if so on merge.


Unifying Cython code for different axes with swapaxes

Following up on #86, I would like to use np.swapaxes (np.transpose more generally) so that we can remove the redundant Cython code for different axes.
For a function like bottleneck.nansum, this should be straightforward. I can insert a call to np.swapaxes in nansum_selector to always switch the array so it should summed over axis=-1, and then none of the API needs to change.
But the design is a little trickier for functions that don't remove a dimension (e.g., all the moving window functions). For these functions, we'll need to call np.swapaxes after performing the operation along a fixed axis. To maintain the current selector API, I could either:

Write a bunch of wrapper functions (similar to move_nansum_slow_axis0) for each combination of axis, dtype and ndim, and include them in the Cython source code. This is not ideal, although I suppose it is still an improvement over what currently exists.
Generate the wrapper function dynamically, when calling move_nansum_selector. This would be my preferred approach if I were writing pure Python, but I'm not sure how Cython changes things.

Thoughts?

The rewrite of bottleneck took care of this issue.


Unable to edit elements or data in reference data types

Describe the bug
Unable edit elements or data in reference data types

I recently restarted work on this after seeing that mdm-core now should have the endpoints required to correctly manage all objects related to Reference Data Models - elements, types, data etc.
However, I have put it on hold again because I think the user design/experience should be determined first. For example, my original intent was to add new elements/types/data inline in the table views of each tab, but think perhaps that it would be better to use a wizard style interface, like those used for the Data Model/Class/Element/Type domains. This needs to be discussed though to agree on the correct approach.
Also, there are also YouTrack tickets also related to this area, and having the same issue listed in two different issue trackers is going to cause confusion. See https://metadatacatalogue.myjetbrains.com/youtrack/issue/MC-9486.
When this is revisited:

Decide which take priority - the YouTrack ticket or this GitHub issue.
Plan out the UX for managing/editing Reference Data Models.

@jamesrwelch I remember working on this originally last year but pausing it for the reasons I've listed above. I feel that Reference Data Models should probably be setup as a new set of issues to properly plan/design their usage in the UI. Then this particular issue would be closed as it would be obsolete.
Would you agree?

@pjmonks Yes, completely agree - good plan 👍


problem in test_cudnn

We downloaded the current tvm today, and meet a problem in test_cudnn.
llvm-version:5.0.0
cuda-version:9
cudnn-version:7
llvm-5.0.0.src/lib/IR/Type.cpp:619: static llvm::PointerType* llvm::PointerType::get(llvm::Type*, unsigned int): Assertion `isValidElementType(EltTy) && "Invalid type for pointer element!"' failed.
Aborted (core dumped)
Does this problem caused by llvm-5.0?

We changed the llvm to 4.0.1, but the problem was not solved.
llvm-4.0.1.src/lib/IR/Type.cpp:619: static llvm::PointerType* llvm::PointerType::get(llvm::Type*, unsigned int): Assertion `isValidElementType(EltTy) && "Invalid type for pointer element!"' failed.
Aborted (core dumped)
@tqchen


Create a new pull request by comparing

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.1%) to 94.889%

Files with Coverage Reduction
New Missed Lines
%

js/src/dom/event-handler.js
1
92.27%

Totals

Change from base Build<PHONE_NUMBER>:
-0.1%

Covered Lines:
1980

Relevant Lines:
2039

💛  - Coveralls