make menuconfig 报错了，什么原因呢？

www@PC-MSI-Z690:/www/lede$ make menuconfig
WARNING: Makefile 'package/small-package/shadowsocks-libev/Makefile' has a dependency on 'kmod-nft-tproxy', which does not exist
'ackage/small-package/aria2/Config.in:1:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:2:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:3:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:4:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:5:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:6:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:7:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:8:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:9:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:10:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:11:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:12:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:13:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:14:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:15:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:16:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:17:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:18:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:19:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:20:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:21:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:22:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:23:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:24:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:25:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:26:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:27:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:28:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:29:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:30:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:31:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:32:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:33:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:34:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:35:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:36:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:37:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:38:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:39:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:40:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:41:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:42:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:43:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:44:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:45:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:46:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:47:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:48:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:49:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:50:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:51:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:52:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:53:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:54:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:55:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:56:warning: ignoring unsupported character '
package/small-package/aria2/Config.in:57:warning: ignoring unsupported character '\'
'ackage/small-package/aria2/Config.in:57:warning: ignoring unsupported character '
package/small-package/aria2/Config.in:58: syntax error
package/small-package/aria2/Config.in:57: invalid statement
'ackage/small-package/aria2/Config.in:58:warning: ignoring unsupported character '
package/small-package/aria2/Config.in:58: invalid statement
'ackage/small-package/aria2/Config.in:59:warning: ignoring unsupported character '
package/small-package/aria2/Config.in:59: invalid statement
'ackage/small-package/aria2/Config.in:60:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:61:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:62:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:63:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:64:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:65:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:66:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:67:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:68:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:69:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:70:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:71:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:72:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:73:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:74:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:75:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:76:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:77:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:78:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:79:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:80:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:81:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:82:warning: ignoring unsupported character '
'ackage/small-package/aria2/Config.in:83:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:1:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:2:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:3:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:4:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:5:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:6:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:8:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:9:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:10:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:11:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:12:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:13:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:14:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:15:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:16:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:17:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:18:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:19:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:20:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:21:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:22:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:23:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:24:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:25:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:26:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:27:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:28:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:29:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:30:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:31:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:32:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:33:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:34:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:35:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:36:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:37:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:38:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:39:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:40:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:41:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:42:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:43:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:44:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:45:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:46:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:47:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:48:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:49:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:50:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:51:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:52:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:53:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:54:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:55:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:56:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:57:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:58:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:59:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:60:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:61:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:62:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:63:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:64:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:65:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:66:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:67:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:68:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:69:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:70:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:71:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:72:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:73:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:74:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:75:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:76:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:77:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:78:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:79:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:80:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:81:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:82:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:83:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:84:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:85:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:86:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:87:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:88:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:89:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:90:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:91:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:92:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:93:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:94:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:95:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:96:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:97:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:98:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:99:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:100:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:101:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:102:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:103:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:104:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:105:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:106:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:107:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:108:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:109:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:110:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:111:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:112:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:113:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:114:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:115:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:116:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:117:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:118:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:119:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:120:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:121:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:122:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:123:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:124:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:125:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:126:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:127:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:128:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:129:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:130:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:131:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:132:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:133:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:134:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:135:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:136:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:137:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:138:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:139:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:140:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:141:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:142:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:143:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:144:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:145:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:146:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:147:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:148:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:149:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:150:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:151:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:152:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:153:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:154:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:155:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:156:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:157:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:158:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:159:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:160:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:161:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:162:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:163:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:164:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:165:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:166:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:167:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:168:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:169:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:170:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:171:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:172:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:173:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:174:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:175:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:176:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:177:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:178:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:179:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:180:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:181:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:182:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:183:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:184:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:185:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:186:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:187:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:188:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:189:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:190:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:191:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:192:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:193:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:194:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:195:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:196:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:197:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:198:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:199:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:200:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:201:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:202:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:203:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:204:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:205:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:206:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:207:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:208:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:209:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:210:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:211:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:212:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:213:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:214:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:215:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:216:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:217:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:218:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:219:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:220:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:221:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:222:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:223:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:224:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:225:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:226:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:227:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:228:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:229:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:230:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:231:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:232:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:233:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:234:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:235:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:236:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:237:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:238:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:239:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:240:warning: ignoring unsupported character '
'ackage/small-package/oscam/Config.in:241:warning: ignoring unsupported character '
make: *** [/www/lede/include/toplevel.mk:134: menuconfig] Error 1

删除报错的两个文件夹就正常了
rm -rf package/small-package/aria2
rm -rf package/small-package/oscam/


ADUser: Add property ThumbnailPhoto

Pull Request (PR) description

Changes to ADUser

Add new property ThumbnailPhoto and read-only property ThumbnailPhotoHash (issue #44).

This is based on the work that @iainbrighton started in VirtualEngineLab. Big thanks for the work he put in, see issue #44 for more information.
This Pull Request (PR) fixes the following issues

Fixes #44

Task list

[x] Added an entry under the Unreleased section in the CHANGELOG.md.
Entry should say what was changed, and how that affects users (if applicable).
[ ] Resource documentation added/updated in resource directory README.md.
[x] Resource parameter descriptions added/updated in schema.mof
and comment-based help.
[x] Comment-based help added/updated.
[x] Conceptual help topic added/updated (cultureFolder\about_ResourceName.help.txt).
[x] Localization strings added/updated in all localization files as appropriate.
[ ] Examples appropriately added/updated.
[x] Unit tests added/updated. See DSC Resource Testing Guidelines.
[ ] Integration tests added/updated (where possible). See DSC Resource Testing Guidelines.
[x] New/changed code adheres to DSC Resource Style Guidelines and Best Practices.

This change is

Codecov Report

Merging #468 into dev will increase coverage by <1%.
The diff coverage is 100%.

@@         Coverage Diff          @@
##            dev   #468    +/-   ##
====================================
+ Coverage    98%    98%   +<1%
====================================
  Files        20     20
  Lines      2823   2873    +50
  Branches     10     10
====================================
+ Hits       2774   2824    +50
  Misses       39     39
  Partials     10     10


[BUG] 9.22.1 crashes MSBUILD on netcoreapp3.0 preview8

Describe the bug

After updating from 9.21.1 to 9.22.1 for a netcoreapp3.0 preview 8 project, MSBUILD crashes on build (Visual Studio 2019).
Steps To Reproduce

Create an empty ASP.NET CORE 3.0 project.
Add a nuget reference to ReactiveUI 9.21.1
Build - All OK.
Update nuget reference to ReactiveUI 9.22.1
Build - Getting a MSBUILD error.

.csproj file:
<Project Sdk="Microsoft.NET.Sdk.Web">   <PropertyGroup>     <TargetFramework>netcoreapp3.0</TargetFramework>   </PropertyGroup> 	<ItemGroup>     <PackageReference Include="ReactiveUI" Version="9.22.1"> 		</PackageReference>   </ItemGroup> </Project>
Expected behavior

Project should build successfully.
Error messages

Getting a modal dialog error message:
MSBuild.exe - "A new guard page for the stack cannot be created".
The output message:
------ Rebuild All started: Project: Web, Configuration: Debug Any CPU ------ You are using a preview version of .NET Core. See: https://aka.ms/dotnet-core-preview
MSBUILD : error MSB4166: Child node "2" exited prematurely. Shutting down. Diagnostic information may be found in files in "C:\Users\[USERNAME]\AppData\Local\Temp\" and will be named MSBuild_*.failure.txt. This location can be changed by setting the MSBUILDDEBUGPATH environment variable to a different directory.
========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========
MSBuild_*.failure.txt contains:

UNHANDLED EXCEPTIONS FROM PROCESS 15672:
=====================
2019-08-28 15:52:22
System.IO.IOException: Pipe is broken.
at System.IO.Pipes.PipeStream.CheckWriteOperations()
at System.IO.Pipes.PipeStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at Microsoft.Build.BackEnd.NodeEndpointOutOfProcBase.RunReadLoop(Stream localReadPipe, Stream localWritePipe, ConcurrentQueue`1 localPacketQueue, AutoResetEvent localPacketAvailable, AutoResetEvent localTerminatePacketPump)
===================

Environment

OS:      Windows
Version   10 Pro
Net CORE: 3.0 Preview 8
Visual Studio 2019 16.3.0 Preview 2 (tried 16.2.3 too)

Additional context

Workaround: revert to 9.21.1

I will most likely introduce a splat.wpf and splat.winforms package now

Thanks for the decent investigation into the issue. It makes providing a solution much easier.

I'm using it in a asp net core 3.0 project - it's a netcoreapp3.0, but, of course, has no dependencies on windows - so I guess that's why msbuild crashes.
I've been experimenting with using ReactiveUI with Blazor and it has been working pretty nicely (until this bump).
Thank you for all the great work!

This will be solved as part of the 9.0 release of Splat.


Added German Local ghost

Got some code for us? Awesome 🎊!
Please include a description of your change & check your PR against this list, thanks!

[ ] There's a clear use-case for this code change, explained below
[ ] Commit message has a short title & references relevant issues
[ ] The build will pass (run yarn test:all and yarn lint)

We appreciate your contribution!

@andinf2310 For future contributors, please share why you made this change. Thanks


Debian packages for 17.5 are missing

Where is msodbcsql17_<IP_ADDRESS>-1_amd64.deb?
https://packages.microsoft.com/debian/9/prod/pool/main/m/msodbcsql17/
Index of /debian/9/prod/pool/main/m/msodbcsql17/
../
msodbcsql17_<IP_ADDRESS>-1_amd64.deb                   12-Mar-2018 19:32             3835428
msodbcsql17_<IP_ADDRESS>-1_amd64.deb                   03-Apr-2018 23:38             3861804
msodbcsql17_<IP_ADDRESS>-1_amd64.deb                   11-Jul-2018 05:45             4068616
msodbcsql17_<IP_ADDRESS>-1_amd64.deb                   26-Feb-2019 04:46              749208
msodbcsql17_<IP_ADDRESS>-1_amd64.deb                   02-Aug-2019 17:49              796686
msodbcsql17_<IP_ADDRESS>-1_amd64.deb                   04-Oct-2019 01:13              797708

Yes @esetnik this morning we realized the microsoft repo removed the 17.5 package, for some reason. We have informed the people in charge. Please check again later

@esetnik The package for 17.5 is now available.


Can we stream output from crew.kickoff() to chainlit app?

Hello,
I am trying to build an interactive crew using chainlit.
Is there any way to stream the output from crew.kickoff() onto chainlit?
Also to take human input through the chainlit app for taking next steps etc.

Additionally, how to visualize/show intermediate steps where agents share some info.

would also love to see this is action

Possibly useful: How to Use Human as Tool with crewai Agents on chainlit UI

Found a more recent article - https://krishankantsinghal.medium.com/supercharge-your-conversational-ai-integrating-chainlit-and-crewai-for-powerful-interactions-ca8a50ec1851
If there are other articles which could be used to stream the intermediate steps in crewai onto chainlit, please comment below.

There's a GitHub repo that has attempted this:
https://github.com/AbubakrChan/crewai-business-product-launch
The approach here is to save sys.stdout as a variable and then perform regex to make the stream not look so raw. But I've found this to not be very aesthetically pleasing.
Does the medium article have a GitHub repo link?


please append to log file, not just new every time

recently was arguing with 6.16 and had to exit to recover only to be treated to a desktop FULL of error logs... all of the same thing, all already reported, so I had to go through and (find) delete all the error files...
Please just Open/Append to the log file... not a new one each time, either that or put them in a folder, not the desktop.

They are in the working directory. If you put the meerk40t.exe in a folder, all the errors would end up there too.

Hum... acceptable, not ideal.  Would prefer a setting (Windows, settings) that specifically said where to put stuff.
Will close it though

Suggest an upgrade that puts a setting in "Windows, settings" of where all MeerK40T errors and files go by default.

The problem with that is when I hit an error, I try to avoid touching anything else. Because if it hits a bug in that routine it's a problem on top of a problem.


Add a static error for @anonymous classes that have a @JS annotation with a non-empty string

For example,
@JS('Foo')
@anonymous
class FooClass {}

These JS interop classes are likely not meant to be annotated as @anonymous since they're bound to a JS class. They should instead just be a @JS class.
Original context here.

Another potential static error for @anonymous on the web, should we be allowed to use @anonymous without @JS?


Project side bar

a lot of stuff.... sorry for no comprehensive notes

the tests are going to fail (which i should fix soon), but I just want to merge for now.

Coverage increased (+1.2%) to 47.638% when pulling 43a126b5e878fef027aa71557b51491cbc4777f4 on project-side-bar into feb4fff3dffd1cb92bbfef6e35252d93566de0df on dev.


Inject namedWriteableRegistry during ser/deser of SearchMonitorAction

Description of changes:
When calling the transport action from another plugin with a separate classloader, the casting will fail and it will hit recreateObject fn to perform serialization/deserialization of the request. During deserialization, without converting the StreamInput to NamedWriteableAwareStreamInput, this was causing failures. This PR adds the NamedWriteableRegistry as part of the transport action constructor and in execute() in order to resolve this.
Stack trace seen (example coming from skills plugin which is calling this transport action via common-utils AlertingPluginInterface.searchMonitors():
»  java.lang.UnsupportedOperationException: can't read named writeable from StreamInput
»       at org.opensearch.core.common.io.stream.StreamInput.readNamedWriteable(StreamInput.java:1122) ~[opensearch-core-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»       at org.opensearch.core.common.io.stream.StreamInput.readOptionalNamedWriteable(StreamInput.java:1149) ~[opensearch-core-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»       at org.opensearch.search.builder.SearchSourceBuilder.<init>(SearchSourceBuilder.java:240) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»       at org.opensearch.core.common.io.stream.StreamInput.readOptionalWriteable(StreamInput.java:974) ~[opensearch-core-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»       at org.opensearch.action.search.SearchRequest.<init>(SearchRequest.java:232) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»       at org.opensearch.commons.alerting.action.SearchMonitorRequest.<init>(SearchMonitorRequest.kt:27) ~[?:?]
»       at org.opensearch.alerting.transport.TransportSearchMonitorAction.doExecute(TransportSearchMonitorAction.kt:58) ~[?:?]

CheckList:

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Note this codepath is only triggered at integration level, and tests are written in skills plugin to test this thoroughly, since it requires executing this action from a separate plugin with a separate classloader.

Tests passing on linux and windows. Security test failures are unrelated.

have you tested this by building common utils jar and setting up local server with all the necessary plugins?

have you tested this by building common utils jar and setting up local server with all the necessary plugins?

Yes


[Build] Drop Python 3.8

Python 3.8 is approaching EoL, and is starting to lose support from our dependencies. Bump our minimum supported version to Python 3.9.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 56.77%. Comparing base (15ff21d) to head (598a4f6).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #873      +/-   ##
==========================================
- Coverage   57.02%   56.77%   -0.26%
==========================================
  Files          63       63
  Lines        4575     4571       -4
==========================================
- Hits         2609     2595      -14
- Misses       1966     1976      +10

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Set Default Language Html5 Client vi language

After I change the overrideLocale to vi_VN. And yes, It works correctly. But when I change to other language in Setting UI. Then the page is reloaded rapidly and never stop reload. Anyone have the same problem? My BBB version is 2.2.0-rc-3
-->
Hi @all

Setup manual for Client BBB change language: https://support.bigbluebutton.org/hc/en-us/articles/1500005217181-Access-the-Options-Menu#:~:text=Application menu By default the Application menu is,You can also change the application Font Size.

Set overrideLocale: null to overrideLocale: de to hard set the default locale for the users.
You may also want to check the background info from https://github.com/bigbluebutton/bigbluebutton/issues/6757

The setting above is located in /usr/share/meteor/bundle/programs/server/assets/app/config/settings.yml on a BBB 2.2.x server. Make sure to restart BigBlueButton after you have saved your changes to the config file -->  https://docs.bigbluebutton.org/administration/bbb-conf/#Check

$ bbb-conf
BigBlueButton Configuration Utility - Version 2.5.2
bbb-conf [options]
Configuration:
--version                        Display BigBlueButton version (packages)
--setip <IP/hostname>            Set IP/hostname for BigBlueButton
--setsecret              Change the shared secret in /etc/bigbluebutton/bbb-web.properties
Monitoring:
--check                          Check configuration files and processes for problems
--debug                          Scan the log files for error messages
--watch                          Scan the log files for error messages every 2 seconds
--network                        View network connections on 80, 443 and 1935 by IP address. 1935 is deprecated. You will need to modify bbb-conf if you have custom ports.
--secret                         View the URL and shared secret for the server
--lti                            View the URL and secret for LTI (if installed)
Administration:
--restart                        Restart BigBlueButton
--stop                           Stop BigBlueButton
--start                          Start BigBlueButton
--clean                          Restart and clean all log files
--status                         Display running status of components
--zip                            Zip up log files for reporting an error

Server Customization:
https://docs.bigbluebutton.org/administration/customize/

To generate thumbnails you can use the following shell snippet:
#!/bin/bash
FULL="/usr/share/meteor/bundle/programs/web.browser/app/resources/images/virtual-backgrounds"
THUMB="${FULL}/thumbnails"

cd "$FULL"
for pic in *.jpg; do
    convert "$pic" -resize 50x50^ -gravity Center -extent 50x50 "${THUMB}/${pic}"
done

Reference them in the configuration file /etc/bigbluebutton/bbb-html5.yml:
public:
virtualBackgrounds:
fileNames:
- image1.jpg
- image2.jpg
- image3.jpg
Background images should not be too large as clients have to download them. You can optimize them using the jpegoptim command which is available as an Ubuntu package.
and then restart BigBlueButton
$ bbb-conf --restart

Frontends
Remove the API demos
If you have earlier installed the API demos for testing (which makes it possible for anyone to use your BigBlueButton server without authentication) and want to now remove them, enter the command:
$ sudo apt-get purge bbb-demo
Modify the default landing page
The default HTML landing page is located in
:  /var/www/bigbluebutton-default/assets/index.html
Change this page to create your own landing page (and keep a back-up copy of it as it will be overwritten during package updates to bbb-conf).
Use the Greenlight front-end
BigBlueButton comes with Greenlight, a front-end application written in Ruby on Rails that makes it easy for users to create meetings, invite others, start meetings, and manage recordings.
greenlight-start
For more information see Installing Greenlight.

How to customizing GUI of bigbluebutton 2.7.11 ?
--> https://docs.bigbluebutton.org/administration/customize/
https://docs.bigbluebutton.org/administration/customize/#configuration-of-global-settings

https://docs.bigbluebutton.org/greenlight/v3/install/
https://docs.bigbluebutton.org/greenlight/v3/customize/


Make fixes to impropers discussed in 147 (proper six-fold handling).

Now handles impropers via separate registry from propers, as discussed in #147, and correctly handles the six-fold symmetrization of impropers.
Supersedes #148 but brings in the sanity checking introduced there.

Thanks!

(Passes tests locally, so I'm cautiously optimistic.)

Much better, thanks!


Need information about API methods

Hi There,
Interested in the possibility of adding api methods:

The method by which we can get the information entered by the agent on the onTyping event
A method in which we can programmatically insert text content into the typing field in CCP

Is it possible to implement these methods with the current functionality, or do you need to write a custom solution?
Thank you for your support!
@jagadeeshaby @jolim24601 @kvs6rj
Best Regards
Ruslan Andreichikov

For the onTyping event handler, could you post the feature request in ChatJS repository?
bbolek's suggestion should work for the second question.


show TCP Ping latency (ms)

It would be sweet if it can also show the latency (ms) of TCP ping. it can only show "open" or "close" for tcp ping funciton.

That is an excellent idea and should be fairly easy to implement.

@betterdoitnow The feature was added to version 1.1.15.

@R-Smith great!!!!!!! thanks for the work!

could we have the official 1.1.15? excited :DD
On Wed, Jul 11, 2018 at 7:15 AM, betterdoitnow<EMAIL_ADDRESS>wrote:

@R-Smith https://github.com/R-Smith great!!!!!!! thanks for the work!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/R-Smith/vmPing/issues/8#issuecomment-403995016, or mute
the thread
https://github.com/notifications/unsubscribe-auth/Afr7Dhi8NMw2JBVIVHiAcqAk2bkkt9juks5uFTV7gaJpZM4TDrRW
.

--
Thanks,
wndnCG
<EMAIL_ADDRESS>| OFC Microsoft Email
“This message contains confidential information and is intended only for
the named addressee. If you are not the named addressee you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately by e-mail if you have received this e-mail by mistake and
delete this e-mail from your system. E-mail transmission cannot be
guaranteed to be secure or error-free as information could be intercepted,
corrupted, lost, destroyed, arrive late or incomplete, or contain viruses.
The sender therefore does not accept liability for any errors or omissions
in the contents of this message, which arise as a result of e-mail
transmission. If verification is required please request a hard-copy
version.”


Token icon links on Rinkeby seem to be broken

The URLs of the icons of the following tokens on Rinkeby seem to be broken. I get an access denied from AWS

https://safe-relay.rinkeby.gnosis.pm/api/v1/tokens/?search=LOVE
https://safe-relay.rinkeby.gnosis.pm/api/v1/tokens/?search=DAIc

Hi @tschubotz ,
It's fixed now, I configured icons for both tokens


Using tiles with alpha channel causes flicker

Current behavior
When using image tiles with an alpha channel, and the map is zoomed in or out (currently using whole steps), each tile first render a lot darker than it should be. I believe this is because it is first rendered without the alpha channel. This darker version lasts for a fraction of second until the tile is properly rendered. Because this transition is abrupt, it causes a flickering effect.
Expected behavior
No flickering. I expect it to work the same with image tiles containing an alpha channel as it does with those without.
Environment
Leaflet 1.6.0

Tested using Safari 14.0.1, Chrome 87.0.4280.141, and Firefox 84.0.2

On MacOS 10.15.7
Additional context
If the requests are made directly to the image file, this behavior happens in unison across all rendered tiles and isn't very jarring. However, my application generates the tiles on the fly which causes the effects to happen at slightly different times for each tile. This makes for a very unpleasant rendering. To replicate this, I redirect requests to a PHP script.
Minimal example reproducing the issue
Unfortunately, due to needing tiles with varying opacity and sending requests to a PHP script, I am unable to use JS Fiddle. I have decided to create a repository (https://github.com/Thoughtyness/Flicker_Example) that contains everything needed to reproduce this. There are two folders "tiles_with_alpha" and "tiles_without_alpha" which can be used to see that the flickering only occurs when alpha tiles are used. I also left a commented line in index.php which will bypass the PHP script. A copy the code is below:
index.php:
<!DOCTYPE html>

    <title>Test Flicker</title>

response.php:
<?php
  $z=$_GET["z"];
  $x=$_GET["x"];
  $y=$_GET["y"];

  header("Content-type: image/png");
  readfile("https://localhost/flicker_test/tiles_with_alpha/".$z."-".$x."-".$y.".png");
?>

For me it only happens on Chrome on mobile (fine on Firefox mobile). Leaflet 1.7.1 is fine but the problem starts at 1.8.0.
Flickering
Not flickering


Enter button + spacebar does not change colour

Only for completely black themes the enter button always defaults to blue. And the spacebar "outline" and lettering never appear.
This has been going on for years but I finally reinstalled this app and realized it's still not fixed.
Screenshot attached.
Android 13
Latest gboard as of this date.
Samsung 22
Canada

This has nothing to do with the manager, that's a theme issue then.


Need maps deleted from database

Details:
With the new map editor these maps need to be fully removed as there is no delete option present.
19
21
27
28
29
31
32
33
34
37
38
39
40
41
42
43
44
45
46
47
48
49
50
Expected result:
These maps made with old editor not to be present
Actual result:
These maps are malformed and need to be removed

these 8 maps won't be able to be deleted because games were started with them:
+-----------+-------+
| count(id) | mapId |
+-----------+-------+
|         1 |    19 |
|         1 |    21 |
|         1 |    38 |
|         1 |    39 |
|         3 |    41 |
|         1 |    45 |
|         1 |    46 |
|         1 |    50 |
+-----------+-------+
8 rows in set (0.001 sec)

I removed the other ones.


Support implicit numeric conversions in filter expressions.

Javascript is not strongly typed, so it implicitly converts strings to numbers when comparing across data types:
# node
> "9" < 73
true
> 9 < "73"
true
> "9" < "73"
false

This adds functionality via a configuration option (IMPLICIT_NUMERIC_CONVERSIONS) to enable the java implementation of JsonPath to mimic this behavior. This behavior was already in place for equals, but this extends it to the other comparators.

@kecaps Any reason this was never merged in? I'm running into a similar issue where a provider sends JSON values as all String values and I need to perform numeric filtering in Java.


Tutorial Brach Merge.

Given the number of commits and files changed, could you fetch upstream/master and rebase your topic branch on it? It would make it easier for us to review changes.

Hi @LarryOsterman.   This PR appears to be related to a recent training rather than a workstream that you'd like to merge in.   I'm going to close this out; please feel free to reopen if you'd like to continue working on this or would like to discuss further.


[SE-0280] Update the phrasing of the 'API resilience' section

As mentioned by Jordan and Slava in the review thread, switching between cases and static properties/methods can break resilience due to ABI and mangling differences. So, re-phrase the "API resilience" section to reflect that.

cc @rjmccall


[zh-cn]  Update create-cluster-kubeadm.md

/retitle [zh-cn]  update tasks/administer-cluster/cluster-upgrade.md

/retitle [zh-cn] update tasks/administer-cluster/cluster-upgrade.md

/retest

/ok-to-test

@yanrongshi you need to perform a rebase.


113 aad 24 title outdated

Updated 2.4 Last Policy-Sub-bullet Title
🗣 Description

Changed sub-bullet title to "MFA SHALL be required for all users"
💭 Motivation and context
The last sub-bullet needed to reflect the change in the Policy title.

🧪 Testing
Ran the unit test for aad,the Invoke-Scuba for aad and checked the report.

✅ Pre-approval checklist

[ ] This PR has an informative and human-readable title.
[ ] Changes are limited to a single goal - eschew scope creep!
[ ] All future TODOs are captured in issues, which are referenced
in code comments.
[ ] All relevant type-of-change labels have been added.
[ ] I have read the CONTRIBUTING document.
[ ] These code changes follow cisagov code standards.
[ ] All relevant repo and/or project documentation has been updated
to reflect the changes in this PR.
[ ] Tests have been added and/or modified to cover the changes in this PR.
[ ] All new and existing tests pass.

✅ Pre-merge checklist

[ ] Revert dependencies to default branches.
[ ] Finalize version.

✅ Post-merge checklist

[ ] Add a tag or create a release.

@ssatyapal123, could you clarify what needed to be changed? Last sub-bullet, as in "SMS or Voice as the MFA method SHALL NOT be used"? If so, I don't see any changes made to that bullet. The only bullet I see a change for is "MFA SHALL be required for all users ", which now has a trailing space. Which makes me think I'm missing something.
Also, this branch has merge conflicts that need to be resolved.

@adhilto sorry for the confusion. In the report output, the last bullet was "Phishing-resistant MFA SHALL be required for all users", which is a repeat of the second bullet:

The change was to make the last sub-bullet "MFA SHALL be required for all users":

I couldn't get the code to work for that exact text. I noticed that any change to that specific text worked so I added a trailing space, thinking it would not harm the report output. Realize that I should have checked why that text wouldn't work but figured this issue should be a quick fix and went ahead. I would  be grateful if anyone knows where the conflict is.

@adhilto Sorry forgot to address your comment about merge conflicts. I think this is due to the reopening of the issue and the fact that I made the latest updates (to the sub-title vs the title) against the main branch but I'm looking into that now. Thank you, shanti

What issue do you run into when you try to remove the trailing space? I tried it just now and it worked. Would be good to remove it, but either way you have my approval.

@adhilto it was failing the unit Rego test for baseline policy 2.4 for some reason. But after checking it again just now, it is passing without any alterations to the text. So, I updated the code to remove the trailing whitespace. Thanks for your review, shanti


[Judgement] Remaining Submissions

[x] Endpoint to return remaining submissions info
[ ] Frontend display of remaining submissions info

Relevant endpoint added in 95da7e246f8daee2d6be557bd2803dcb8874d597

@sauterl is the current endpoint sufficient or would you need additional information?

@lucaro yes, this is sufficient, as far as my notes of our last discussion are correct, one has to sum up over all validators to get the remaining submissions for a particular task.
I will later this day push the frontend-updates and hopefully close this issue


Split spdx-tools to multiple modules

The set of dependencies for spdx-tools is huge, and it does not seem to play well when used as a library:
\--- org.spdx:spdx-tools:2.1.16
     +--- org.apache.thrift:libthrift:0.12.0
     |    +--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    +--- org.apache.httpcomponents:httpclient:4.5.6
     |    |    +--- org.apache.httpcomponents:httpcore:4.4.10
     |    |    +--- commons-logging:commons-logging:1.2
     |    |    \--- commons-codec:commons-codec:1.10 -> 1.11
     |    \--- org.apache.httpcomponents:httpcore:4.4.1 -> 4.4.10
     +--- com.fasterxml.jackson.core:jackson-databind:2.9.8
     |    +--- com.fasterxml.jackson.core:jackson-annotations:2.9.0
     |    \--- com.fasterxml.jackson.core:jackson-core:2.9.8
     +--- org.apache.commons:commons-compress:1.18
     +--- org.apache.jena:apache-jena-libs:3.10.0
     |    +--- org.apache.jena:jena-tdb:3.10.0
     |    |    +--- org.apache.jena:jena-arq:3.10.0
     |    |    |    +--- org.apache.jena:jena-core:3.10.0
     |    |    |    |    +--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    |    |    |    +--- org.apache.jena:jena-iri:3.10.0
     |    |    |    |    |    \--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    |    |    |    +--- commons-cli:commons-cli:1.4
     |    |    |    |    +--- commons-codec:commons-codec:1.11
     |    |    |    |    \--- org.apache.jena:jena-base:3.10.0
     |    |    |    |         +--- org.apache.jena:jena-shaded-guava:3.10.0
     |    |    |    |         |    \--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    |    |    |         +--- org.apache.commons:commons-csv:1.5
     |    |    |    |         +--- commons-io:commons-io:2.6
     |    |    |    |         +--- org.apache.commons:commons-lang3:3.4
     |    |    |    |         +--- org.apache.commons:commons-compress:1.17 -> 1.18
     |    |    |    |         +--- com.github.andrewoma.dexx:collection:0.7
     |    |    |    |         \--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    |    |    +--- org.apache.jena:jena-shaded-guava:3.10.0 (*)
     |    |    |    +--- org.apache.httpcomponents:httpclient:4.5.5 -> 4.5.6 (*)
     |    |    |    +--- com.github.jsonld-java:jsonld-java:0.12.1
     |    |    |    |    +--- com.fasterxml.jackson.core:jackson-core:2.9.6 -> 2.9.8
     |    |    |    |    +--- com.fasterxml.jackson.core:jackson-databind:2.9.6 -> 2.9.8 (*)
     |    |    |    |    \--- commons-io:commons-io:2.6
     |    |    |    +--- org.apache.httpcomponents:httpclient-cache:4.5.5
     |    |    |    |    \--- org.apache.httpcomponents:httpclient:4.5.5 -> 4.5.6 (*)
     |    |    |    +--- org.apache.thrift:libthrift:0.10.0 -> 0.12.0 (*)
     |    |    |    +--- org.slf4j:jcl-over-slf4j:1.7.25
     |    |    |    |    \--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    |    |    +--- org.apache.commons:commons-lang3:3.4
     |    |    |    \--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    |    \--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    +--- org.apache.jena:jena-tdb2:3.10.0
     |    |    +--- org.apache.jena:jena-dboe-trans-data:3.10.0
     |    |    |    +--- org.apache.jena:jena-dboe-transaction:3.10.0
     |    |    |    |    +--- org.apache.jena:jena-dboe-base:3.10.0
     |    |    |    |    |    +--- org.apache.jena:jena-arq:3.10.0 (*)
     |    |    |    |    |    \--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    |    |    |    \--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    |    |    +--- org.apache.jena:jena-dboe-index:3.10.0
     |    |    |    |    +--- org.apache.jena:jena-dboe-base:3.10.0 (*)
     |    |    |    |    \--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    |    |    \--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    |    \--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    +--- org.apache.jena:jena-rdfconnection:3.10.0
     |    |    +--- org.apache.jena:jena-arq:3.10.0 (*)
     |    |    \--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     |    \--- org.slf4j:slf4j-api:1.7.25 -> 1.8.0-alpha2
     +--- org.apache.jena:jena-iri:3.10.0 (*)
     +--- org.antlr:antlr:3.4
     |    +--- org.antlr:antlr-runtime:3.4
     |    |    +--- org.antlr:stringtemplate:3.2.1
     |    |    |    \--- antlr:antlr:2.7.7
     |    |    \--- antlr:antlr:2.7.7
     |    \--- org.antlr:ST4:4.0.4
     |         \--- org.antlr:antlr-runtime:3.3 -> 3.4 (*)
     +--- org.apache.commons:commons-lang3:3.1 -> 3.4
     +--- org.apache.poi:poi:3.17
     |    +--- commons-codec:commons-codec:1.10 -> 1.11
     |    \--- org.apache.commons:commons-collections4:4.1
     +--- org.apache.poi:poi-ooxml:3.17
     |    +--- org.apache.poi:poi:3.17 (*)
     |    +--- org.apache.poi:poi-ooxml-schemas:3.17
     |    |    \--- org.apache.xmlbeans:xmlbeans:2.6.0
     |    |         \--- stax:stax-api:1.0.1
     |    \--- com.github.virtuald:curvesapi:1.04
     +--- net.sf.opencsv:opencsv:2.3
     +--- nu.validator.htmlparser:htmlparser:1.4
     +--- net.sf.saxon:saxon:8.7
     +--- org.jsoup:jsoup:1.11.3
     +--- com.google.guava:guava:24.1.1-jre
     |    +--- com.google.code.findbugs:jsr305:1.3.9
     |    +--- org.checkerframework:checker-compat-qual:2.0.0
     |    +--- com.google.errorprone:error_prone_annotations:2.1.3
     |    +--- com.google.j2objc:j2objc-annotations:1.1
     |    \--- org.codehaus.mojo:animal-sniffer-annotations:1.14
     +--- com.github.spullara.mustache.java:compiler:0.7.9
     |    \--- com.google.guava:guava:11.0 -> 24.1.1-jre (*)
     +--- org.apache.logging.log4j:log4j-api:2.10.0
     +--- org.apache.logging.log4j:log4j-core:2.10.0
     |    \--- org.apache.logging.log4j:log4j-api:2.10.0
     +--- org.apache.logging.log4j:log4j-slf4j-impl:2.10.0
     |    +--- org.slf4j:slf4j-api:1.8.0-alpha2
     |    \--- org.apache.logging.log4j:log4j-api:2.10.0
     +--- com.google.code.gson:gson:2.8.0
     +--- net.sf.saxon:saxon-dom:8.7
     |    \--- net.sf.saxon:saxon:8.7
     \--- com.github.cliftonlabs:json-simple:2.3.1

Is there a chance to split tools into multiple modules, so bits like "detect license" and "isTextStandardLicense" could be used without pulling too many dependencies?

@vlsi Agree completely.  The dependencies have kind have grown out of control over the last few years.  The biggest set of dependencies are due to the use of Jena.  Pulling out this dependency is a rather large effort.  @yevster and I have discussed some better designs we are considering for 3.0, but this is a ways away.
Focusing in on isTextStandardLicense may make it a bit easier to split out.  I could split some of the license matching code into a separate module, but it would depend on the core SPDX libraries which would have a similar dependency tree.  This is due to a dependency on the class License which pulls in the model code and all the Jena dependencies.  There may be a way to refactor out those dependencies, but an easy solution doesn't come immediately to mind.

@goneall , thanks for the quick response.
I'm implementing a Gradle plugin that would analyze third-party dependencies and summarize licenses to "merged" LICENSE file.
Unfortunately, the above set of dependencies looks like a non-starter :(
I think for now I would use a Git submodule for https://github.com/spdx/license-list-data and precompile it somehow during build stage. The code might be moved later under SPDX, however as you say spdx refactoring would take a while, so I'll play a bit in my own repo.

Glad to hear you're working on a Gradle plugin - sounds like  a great contribution.
I'll post back here if I can think of a way to refactor out the Jena code without having to redesign the way the model code works.
@vlsi Let me know how it goes and if you have any ideas on refactoring out the matching code to make the SPDX tools easier to integrate.

any ideas on refactoring out the matching code

For now I think behind the lines of building bloom filter at build time, then verify actual license against bloom filters and find the closes one.
Alternative option is to scan through license texts and figure out "unique" words, then find those words in "given text" to reduce the set of candidates.
For instance, "apache" appears only in Apache-1.0.json, Apache-1.1.json, Apache-2.0.json, RPSL-1.0.json, AFL-1.2.json, and ECL-2.0.json.
So if input text contains "apache", then we don't need to try other licenses.

Let me know how it goes

I ended up implementing TFIDF-based text classifier.
It takes ~2-3 seconds to parse all the license files and build the prediction model.
Then it takes something like 1 millisecond to calculate the probabilities (1ms for all 500 etalon licenses).
The plugin is not yet ready for the release (e.g. API might need to be revised).
As of now I generate a enum like the below. It looks like "most important SPDX bits" are the list of licenses and the programmatic API to access "etalon texts" (which I package to jar file as text files).
I do not really expect for the Java library to download licenses from the internet. It might be a "nice to have" feature, however I do expect that when I add a "license-management" library, then I expect I can access the list of licenses without dealing with internet connectivity and/or dealing with filesystem (e.g. manually extracting files somewhere).
enum class License private constructor(
    val licenseId: String,
    val licenseName: String,
    detailsUrl: String,
    seeAlso: Array<String>
) {
    `0BSD`(
        "0BSD", "BSD Zero Clause License", "http://spdx.org/licenses/0BSD.json",
        arrayOf("http://landley.net/toybox/license.html")
    ),

    AAL(
        "AAL", "Attribution Assurance License", "http://spdx.org/licenses/AAL.json",
        arrayOf("https://opensource.org/licenses/attribution")
    ),

    ADSL(
        "ADSL", "Amazon Digital Services License", "http://spdx.org/licenses/ADSL.json",
        arrayOf("https://fedoraproject.org/wiki/Licensing/AmazonDigitalServicesLicense")
    ),
...

    val detailsUrl: URI = URI(detailsUrl)

    val seeAlso: List<URI> = seeAlso.map { URI(it) }

    companion object {
        val licenseIds: Map<String, License> = values().associateBy { it.licenseId }

        fun fromLicenseId(licenseId: String) = licenseIds.getValue(licenseId)

        fun fromLicenseIdOrNull(licenseId: String) = licenseIds[licenseId]
    }
}

From library perspective, it probably makes sense to create a set of interfaces and put it to api module, so it does not accidentally depend on something like RDF. Then there could be a module that packages certain build of license texts to a jar file. https://github.com/spdx/license-list-data is fun, however I can't use it as a Maven/Gradle dependency which is sad taking into account that spdx/tools is Java-based.

The interesting bit is "exceptions", and I'm not sure the way to approach that.

The SPDX tools don't deal with exceptions either.  It definitely complicates matching.  Most other license matchers I have seen treat the license + exception as a separate license.

The SPDX tools don't deal with exceptions either.

Now that I'm looking at the code, I realize that the mathcing code will match exception text only if the exact exception is passed into the method.
What it won't do is match a blob of text that includes the license and an exception.

I ended up implementing TFIDF-based text classifier.

Must higher performance than the approach taken here.  The disadvantage is that it doesn't implement the SPDX license matching guidelines.
Based on your performance results, I'm thinking of implementing a TFIDF first pass then do the more compute intensive license matching pass if the score is between certain thresholds.  Above the thresholds, we would just call it a match, below the thresholds we would not both to try to match.

From library perspective, it probably makes sense to create a set of interfaces and put it to api module, so it does not accidentally depend on something like RDF.

I'm starting to experiment with a couple different API approaches.  I would like to solve this for the model in general since the dependency on Jena creates a lot of complexity and overhead.
If we come up with a good interface strategy that allows a migration from the current approach, we can start with the license model to allow the license matching code to be split out.
There is an interface IRdfModel which depends on Jena.  Perhaps this can be changed to a more general storage and retrieval interface.
In any case, it would require a lot of refactoring since the RDF model permeates all the model code implementation.
Just a bit of context - this was originally written as a "pretty printer" for the SPDX RDF files only (hence the heavy RDF dependency).  We added all the other tools and code on top of it.  It is overdue to remove the RDF dependencies.

The disadvantage is that it doesn't implement the SPDX license matching guidelines.

I perfectly get that.
However:

Current state of spdx-tools is "I can't really use it" :(
That is really unfortunate, however I'm on train of migrating a couple of projects (e.g. Apache JMeter (see https://github.com/apache/jmeter/pulls/448), then Apache Calcite and Apache Avatica) to Gralde + automatic license discovery for third-party dependencies.

The licenses used in the wild (as you might know) have random additions/spelling errors. So even if I get "100% matching guideline", then it would fail in 99.42% of the cases.

I guess "proper matching" can be implemented with similar performance provided the engine does not parse the same license file 100'000 times. However I'm still figuring out which features I need (which I would know better as I migrate at least a second project).

the more compute intensive license matching pass if the score is between certain thresholds

Well. We could use TFIDF to arrange the licenses so "more likely matching" would go first, thus "compute-intensive" step does not check all the licenses in case of a match.
Unfortunately, in case of a failure (e.g. license differs from the etalon) it would still have to compare all the texts unless we implement a threshold to stop in cases like "TFIDF says the probability is less than 30%, so we just stop processing and assume it can't match anyway".

I do not really expect for the Java library to download licenses from the internet.

There is a property SPDXParser.OnlyUseLocalLicenses which will use local license files rather than pulling from the internet.
This was an important feature for some of the tools use cases since the listed licenses are updated every 3 months and the tools code updates were much less frequent.
Once thing we could do is automatically generate a compiled class of licenses in the license list data repository.  The code that generates this is the License List Publisher

unless we implement a threshold to stop in cases like "TFIDF says the probability is less than 30%, so we just stop processing and assume it can't match anyway

That's what I am thinking.

Once thing we could do is automatically generate a compiled class of licenses in the license list data repository

Exactly. To use SPDX as a library, the resulting license texts should be available as a JAR artifact. It can be resource-only jar so one can add it as a dependency and get the texts in the Java app.
Note: it does not have to be compiled to *.class format. It just needs to be an artifact with a well-known locations of the files (or a well-known public API to access the contents).

It looks like we've just figured out the first module: an artifact with license texts.
There's a question which format should it use.
For instance:

*.txt.  For instance org/spdx/license/texts/*.txt, org/spdx/license/exceptions/*.txt. The sad point it is hard to keep metadata (e.g. license ids, names, flags) in txt. However we could store it as a class/enum/whatever. The good side is we have both metadata and the licenses, and we don't need JSON/XML parser.
*.json. It is good, however Java has no built-in JSON parser. So we would force everybody to pull a JSON parser which is sad, because there are lots of opinions, and there are CVE's in JSON parsers as well.
*.xml. It looks like old school, however on a plus side, Java has a built-in XML parser.
Other?

Frankly speaking, I'm inclined to 1) which is store texts as *.txt in UTF-8, and bake metadata into class files.
WDYT?

The JSON format has the advantage of the SPDX elements being somewhat standardized.  We will be adding XML in SPDX 3.0, but it isn't solid yet.
There is also the choice of storing the template text, plain text or both.
One other variation on a theme - we could store each license text separately and have the metadata in a single JSON file.
For the next generation of SPDX tools, I think I would also prefer the metadata in class files along with both the template and text files.  The current version parses .json files, but I don't think that should overly influence the decision.
@yevster Any input on this approach?

What would be the preferred method of publication for the .jar file?
We could:

push to Maven Central
store in the license-list-data github repo
Store as a release artifact in the license-list-data github repo
download directly from spdx.org/licenses/licenses.jar
any combination of the above.

Publish to Maven Central. It is enough, as even standalone tools can
download from there.

For as long as RDF is involved, All those Jena dependencies remain in the BOM. I would postpone until after  RDF manipulation becomes optional.

@yevster , can you please clarify?
It is not clear what you mean by "All those Jena dependencies remain in the BOM"

After dwelling on the design for the last couple of days, I decided to start a google document to capture and collaborate on some of the design ideas: SPDX Java Tools (re)Design
Feel free to comment.
@yevster Some of the tools like the license compare don't really depend on RDF and it would be nice to split those out.  Take a look at the doc.  I leveraged a lot from your spdxextra.  The one thing I would like to do different from spdxextra is to allow use of setters in the model object just to make it easier to migrate and use.

@goneall ,  apparently by "design" you mean more than API for licenses.
I appreciate that, however I cannot really comment on that, because I'm not familiar with all the datamodel/usecases for SPDX.
What I can say is:

It would be super-nice to have Java API for accessing licenses which does not require huge dependencies. Neither spdx-tools nor spdxtra solve that problem.
It would be nice to have "license compatibility" API as well. For instance, it is known that you can't bundle GPL-licensed component into Apache-2.0 licensed one. Of course it is not always black and white, however it would be nice to have compatibility matrix implemented by default.
It would be nice to have a normalization algorithm as well. Otherwise projects tend to invent their own normalizations. For instance (Apache2.0 code ahead): https://github.com/jk1/Gradle-License-Report/blob/master/src/main/resources/default-license-normalizer-bundle.json#L41-L44
I see spdxtra and your design document promotes "statics", however it does feel odd. Well, I have not seen SPDX problems, however I routinely tune Java code and analyze memory dumps, so the wording in (re)Design looks odd. Making everything static does not the code becomes faster.
It is not clear if you consider spdx a library or a framework. The answer does not matter for a standalone tool. However if you expect others to integrate SPDX, then you should probably consider building a library.
I would strongly recommend avoid usage of System.getProperty in the "library" code. System.getProperty is fine for standalone, however it is a mess for libraries.

@vlsi Thanks for the review and comments.

apparently by "design" you mean more than API for licenses.

Yes - the SPDX tools includes a datamodel for the entire SPDX document model which includes licenses.

It would be super-nice to have Java API for accessing licenses which does not require huge dependencies.

My proposed solution is to create an API that separates the storage from the model for the license (and the rest of the SPDX model).   We could create a very simple POJO based in memory storage that would require almost no dependencies.

It would be nice to have "license compatibility" API as well.

This would definately be useful, but outside the scope of SPDX since the SPDX community is averse to codifying any interpretation of the license.  There are, however, 2 other project that are codifying license terms that can be used to create a compatibility matrix.  The https://github.com/finos-osr/OSLC-handbook is a project I've worked with to make sure the terms they use are compatible with SPDX.

It would be nice to have a normalization algorithm as well.

I noticed you submitted an issue for this - waiting to see if others in the community support this approach.  If so, I agree it would be beneficial to have one implementation.  I tend not to rely on the name for identification - but rather use the URL or, preferable the text (or even better yet an SPDX ID).

I see spdxtra and your design document promotes "statics", however it does feel odd.

Good point.  For the (re)design, it doesn't have to be static to solve the memory problem.  It just has to be designed in such a way that we do not carry around references to other java object which would create a memory problem for some of the use cases.

It is not clear if you consider spdx a library or a framework.

It started off life as a tool but is now being used as a library.  Hence the need for the (re)design.

I would strongly recommend avoid usage of System.getProperty in the "library" code.

I'll take your advice on this.  I'll add this to the document in "problems".

@goneall, apologies for slow responses. I'm on a customer engagement in the UK.
The reason for having no setters in the model is to prevent the model from being treated as a complete representation of an SPDX document. The idea was to avoid having to load all the triples in RDF-based SPDX into memory. Instead, SpdXtra assumes the document itself is stored in some tuple store (could be on disk), and is queried or manipulated the way you might manipulate a relational table.
This is good for performance and perhaps semi-intuitive when your understanding of SPDX is RDF-centric. However, when a user's view of SPDX is as a collection of documents, I'm not sure this makes sense.

@yevster Thanks for the response.  Makes perfect sense.  I think I may have a way of avoiding the memory intensive nature of the current SPDX tools implementation by having the setters immediately store the information through an interface.  I have a prototype on my local machine I plan to publish to the goneall github repository once I have it in a less embarrassing state.  I hope you don't mind, but I'm borrowing quite a few ideas from spdxextra.  I'll need to make sure I give you credit without implicating you in any of my own bad design choices ;)
My current challenge is how to implement anonymous nodes through an interface and not have it end up duplicating nodes.  I solved this in the current implementation by searching for equivalent nodes (easy in RDF, hard through an interface supporting a more generic storage model).  I may be over-engineering it - perhaps I should just allow the duplication.  Since the nodes are anonymous, the only practical issue I can think of is memory and disk space.
BTW - I still plan on using RDF for my own usage, so I want to make sure that performs well but allow for natural use by non RDF users.

Anonymous nodes are a feature of the RDF model, aren’t they? How would you
have anonymous nodes in a purely SPDX model-centric view of the world?
On Thu, Jun 6, 2019 at 6:36 PM goneall<EMAIL_ADDRESS>wrote:

@yevster https://github.com/yevster Thanks for the response. Makes
perfect sense. I think I may have a way of avoiding the memory intensive
nature of the current SPDX tools implementation by having the setters
immediately store the information through an interface. I have a prototype
on my local machine I plan to publish to the goneall github repository once
I have it in a less embarrassing state. I hope you don't mind, but I'm
borrowing quite a few ideas from spdxextra. I'll need to make sure I give
you credit without implicating you in any of my own bad design choices ;)
My current challenge is how to implement anonymous nodes through an
interface and not have it end up duplicating nodes. I solved this in the
current implementation by searching for equivalent nodes (easy in RDF, hard
through an interface supporting a more generic storage model). I may be
over-engineering it - perhaps I should just allow the duplication. Since
the nodes are anonymous, the only practical issue I can think of is memory
and disk space.
BTW - I still plan on using RDF for my own usage, so I want to make sure
that performs well but allow for natural use by non RDF users.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/spdx/tools/issues/191?email_source=notifications&email_token=ABX3FBOOKHAXAZMUNMKJMI3PZFDJXA5CNFSM4HSAQJ52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXDTINY#issuecomment-499594295,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABX3FBMSGNGRLY3KEJ4WQPLPZFDJXANCNFSM4HSAQJ5Q
.

How would you have anonymous nodes in a purely SPDX model-centric view of the world?

The interface for storage I am thinking of would store properties associated with an id within a Document.   Originally I was thinking that all ID's would be either an SPDX element ID or an License ID.  After going through some implementation exercise, there are some objects (like Checksum) that do not have an SPDX ID or License ID.  So, I thought I would borrow a concept from RDF and have anonomous ID's that would only be valid within the SPDX document to support the interface.  I'm not sure yet if this approach will work, but so far it looks promising.

A file checksum could be an object of type Checksum that is a member of the
containing object (eg file).
Why would you need anonymous nodes in that model?
On Fri, Jun 7, 2019 at 6:00 AM goneall<EMAIL_ADDRESS>wrote:

How would you have anonymous nodes in a purely SPDX model-centric view of
the world?
The interface for storage I am thinking of would store properties
associated with an id within a Document. Originally I was thinking that
all ID's would be either an SPDX element ID or an License ID. After going
through some implementation exercise, there are some objects (like
Checksum) that do not have an SPDX ID or License ID. So, I thought I
would borrow a concept from RDF and have anonomous ID's that would only be
valid within the SPDX document to support the interface. I'm not sure yet
if this approach will work, but so far it looks promising.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/spdx/tools/issues/191?email_source=notifications&email_token=ABX3FBLVSE6A3ULUOSKC2NDPZHTQXA5CNFSM4HSAQJ52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXE3AKY#issuecomment-499757099,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABX3FBOWAGT2D76Q5FR4HA3PZHTQXANCNFSM4HSAQJ5Q
.

Why would you need anonymous nodes in that model?

To persist the information using a more generic storage interface.  I want to be able to implement Jena as one of the storage implementation but also support something as simple as a set of hash maps.
I am thinking of using ID's as a way to identify the properties needed to reconstitute the Java Object.  So, if an SPDX File contains an SPDX Checksum, I would like a way to refer to the checksum in a way that can be persisted.  For this, I am thinking of having a string ID for the checksum that can be stored in the property.
Properties would be read using something like Object readProperty(URI spdxDocument, String propertyName, String id, @Nullable Object transaction);
This works well for "public" SPDX classes that must contain a document unique ID.  The problem comes when we want to include a complex object (like a checksum) that doesn't necessarily have an ID.  For this, I was thinking of borrowing a technique from RDF an generate an ID which is only valid within the document and whose only purpose is to relate the property to the object.
Maybe there is a simpler/better interface to storage?
One test I'm applying to the interface is it must support in memory Java objects (perhaps implemented in HashMaps), RDF/Jena and a relational database.  I think if the interface supports all three, it should be general enough to fit many other storage models.

I started a prototype to experiment with the proposed design.
Here's the storage interface definition: https://github.com/goneall/Spdx-Java-Library/blob/master/src/main/java/org/spdx/storage/IModelStore.java
There is an abstract class to manage most of the translations between the Java objects and the underlying storage interface:  https://github.com/goneall/Spdx-Java-Library/blob/master/src/main/java/org/spdx/library/model/ModelObject.java
I've ported over the SPDX classes for the SPDX licenses to test out the design.  It seemed to hold up OK.  Maybe a bit more complex that I thought, but for users of the library interface it should look just like plain old Java objects.  This will make it easy to port over all the existing tools and helper classes.
The only dependencies so far are JSoup to translate HTML to text, log4j and Apache commons-lang.
@yevster @vlsi Please take a look and let me know if you have any improvement ideas or better approaches.
After some review, I'll try implementing a simple HashMap store supporting the interface.

@sschuberth Any thoughts on this thread?  As someone familiar with the libraries, I would be quite interested in your feedback on other issues we may solve if we're redesigning the SPDX tools and also your opinion on the proposed approach.
Here's a link to a Google doc summarizing the approach: SPDX Java Tools (re)Design.  The comment above has links to the start of a prototype design.

I am working generating Java code for the listed licenses.
I created a read-only interfaces to the listed licenses (ISpdxListedLicense.java) and listed exceptions(ISpdxListedException.java).
I plan on generating some Java code for each release of the SPDX licenses.
There are a few choices.  I could create a public static immutable map with the license ID as the key and an implementation of ISpdxListedLicense as the value.
Another approach would be to wrap the map in a class or an enum (similar to what was done in the spdxtra implementation of the license list).
Wrapping it in an enum would be more forward compatible if we want to change the underlying storage.
We could even be further abstract by creating an interface for accessing the licenses and exceptions.
@vlsi @yevster let me know if you have any preference on the approach.
BTW - implementing this will solve a very significant performance problem in the license matching code.  It currently takes several seconds to download and initialize the listed licenses from the JSON files.  I'm hoping this will trim that down to milliseconds.

I experimented with generating a static enum class with all licenses.  The static strings for some of the licenses caused problems with Eclipse (>32K characters in GPL).
The file ended up being about 7.4 MB - very large.
I am thinking that generating a static class isn't a very good approach.  The size and volume of the licenses may be better left to a database type storage.
@vlsi Let me know if you still think it may be useful to generate a static file.
I attached thegenerated file for reference.
SpdxListedLicenses.zip

@goneall , Just letting you know, I've implemented enum and expressions in Kotlin.
Of course putting all the texts to a classfile is a bad idea.
Texts should be split by files.
So far the implementation lives in https://github.com/vlsi/vlsi-release-plugins/tree/master/plugins/license-gather-plugin
Equivalence tests (e.g. GPL-2. look as follows: https://github.com/vlsi/vlsi-release-plugins/blob/master/plugins/license-gather-plugin/src/test/kotlin/com/github/vlsi/gradle/license/EquivalenceTest.kt#L56-L58
License interpretation look as follows: https://github.com/vlsi/vlsi-release-plugins/blob/master/plugins/stage-vote-release-plugin/src/test/kotlin/com/github/vlsi/gradle/release/Apache2LicenseInterpreterTest.kt#L31
The use in Gradle build scripts (to generate LICENSE file) looks as follows: https://github.com/vlsi/jmeter/blob/gradle/src/licenses/build.gradle.kts#L102-L110
The generated enum looks as follows (1500 lines):
enum class SpdxLicense private constructor(
  override val id: String,
  override val title: String,
  detailsUri: String,
  seeAlso: Array<String>
) : StandardLicense {
  `0BSD`("0BSD", "BSD Zero Clause License", "http://spdx.org/licenses/0BSD.json",
      arrayOf("http://landley.net/toybox/license.html")),

  AAL("AAL", "Attribution Assurance License", "http://spdx.org/licenses/AAL.json",
      arrayOf("https://opensource.org/licenses/attribution")),

...

  val detailsUri: URI = URI(detailsUri)

  override val uri: List<URI> = seeAlso.map { URI(it) }

  override val providerId: String
    get() = "SPDX"
  companion object {
    private val idToInstance: Map<String, SpdxLicense> = values().associateBy { it.id }

    fun fromId(id: String) = idToInstance.getValue(id)

    fun fromIdOrNull(id: String) = idToInstance[id]
  }
}

The new library is now available at: https://github.com/spdx/Spdx-Java-Library

@vlsi, I somehow missed your last comment so far, but what you've been doing looks quite similar to what we have in ORT's spdx-utils module, in particular the generated SpdxLicense and SpdxLicenseException classes. Maybe you're interested in joining forces on such a Kotlin library?

@sschuberth , it turns out spdx-utils is not published to OSSRH, which makes it hard to use as a library :-/ (see https://github.com/oss-review-toolkit/ort/issues/2906 )

It's actually not hard to use, you simply have to add JitPack as a Maven repository like we do e.g. in the Eclipse Antenna project, see
https://github.com/eclipse/antenna/blob/808732298969f2a4d4b86fcccf4903662e86432a/pom.xml#L38
https://github.com/eclipse/antenna/blob/808732298969f2a4d4b86fcccf4903662e86432a/pom.xml#L262-L266
https://github.com/eclipse/antenna/blob/808732298969f2a4d4b86fcccf4903662e86432a/pom.xml#L380-L383
But I agree having spdx-utils published to OSSRH / Maven Central or JCenter would be nicer.

I'm afraid you miss the point.
As I said, I implement a Gradle plugin, and I do not want the users of my plugin to add JitPack as a repository for their build scripts.

Can't you just declare the JitPack Maven repository in the plugin's deployed pom.xml?

I can't.
For security reasons, it is the client who should declare the full set of repositories used during a build.
Imagine what happens if somebody declares their own repository that contains fishy versions of log4j and friends? Do you think everybody should blindly use that repository?
Gradle ignores repository declarations in the fetched pom.xml files. I'm not sure if Maven already ignores repositories in the transitive pom files, however, I hope it would stop soon.
Here Cédric Champeau explains the issue of Maven using repositories from the transitive dependencies: https://www.youtube.com/watch?v=GWGNp3a3hpk&feature=youtu.be&t=2339


honor --no-open flag in floyd run

While writing some tests I learned that the --no-open flag for floyd run isn't being honored. This fixes it. @houqp @narenst

I now see that #115 already implements these changes. Closing for now, as I expect 115 to be merged. If it doesn't, we can re-open this.


create transfer()

In dvc (or dvc-data) we currently have a transfer() https://github.com/iterative/dvc/blob/main/dvc/data/transfer.py method that handles tranfering objects from one odb to another. We need a simpler version of that for dvc-objects that would not care about Trees (and probably their indexes), so that it could be used both in dvc-data and in other applications (e.g. in run-cache)
This will allow us to reduce the boilerplate and improve, test and benchmark this core functionality in a convenient way.

Fixed by https://github.com/iterative/dvc-objects/pull/64


Combine .bingo configuration files

Combine .bingo directory from the catalogd and operator-controller projects.

Linked to internal Jira story EET-4422.

Bingo files with issues  are shown below:
➜  operator-controller git:(main) diff --brief --recursive .bingo ../catalogd/.bingo
Files .bingo/controller-gen.mod and ../catalogd/.bingo/controller-gen.mod differ
Files .bingo/controller-gen.sum and ../catalogd/.bingo/controller-gen.sum differ
Only in ../catalogd/.bingo: crd-diff.mod
Only in ../catalogd/.bingo: crd-diff.sum
Only in .bingo: crd-ref-docs.mod
Only in .bingo: crd-ref-docs.sum
Only in ../catalogd/.bingo: ginkgo.mod
Only in ../catalogd/.bingo: ginkgo.sum
Files .bingo/golangci-lint.mod and ../catalogd/.bingo/golangci-lint.mod differ
Files .bingo/golangci-lint.sum and ../catalogd/.bingo/golangci-lint.sum differ
Files .bingo/goreleaser.mod and ../catalogd/.bingo/goreleaser.mod differ
Files .bingo/goreleaser.sum and ../catalogd/.bingo/goreleaser.sum differ
Files .bingo/kind.sum and ../catalogd/.bingo/kind.sum differ
Files .bingo/kustomize.mod and ../catalogd/.bingo/kustomize.mod differ
Files .bingo/kustomize.sum and ../catalogd/.bingo/kustomize.sum differ
Only in .bingo: operator-sdk.mod
Only in .bingo: operator-sdk.sum
Only in .bingo: opm.mod
Only in .bingo: opm.sum
Files .bingo/setup-envtest.mod and ../catalogd/.bingo/setup-envtest.mod differ
Files .bingo/setup-envtest.sum and ../catalogd/.bingo/setup-envtest.sum differ
Files .bingo/variables.env and ../catalogd/.bingo/variables.env differ
Files .bingo/Variables.mk and ../catalogd/.bingo/Variables.mk differ
➜  operator-controller git:(main)


[Linux, Intel x86_64]  RVar not defined in the sources but exists in Math.exe

Hello,
Thanks for your great idea, and sharing your code, much appreciated !
Since I'm using ImGui too, I'm trying to test it on Linux, and I think I'm very close ... except linking is no possible since RVar is undefined. Would you mind to define it or provide me some links to add / replace the missing definitions ?  (tested with -std=c++11 to 17 without any success)
My g++ command line:
g++-8 -I/usr/local/include -I. -I./imgui -I./MathExpr/MathExpr -I./imgui/examples/libs/gl3w -DLinux -DIMGUI_UNLIMITED_FRAME_RATE -D_NO_DEBUG_HEAP=1 -Wall -std=c++17 -o build/imgui_math ./imgui/imgui.cpp ./imgui/imgui_demo.cpp ./imgui/imgui_draw.cpp ./imgui/imgui_widgets.cpp ./imgui/imgui_impl_glfw.cpp ./imgui/imgui_impl_opengl3.cpp ./imgui/examples/libs/gl3w/GL/gl3w.c ./main.cpp -lGL -lGLU   -L./MathExpr -lmathexpr

The errors:
./main.cpp:40:1: error: ‘RVar’ does not name a type; did you mean ‘char’?
 RVar *xvar;
 ^~~~
 char
./main.cpp:41:1: error: ‘RVar’ does not name a type; did you mean ‘char’?
 RVar* vararray[1];
 ^~~~
 char
./main.cpp:42:1: error: ‘ROperation’ does not name a type
 ROperation *op;
 ^~~~~~~~~~
./main.cpp: In function ‘double Func(double)’:
./main.cpp:138:6: error: ‘op’ was not declared in this scope
  y = op->Val();
      ^~
./main.cpp: In function ‘void ElaborateFunction()’:
./main.cpp:147:2: error: ‘xvar’ was not declared in this scope
  xvar = new RVar("x", &funx);
  ^~~~
./main.cpp:147:2: note: suggested alternative: ‘char’
  xvar = new RVar("x", &funx);
  ^~~~
  char
./main.cpp:147:13: error: expected type-specifier before ‘RVar’
  xvar = new RVar("x", &funx);
             ^~~~
./main.cpp:148:2: error: ‘vararray’ was not declared in this scope
  vararray[0] = xvar;
  ^~~~~~~~
./main.cpp:148:2: note: suggested alternative: ‘va_arg’
  vararray[0] = xvar;
  ^~~~~~~~
  va_arg
./main.cpp:149:2: error: ‘op’ was not declared in this scope
  op = new ROperation(function, 1, vararray);
  ^~
./main.cpp:149:11: error: expected type-specifier before ‘ROperation’
  op = new ROperation(function, 1, vararray);
           ^~~~~~~~~~

Thanks in advance :-)
FYI, RVar in Math.exe (using nm) reveals a lot of functions using RVar symbols.
nm Math.exe | grep RVar
000000000047ed83 T _Z11IsolateVarsRPciPP4RVariPP9RFunction
000000000047ef61 T _Z14IsolateNumbersRPciPP4RVariPP9RFunction
000000000047e967 T _Z5IsVarPKciiPP4RVar
0000000000480e48 T _ZeqRK4RVarS1_
000000000047f130 T _ZN10ROperationC1EPciPP4RVariPP9RFunction
000000000047caa4 T _ZN10ROperationC1ERK4RVar
000000000047f130 T _ZN10ROperationC2EPciPP4RVariPP9RFunction
000000000047caa4 T _ZN10ROperationC2ERK4RVar
000000000047bb70 T _ZN4RVarC1EPKcPd
000000000047bb24 T _ZN4RVarC1ERKS_
000000000047bb70 T _ZN4RVarC2EPKcPd
000000000047bb24 T _ZN4RVarC2ERKS_
000000000047bbac T _ZN4RVarD1Ev
000000000047bbac T _ZN4RVarD2Ev
000000000047c05c T _ZN9RFunctionC1ERK10ROperationiPP4RVar
000000000047bf94 T _ZN9RFunctionC1ERK10ROperationP4RVar
000000000047c05c T _ZN9RFunctionC2ERK10ROperationiPP4RVar
000000000047bf94 T _ZN9RFunctionC2ERK10ROperationP4RVar
0000000000481016 T _ZNK10ROperation10ContainVarERK4RVar
00000000004817e4 T _ZNK10ROperation10SubstituteERK4RVarRKS_
000000000048198a T _ZNK10ROperation4DiffERK4RVar

Hi.
I'm really happy to see your interest.
May be too late but the code is old and i lost the command.
I was very inexperienced and i made few errors. The code also doesnt look good: its missing comments and explanations, it may contains memory leaks and similar.
If you're still interested in this project, i can consider searching for the command to build it and take into account a big refactor.
Sorry for the 1-year delay and thank your for your kindness.
Davide

Btw, your linking error comes from the mathexp library. Try using this http://www.yann-ollivier.org/mathlib/mathexpr. Download and build with linking options wrote inside readme.md.


Some improvements

LGTM.
Thanks for the PR!
I will clean up Console part later today and publish updated version.


Storybook changes

[x] Migrate to storybook v8
[x] Replace JS/JSX w/ TS/TSX

Resolved w/ PR #2


Add support for clang-tidy-17 and cmake 3.27

@ZedThree Hi, I thought I could add support for clang-tidy-17 by adding the official LLVM repository to apt and do the same with cmake by adding its official repository.
My solution mostly follows the official instructions, but is slightly adopted for Docker. See LLVM and Kitware respectively.

@ZedThree Any thoughts on this?

I am a little bit wary of this, because I'd prefer to keep the base image as a LTS OS if possible, as that would give the widest compatibility.
We could install the latest CMake via pip, that would remove the dependency on the Kitware repo. Is there an LLVM repo for Ubuntu 23.04?
Ideally, we'd somehow either remove the docker image entirely or make it customisable. I rather suspect the only way to do that would be to rewrite the whole thing in javascript.

I'd prefer to keep the base image as a LTS OS if possible

But Ubuntu 23.04 is not an LTS OS, right? I still agree with the idea though.

@ZedThree I am not sure if I understand what you mean.

I'd prefer to keep the base image as a LTS OS if possible, as that would give the widest compatibility

Widest compatibility with what? This is a GitHub Actions Docker image and will thus run on the Docker engine on the GitHub servers. If the engine is able to create a container from the image, you have already achieved the maximum compatibility for the purpose.

We could install the latest CMake via pip

Ok, sure, but if you are already using a ubuntu base image, which includes apt, why not just use apt? In fact, pip is even installed manually on top of the base image.

Is there an LLVM repo for Ubuntu 23.04

You mean an APT source? Yeah, it's what I added in my commit. Also see the instructions in the link I provided in my first comment.

Ideally, we'd somehow either remove the docker image entirely or make it customisable.

I am not sure how you plan to install and run Clang/LLVM and related tools during a GitHub Actions run without Docker.