上一篇是基礎的數據輸入,然而繼續查看模型的輸入數據會遇到各種數據結構,比如迭代器yield
本文將繼續尋根究底,看看是如何輸入audio及rgb 特徵,然後得到NeXtVLAD聚類後的特徵。
我看了下模型的輸出,似乎只能輸出類別,而我想得到某一層的輸出結果。這個咋整呢??
屁大點事結果我麻煩了百度的很多人幫我解決,真是辛苦了,從此再也不說飛漿的壞話了,哈哈。
我覺得殺雞就得用牛刀,快!自己瞎摸索半天甚至更久真的是浪費時間。
以後有問題我還是會讓全世界的人都知道,感謝所有幫助我的大佬。
下面直接看模型的該層輸出,以及模型的加載方法。難道說沒有GPU就不行嗎???
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 783, in run
six.reraise(*sys.exc_info())
File "D:\python36\lib\site-packages\six.py", line 703, in reraise
raise value
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 778, in run
use_program_cache=use_program_cache)
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 831, in _run_impl
use_program_cache=use_program_cache)
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 851, in _run_program
assert len(feed) == 1, "Not compiled with data parallel"
AssertionError: Not compiled with data parallel
但我在服務器用GPU也是得到同樣的結果,臥槽,這是爲啥子,後來一想沒有變成飛漿的tensor,我改成tensor呢?
改成了飛漿的tensor也不行,必須用下面的格式輸入,臥槽,這就奇怪了。下面老子解剖下這個函數到底是啥玩意。
feed=infer_feeder.feed(data_feed_in))
解讀下面的我以爲是套個列表即可:
>>> for infer_iter, data in enumerate(infer_reader()):
print(infer_iter,data.shape)
Traceback (most recent call last):
File "<pyshell#2>", line 2, in <module>
print(infer_iter,data.shape)
AttributeError: 'list' object has no attribute 'shape'
>>> for infer_iter, data in enumerate(infer_reader()):
print(infer_iter,len(data))
0 1
>>> for infer_iter, data in enumerate(infer_reader()):
print(infer_iter,len(data[0]))
0 2
>>> for infer_iter, data in enumerate(infer_reader()):
print(infer_iter,data[0][0].shape[0])
0 3
>>> for infer_iter, data in enumerate(infer_reader()):
print(infer_iter,data[0][0].shape)
0 (3, 1024)
>>> for infer_iter, data in enumerate(infer_reader()):
print(infer_iter,data[0][1].shape)
0 (3, 128)
>>> data_feed_in = [items for items in data]
>>> data_feed_in==data
True
後來發現還是有infer_feeder的問題:
Warning (from warnings module):
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 782
"The following exception is not an EOF exception.")
UserWarning: The following exception is not an EOF exception.
Traceback (most recent call last):
File "D:/python36/new/rcdnn/image/my_predict_2.py", line 123, in <module>
feed=[[image_feature,audio_feature]])#feed=infer_feeder.feed(data_feed_in))
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 783, in run
six.reraise(*sys.exc_info())
File "D:\python36\lib\site-packages\six.py", line 703, in reraise
raise value
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 778, in run
use_program_cache=use_program_cache)
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 831, in _run_impl
use_program_cache=use_program_cache)
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 857, in _run_program
(type(feed)))
TypeError: feed requires dict as its Parameter. But you passed in <class 'list'>
人家說要dict,我給的列表不對,哦對啊,應該是字典纔對,試試
>>> infer_feeder.feed(data)
{'test_rgb': <paddle.fluid.core_avx.LoDTensor object at 0x000002D8B6262068>, 'test_audio': <paddle.fluid.core_avx.LoDTensor object at 0x000002D8B625DFB8>}
但下面還是不對,我去,非要用infer_feeder啊??
Warning (from warnings module):
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 782
"The following exception is not an EOF exception.")
UserWarning: The following exception is not an EOF exception.
Traceback (most recent call last):
File "D:/python36/new/rcdnn/image/my_predict_2.py", line 123, in <module>
feed={'test_rgb':image_feature,'test_audio':audio_feature})#feed=infer_feeder.feed(data_feed_in))
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 783, in run
six.reraise(*sys.exc_info())
File "D:\python36\lib\site-packages\six.py", line 703, in reraise
raise value
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 778, in run
use_program_cache=use_program_cache)
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 831, in _run_impl
use_program_cache=use_program_cache)
File "D:\python36\lib\site-packages\paddle\fluid\executor.py", line 905, in _run_program
fetch_var_name)
paddle.fluid.core_avx.EnforceNotMet:
--------------------------------------------
C++ Call Stacks (More useful to developers):
--------------------------------------------
Windows not support stack backtrace yet.
------------------------------------------
Python Call Stacks (More useful to users):
------------------------------------------
File "D:\python36\lib\site-packages\paddle\fluid\framework.py", line 2525, in append_op
attrs=kwargs.get("attrs", None))
File "D:\python36\lib\site-packages\paddle\fluid\layer_helper.py", line 43, in append_op
return self.main_program.current_block().append_op(*args, **kwargs)
File "D:\python36\lib\site-packages\paddle\fluid\layers\sequence_lod.py", line 350, in sequence_pool
"pad_value": pad_value
File "D:/python36/new/rcdnn/image\models\nextvlad\nextvlad_model.py", line 76, in forward
a_sum = fluid.layers.sequence_pool(activation, 'sum')
File "D:/python36/new/rcdnn/image\models\nextvlad\nextvlad_model.py", line 162, in create_model
vlad_video = video_nextvlad.forward(video_input)
File "D:/python36/new/rcdnn/image\models\nextvlad\nextvlad.py", line 117, in build_model
rgb, audio, is_training=(self.mode == 'train'), **model_args)
File "D:/python36/new/rcdnn/image/my_predict_2.py", line 84, in <module>
infer_model.build_model()
File "D:\python36\lib\idlelib\run.py", line 474, in runcode
exec(code, self.locals)
File "D:\python36\lib\idlelib\run.py", line 144, in main
ret = method(*args, **kwargs)
File "<string>", line 1, in <module>
----------------------
Error Message Summary:
----------------------
InvalidArgumentError: Input(X) Tensor of SequencePoolOp does not contain LoD information.
[Hint: Expected lod_level > 0, but received lod_level:0 <= 0:0.] at (D:\1.7.1\paddle\paddle/fluid/operators/sequence_ops/sequence_pool_op.h:43)
[operator < sequence_pool > error]
加個[]也是不行,
RuntimeError: Some of your feed data hold LoD information.
They can not be completely cast from a list of Python
ndarray to LoDTensor. Please convert data to LoDTensor
directly before feeding the data.
人家說了要改成飛漿的tensor,我試試numpy咋改成LoDTensor??結果仍舊不行,百度的大佬說yield返回numpy即可。
但是寡人看了即使有沒有yield不重要,但必須有data_feeder,這是標準化的投餵數據,不能少,這一點還沒tf的方便。
官方例子:
| import numpy as np
| import paddle.fluid as fluid
|
| def generate_reader(batch_size, base=0, factor=1):
| def _reader():
| for i in range(batch_size):
| yield np.ones([4]) * factor + base, np.ones([4]) * factor + base + 5
| return _reader()
|
| x = fluid.data(name='x', shape=[None, 2, 2])
| y = fluid.data(name='y', shape=[None, 2, 2], dtype='float32')
|
| z = fluid.layers.elementwise_add(x, y)
|
| feeder = fluid.DataFeeder(['x','y'], fluid.CPUPlace())
| place_num = 2
| places = [fluid.CPUPlace() for x in range(place_num)]
| data = []
| exe = fluid.Executor(fluid.CPUPlace())
| exe.run(fluid.default_startup_program())
| program = fluid.CompiledProgram(fluid.default_main_program()).with_data_parallel(places=places)
|
| # print sample feed_parallel r result
| # for item in list(feeder.feed_parallel([generate_reader(5, 0, 1), generate_reader(3, 10, 2)], 2)):
| # print(item['x'])
| # print(item['y'])
|
| reader_list = [generate_reader(5, 0, 1), generate_reader(3, 10, 2)]
| res = exe.run(program=program, feed=list(feeder.feed_parallel(reader_list, 2)), fetch_list=[z])
| print(res)
因此我也不改了,就這樣吧,準備寫文檔。
另外有相關問題可以加入QQ羣討論,不設微信羣
QQ羣:868373192
語音圖像視頻深度-學習羣